Unable to email anyone from my primary domain name; thanks Google Mail and G Suite.

Thu Oct 24 16:32:21 UTC 2019

This was recently thrashed out in the MailOp mailing list, https://chilli.nosignal.org/cgi-bin/mailman/listinfo/mailop.

I can point you to the beginning on the email chain, but it became a mess - https://chilli.nosignal.org/cgi-bin/mailman/private/mailop/2019-October/014854.html (And yes, please ignore the bad certificate…..)

The link is private, so you'll have to sign up first.  The 'google' insights come from the author Brandon Long.

From Google's PoV, there wasn't enough good signals from his emails being sent and his messages being acted upon within his mailbox for Google to determine it legitimate. Google watches everything, from the moment the email touches their edge until the email is ultimately disposed. Everything in-between provides signals to Google if that email was something that the receiver wanted. Think of things like never reading the email and auto-filtering it (tagging, I guess for gmail), for example, are negative signals.

I suspect that by changing your 5321.MailFrom, you changed the signal calculus, for now. I bet in a bit, provided that you don’t change any other behaviors, that these emails will eventually be rejected too.

This is done by all the big players, but Microsoft is the most aggressive.

The recommendations for this guy were to have his email relayed through a bigger company before heading out to Google, especially if he wasn't going to dump OVH. For example, OVH maintains email relays that have way better reputations than the space used by customers (this was a recommendation by someone else, I have no clue personally).

Also, free is free. If you pay for G-Suite, then the admin gets a LOT of extra bonuses that anyone would expect out of a paid mailbox. I don’t know about G-Suites (wouldn’t touch the stuff personally), but you can get a O365-hosted exchange mailbox for like $5/month these days with all the aliases you need and all the post-processing transport rules you want. In line with the paid Microsoft mailbox – an email does not get delivered for no reason except in the rarest of cases. The same is not true with the free mailboxes hosted by Microsoft or Google.

And if having an account opened with one of the “big guys” ruffles your feathers the wrong way, try a smaller paid provider. I’ve used Hushmail personally for the last 8-10 years.

Some other pieces of insight provided by Brandon in that thread that are counter-points to your suggestions –

A lot of email that they tag as spam or straight-out block in the same way yours are now is due to 419 scams and spearfishing. Both are low-volume, may not contain links and can be sent text, so they share many of the same signals that your emails do today or that you propose as getting a free pass.

TL;DR for this guy - his domain is a free one off of eu.org, he's coming out of OVH network space and he sends a few messages a day, so the OVH signaling eventually won. As a paraphrase from that thread, 1 out of 10000 messages aren't spam from that netblock in Google’s eyes. And sign-up for the MailOp list if you want to discuss this further in a better forum.

Good Luck!

From: NANOG <nanog-bounces at nanog.org> On Behalf Of Constantine A. Murenin
Sent: Wednesday, October 23, 2019 7:19 PM
To: North American Network Operators' Group <nanog at nanog.org>
Cc: Constantine A. Murenin <cnst at netbsd.org>
Subject: Unable to email anyone from my primary domain name; thanks Google Mail and G Suite.

 External Mail

Dear NANOG@,

I'm not sure where else to post this, and this is not really new, either, but I think I have a new take here.

I use my own personal domain name for various UNIX stuff, including sending log-related things to myself out of cron, which end up in my own Gmail.com account, either directly, or through forwarding (w/o SRS).  (I do not use G Suite for my own domain name, for obvious reasons; just the consumer-based gmail.com<http://gmail.com> email address from the old times of invitation-based registrations.)

Over the years, I sometimes had certain messages rejected by Gmail, but it was a very low rate of rejection (less than 5% for any mail I cared about), and wasn't a major problem (usually only some automated messages would be rejected).

A couple of months ago, I setup some new scripts that would send me new nightly emails.  It's all plain text, but had a few dozen of domain names present (it's logs).  Absolutely no links, just plenty of domains which I don't control.  So, Gmail has been presenting most of these messages with their red warning label that the email contains malicious links, even though all of these emails contained zero links, zero URLs to any of these unknown domain names, zero URL schemes, zero "http://", zero "https://" etc.  You get the idea.

Since about a few weeks ago, I am now seeing at least a 95% rejection rate for my domain name, for ALL email, including the forwards.  Including emails which I send to myself from within Google, and which get forwarded back to Gmail by my UNIX machine (which is not known to break Gmail's DKIM, either, although it's also difficult to test, because when it does get through, it's automatically marked as a duplicate by Gmail, so, you don't get DKIM status from Gmail by looking at the headers, since you only get to examine the original copy that was sent, not the forwarded duplicate that was subsequently accepted).  I.e., emails with a passing DMARC still get rejected.

The funny thing is, Google doesn't actually blacklist my primary IPv6 address in my own /48 from which all of my messages originate; even though the rDNS resolves to a subdomain on the very same domain name which they've blacklisted "due to the very low reputation".  They've blacklisted just the main domain name that I use for my own non-Gmail-hosted mail.  Sending the same messages into my Gmail.com from a different domain name in MAIL FROM, which is served from the same zone file DNS-wise (e.g., an SPF pass), through sendmail's `-f` option, or with Mutt, makes the messages go through (even with rDNS being "low reputation"); sending it from my primary domain name in MAIL FROM results in the following:

>>> DATA
<<< 550-5.7.1 [2001:470:xxxx::      19] Our system has detected that this message is
<<< 550-5.7.1 likely suspicious due to the very low reputation of the sending
<<< 550-5.7.1 domain. To best protect our users from spam, the message has been
<<< 550-5.7.1 blocked. Please visit
<<< 550 5.7.1  https://support.google.com/mail/answer/188131 for more information. 135si403977wma.43 - gsmtp
554 5.0.0 Service unavailable

The support article suggests using Postmaster Tools; great, never heard of it, sounds cool; let's verify our domain, and try it out, hopefully, there's a solution right there.

However, after verifying my domain name through DNS for Postmaster Tools, it is revealed that Postmaster Tools cannot tell me anything at all, with all tabs and screens being 100% blank, allegedly because I'm not actually a mass email sender (I don't send hundreds of emails a day or whatnot), and they're too afraid that I'll figure out why my mail doesn't actually go through, instead of signing up for G Suite.

Right now, I've had a business need to reply to a work-related email from some other business.

This is what I got after sending my reply from my primary domain name through mutt — a nice double rejection by both the G Suite and Gmail in a single bounce generated by my server:

   ----- Transcript of session follows -----
... while talking to aspmx.l.google.com<http://aspmx.l.google.com>.:
>>> DATA
<<< 550-5.7.1 [2001:470:xxxx::      19] Our system has detected that this message is
<<< 550-5.7.1 likely suspicious due to the very low reputation of the sending
<<< 550-5.7.1 domain. To best protect our users from spam, the message has been
<<< 550-5.7.1 blocked. Please visit
<<< 550 5.7.1  https://support.google.com/mail/answer/188131 for more information. z11si12494671wrw.137 - gsmtp
554 5.0.0 Service unavailable
... while talking to gmail-smtp-in.l.google.com<http://gmail-smtp-in.l.google.com>.:
>>> DATA
<<< 550-5.7.1 [2001:470:xxxx::      19] Our system has detected that this message is
<<< 550-5.7.1 likely suspicious due to the very low reputation of the sending
<<< 550-5.7.1 domain. To best protect our users from spam, the message has been
<<< 550-5.7.1 blocked. Please visit
<<< 550 5.7.1  https://support.google.com/mail/answer/188131 for more information. 135si403977wma.43 - gsmtp
554 5.0.0 Service unavailable

Changing MAIL FROM into a non-primary domain name (served out of an identical zone file, basically) gets the message accepted, without DKIM, without the 4-minute delay that many "suspicious" messages have had for months now, from the very same IPv6 address with the rDNS pointing to the domain name with "the very low reputation", and it shows up in both my own Gmail as well as, presumably, in the G Suite account of the business partner I was replying to.  (Note that this trick where the rDNS domain gets ignored works only for new emails with a passing SPF; I presume the rDNS still prevails in bringing the "low reputation of the sending domain" for forwards, as they don't seem to succeed any longer now.)

There are a number of possible tl;dr: takeaways here:

* don't spread the monoculture — don't use G Suite for your organisation;

* don't send crontab output to your Gmail from your primary domain name;

* don't use Gmail.

What is my own takeaway here?

* Without being an anti-Google zealot, from a purely practical perspective, since my Gmail account no longer contains the mail I care most about, as it gets rejected on the SMTP layer, I'll have fewer reasons to use it.

* I'll now have no other choice but to modify my setup to stop forwarding to Gmail, because my business contacts don't need to see all these bounces that are now taking place.

* Since so many businesses are G Suite useds, I'm still looking for a solution to get rid of the "the very low reputation of the sending domain" from a domain name I've been using since 2007, and which I'm 100% convinced was blacklisted by Google entirely for me sending crontab with system logs (zero HTML, zero URLs) to my own Gmail.  I have SPF and DMARC all setup and passing since years ago, which doesn't stop this issue from occurring.  Merely switching the name of the domain in From and MAIL FROM to any other domain I own (which points to the very same MX) appears to be my workaround for now.

Some possible suggestions for Google, if I may:

* Maybe don't convert schemeless domain names which are non-URLs into "malicious" URLs?  (They already do seem to block them from being presented as links in the UI in such an instance, but there's little reason for trying to convert these domain names into links in the first place.)

* Maybe don't consider harmless plain text emails with a bunch of domain names to contain malicious links when they don't?

* Maybe don't assume everyone with a domain name runs a G Suite?  (Their whole troubleshooting guide is built around it.)

* Maybe don't assume everyone with a domain name sends hundreds of emails from their domain name per day?  (They seem to limit Postmaster Tools based on such a criterion.)

* Maybe don't blacklist a domain name for sending harmless logs to a Gmail account that lists said domain name as an alternative From address?

Cheers,
Constantine.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20191024/509e0ca1/attachment.html>