What must one do to avoid Gmail's retarded non-spam filtering?

Tue Sep 28 23:16:14 UTC 2010

Bill,

Thanks for the response and excellent ideas.

The OSS/BSS box is in our internal network with a RFC1918 address and it relays to outside via another box which sits in the DMZ and which is in SPF records for the domain that appears in From/Return-Path, etc. 

That DMZ machine does not appear on RBLs nor is it whitelisted anywhere. 

Google adds headers like these:
Received-SPF: pass ...
Authentication-Results: mx.google.com; spf=pass ...

So the problem is unlikely to be a SPF issue, as mentioned in my first e-mail.

We haven't sent any marketing mail from the same IP in months, but I believe that we did previously. We do send out once every few months a network maintenance notice with identical content for all recipients, except their name and To header. 

Clocks are set correctly everywhere via NTP and Received headers in the messages are >= each of the previous Received headers as you go up.

We do use ALL CAPS in some subjects (rare) but more commonly, some customer names and other data in the body does appear in caps. We also have a www.caneris.com link on most e-mails (removing it doesn't change gmail's behaviour). Messages without anything ALL CAPS in them end up in spam as well.

I created a new mailbox under a Google Apps domain and sent it one of the typical messages. It went into spam. I also ran the same message through SpamAssassin as per your suggestion and it came out clean.

A test message sent directly from the same edge MTA but with a From address in a different domain is not delivered to spam, even though Return-Path there is still an @caneris.com address (but different than our usual one). A test message sent from our corporate mail system (Zimbra box in internal network relaying to the same edge MTA as the one used by the OSS/BSS box) goes to spam.

I've received several off-list responses, including those from Google Apps users, asking for sample messages, so will be forwarding those.

Ideas:
-Message-Id @ "non-existant host" (ones in our internal network)
-Received headers showing that the message was relayed, and again from a "non-existant host"

Unlikely to be an IP repuation issue given the one message above did go through fine.

Erik

----- Original Message -----
From: "William Herrin" <bill at herrin.us>
To: "Erik L" <erik_list at caneris.com>
Cc: nanog at nanog.org
Sent: Tuesday, September 28, 2010 4:44:24 PM
Subject: Re: What must one do to avoid Gmail's retarded non-spam filtering?

On Tue, Sep 28, 2010 at 4:15 PM, Erik L <erik_list at caneris.com> wrote:
> An increasingly large number of our customers are using
> Gmail or Google Apps and almost all of our OSS/BSS mail
> is getting spam filtered by Google. Among others, these
> e-mails include invoices, order confirmations, payment
> notifications, customer portal logins, and tickets. Almost
> anything we send to customers on Google ends up in their
> spam folder.

Erik,

Do you send marketing information to your customers from the same IP
addresses as you send your invoices?

Have you checked the source IPs against the RBLs with, e.g.
http://rbls.org/? What about the source IP of the specific machine the
originates the invoices et. al?

Is the invoicing machine also in SPF? One of the companies I do
business with accidentally excluded their billing mailer from the SPF
record.

Have you sent copies of the various types of documents to a mail
server running Spam Assassin in order to see which qualms Spam
Assassin flags? Careless mistakes with the headers or the clock can
get your message flagged as spam. Even something as simple as ALL CAPS
can get you flagged.

Have you added your invoicing mailer, the one you can guarantee sends
no marketing materials, in to the whitelist at http://www.dnswl.org/?

Regards,
Bill Herrin

-- William D. Herrin ................ herrin at dirtside.com bill at herrin.us
3005 Crane Dr. ...................... Web: <http://bill.herrin.us/>
Falls Church, VA 22042-3004