Yahoo and their mail filters..

Wed Feb 25 21:05:30 UTC 2009

Outbound filtering is a good idea..however after investing lots of money on hardware appliances (old company $100,000 on equipment to do just this...) you realize you have more issues then solutions. Now you allow forwarded mail, and as you stated most systems accept the messages into the queue process the message and then either bounce/quarentine/allow. You can't bounce the message because it goes back to the sender which is almost always spoofed and thus you create backscatter. You cant quarentine because then you may flag some of your customers legitimate email.

Isolating your forwarded mail to a separate ip address is really, I think, the best way to handel forwarded mail.

-r

-----Original Message-----
From: Brian Keefer [mailto:chort at smtps.net] 
Sent: Wednesday, February 25, 2009 3:48 PM
To: Micheal Patterson
Cc: nanog at nanog.org
Subject: Re: Yahoo and their mail filters..

On Feb 24, 2009, at 6:27 PM, Micheal Patterson wrote:

> This may be old news, but I've not been in the list for quite some 
> time. At any rate, is anyone else having issues with Yahoo blocking / 
> deferring legitimate emails?
>
> My situation is that I host our corporate mx'ers on my network, one of 
> the companies that we recently purchased has Yahoo hosting their 
> domains mail. Mail traffic to them is getting temporarily deferred 
> with the "421 4.7.0 [TS01] Messages from xxx.xxx.xxx.xxx temporarily 
> deferred due to user complaints - 4.16.55.1; see 
> http://postmaster.yahoo.com/421-ts01.html"
>
> The admin of the facility has contacted Yahoo about this but their 
> response was for "more information" when they were told that traffic 
> from my mx to their domain was to being deferred.  I may end up just 
> having them migrate to my systems just to maintain company 
> communications if we can't clear this up in a timely manner.
>
> --
> Micheal Patterson

A few comments on this thread in general (speaking only for myself, not in any way representing my employer)...

Yes, Yahoo! tend to throttle IPs at the drop of a hat, but those blocks are often gone in a few hours as well.  Others have pointed out some procedures to follow to minimize the possibility of being  
blocked.  At least they give you a useable SMTP error (usually).   
Incidentally this is why all my test accounts are on Gmail, because delivery to Yahoo! is often deferred for minutes to hours.  Of course, given the recent Gmail outages I might have to diversify even more...

As for "blackholes" that messages fall into, what is the alternative?   
You could say reject it in session with a readable error, but that would give spammers instant confirmation on whether their campaign is working.  Also, the majority of anti-spam products I've seen have to spool the message before they scan it, so rejecting in session is simply not an option on a lot of commercial platforms.

The other options is to stuff all the spam messages in a folder and expose them to the user, taking up a huge amount of storage space for something the vast majority of users are never going to look at any way.  Again, a lot of commercial solutions have a scoring methodology where you can be pretty certain stuff at the top end of the scale is virtually never going to be a false positive.  The amount of savings in not having to handle and store that crud massively outweighs one or two users missing a newsletter once in a while.  It can make sense to expose the "mid-range spam" to users and let them decide, but why store terabytes of stuff that only a tiny fraction of the users may ever care about?

If you're sending important mail that's not reaching the recipient, and you have the server logs to prove you handed it off to the destination MTA, open a ticket with them and they'll have logs to track it down.

Regarding taking automatic action based on luser feedback, that is ridiculous in my opinion.  From the data I see, the lusers classify mail incorrectly far more than correctly.  In fact there's a running joke around here that we should simply flip the false-positive and false-negative feeds and enable auto-train, since the only thing you can reliably count on users to do is get things wrong.  Submissions from administrators are _far_ more accurate (although even then, not to the point that it always makes sense to take automatic action).

Blocking an entire site just because one John Doe user clicked a button they don't even understand just does not make sense.

Last, anywhere that I've seen extensive use of forwards has had a maze  
of difficult to untangle abuse problems related to forwarded spam.   
Any site allowing forwarding should apply very robust filtering of outbound mail.

--
bk