the O(N^2) problem
jgreco at ns.sol.net
Mon Apr 14 13:51:26 UTC 2008
> The risk in a reputation system is collusion.
/One/ risk in a reputation system is collusion.
Reputation is a method to try to divine legitimacy of mail based on factors
other than whether or not a recipient authorized a sender to send mail. To
a large extent, the majority of the focus on fighting spam has been to try
to do this sort of divination by coding clever things into machines, but it
should be clear to anyone who has ever had legitimate mail mysteriously go
missing, undelivered, or delayed that the process isn't without the
There are both positive (whitelist) and negative (DNSBL, local This-Is-Spam,
etc) reputation lists, and there are pros and cons to each.
Consider, for example, Kevin Day's example of the Group-B-Objectionable
scenario. This is a nonobvious issue that can subvert the reputation of
a legitimate mailer.
On the flip side, what about someone who actually wants to receive mail
that an organization such as Spamhaus has deemed to be hosted on a spammy
IP? (And, Steve and the Spamhaus guys, this is in no way a criticism of
the job you guys do, the Internet owes you a debt of gratitude for doing
a nearly impossible job in such a professional manner)
There are risks inherent with having any third party, specifically
including the ISP or mailbox provider, trying to determine the nature of
the communications, and filtering on that basis.
This is why I've been talking about paradigms that eliminate the need for
third parties to do analysis of e-mail, and rely on the third parties to
simply implement systems that allow the recipient to control mail. There
are a number of such systems that are possible.
However, the current systems of divining legitimacy (reputation, filtering,
whatever) generate results that loosely approximate the typical mail that
the average user would wish to receive. Users have been trained to consider
errors in the process as acceptable, and even unavoidable.
It's ridiculous when systems like Hotmail silently bitbucket e-mail from
a sender (and IP) that has never spammed, and have ONLY sent transactional
e-mail and customer support correspondence, and the individually composed
non-HTML REPLIES to customer inquiries are eaten by Hotmail, or tossed in
the spam folder. Nice. (I know, we all have our stories)
Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net
"We call it the 'one bite at the apple' rule. Give me one chance [and] then I
won't contact you again." - Direct Marketing Ass'n position on e-mail spam(CNN)
With 24 million small businesses in the US alone, that's way too many apples.
More information about the NANOG