Email Server and DNS

Fri Nov 8 12:37:40 UTC 2013

I suggest moving this to mailop, where it arguably belongs.  But I'm
going to follow up on a few points, anyway.

First, I forgot to mention two other highly effective mail system
defense methods: geoblocking and passive OS fingerprinting.

Geoblocking: A mail server for a local construction business in Arizona
is unlikely to require mail from Poland, Peru or Pakistan.  So there's
no reason to go with a default-permit model: use default-deny and
only allow mail from places where legitimate mail might originate.
(In this case, perhaps: the US, Mexico, and Canada.)  Use the ranges
from ipdeny.com.  This will stop an astonishing amount of spam (and
other SMTP-borne abuse) cold.  And it can be done at the MTA or in
the firewall: which is better depends on circumstances.

Obviously this doesn't work for everyone.  Obviously this (like
everything else) runs the risk of false positives -- but it's easy
to mitigate that.  Obviously it does require understanding the
patterns in your mail traffic, but any competent mail system admin
has long since performed detailed statistical analysis and has a
pretty good idea what the characteristics of their incoming mail
stream look like.

Passive OS fingerprinting: regard anything originating from an OS
that fingerprints as Windows as dubious, at best.  Possible actions
vary: graylisting (more precisely, graylisting regardless of previous
traffic) is one good option.  Utilizing this in concert with geoblocking
(above) works beautifully, e.g., "I'm in Arizona and something in Portugal
that fingerprints as Windows is trying to send me SMTP traffic: the
probability approaches unity that this is spam".  When combined with
rDNS information, this becomes a highly efficient mechanism with a FP
rate that's ridiculously low.  (In other words, if that same system
has rDNS that looks like 123-45-67-8.example.com then it either really
is a bot or it's a mail system run by someone with no clue.)

A few short points and one long one in response:

On Sun, Nov 03, 2013 at 12:00:23PM -0600, Jimmy Hess wrote:
> The RFC contains a MUST NOT  in regards to verifying the  HELO name matches.
> So, the HELO can use any hostname ---  as long as the hostname forward
> resolves to something;  it should resolve to the IP address of one of your
> mail servers.    Some mail servers provide service for many domains, and
> have many DNS names that could be placed in a HELO.

All true.  But none of this argues against using the canonical hostname
in the HELO.  It's the simplest, easiest option (and quite often the
one that software will pick by default).

> > SPF is worthless crap: don't bother.  Use a real MTA, e.g., postfix
> >
> 
> I do not believe that is the consensus of the community -- or the working
> groups behind the SPF-related RFCs.

I'm well aware it's not the consensus.  It's my opinion.

Clue #1 that SPF is crap should have been its grandiose self-promoting
announcement ("Spam as a technical problem is solved by SPF").  Clue #2
should have been the observation that -- by far -- the most prolific
early adopters were spammers.  When your enemy latches on to your
new weapon much faster than you do, that should be a tipoff that maybe it's
not what you hope it is.  Clue #3 is available to anyone who deploys a
sufficiently large and diverse set of spamtraps for several years and
analyzes the data that arrives: SPF presence/absence or contents have
no statistically useful anti-spam value in a properly-designed mail
defense architecture.  

Don't believe me?  Okay.  Fine.  Set up a few thousand spamtraps, gather
data for 3-5 years, see for yourself.

So yes, it's standardized; so what?  So is (sort of) DNS forgery, see
http://tools.ietf.org/html/draft-livingood-dns-redirect-03 for example.
That's also crap, it just happens to be well-documented crap.

So: if you feel you must use something, use DKIM, which I think shows
vastly more promise.  Just don't expect it to be a panacea, because
the current miserable state of security at *all* levels undercuts it
badly.  Not DKIM's fault, really, but it does impact its usefulness
in the real world.

> Message quarantines are great;  they are helpful  for mitigating the false
> positives of overly-agressive filters.

This one I'll spend more time on.  Quarantines are a worst practice 
in mail systems engineering.  Here are some assorted reasons why, briefly:

- One of the fundamental principles of mail system defense is that you
should make your mistakes early, consistently, and loudly.  (And you
WILL make mistakes.   Everyone does.)  The point of doing this is that
it enables all concerned, including you, to have a decent chance of
noticing them, figuring out that they're mistakes, and correcting them.
Quarantines hide, defer, and silence your mistakes, making it much more
difficult to run your system properly.  They're a lazy admin's coverup,
not a fix.

- Quarantines present deadlock problems.  Mail from user A to user B
which is quarantined and causes B to eventually send a message to user A
enquiring about the missing missive stands a decent chance of ending up
in B's quarantine.  The end result is that human time (which is a scarce
and expensive commodity) is needlessly wasted figuring out the problem
and then trying to evade it, usually by trying to tapdance a way past
the filters on both sides.  Contrast with a hard, immediate rejection,
which -- coupled with an appropriate resolution mechanism -- facilitates
quick, easy diagnosis and repair.

- Some people use quarantines because they're unduly concerned about
their false positive rate.  But slapping the ill-advised band-aid of a
quarantine on an underlying problem like a high FP rate solves nothing.
It merely obscures the mistake.  The correct solution is to figure out why
the FP rate is high, and fix that, because that's where the problem is.

- Users have spent the past two decades or so conclusively proving, beyond
any possible argument, that they are utterly incapable of distinguishing
spam from non-spam, phish from non-phish.  (Consider carefully: if they
could, en masse and accurately, would we have the scope and scale of the
problem set we currently face?  No.  We would not.)  "Educating users" is
a complete failure as a strategy; see point #5 here:

	The Six Dumbest Ideas in Computer Security
	http://www.ranum.com/security/computer_security/editorials/dumb/

Even personnel who *work in security* can't solve this classification
problem correctly (which is why I said "ask RSA how that's working
out for them").  Your users are no better.  They're probably worse.
So punting the task of correctly classifying incoming mail to them is
reckless and irresponsible.  With precious few and rare exceptions, they
WILL fail.  Freuqently.  And sometimes (again: see RSA) the consequences
of even a single failure can be catastrophic.

More bluntly, allowing users anywhere near the machinery of security,
when they've proven for decades that they're absolutely unqualified to
handle even its simplest aspects, is clearly unprofessional and highly
negligent.  Yes, this is somewhat a fascist, uncompromising, condescending,
etc. attitude.  It's also the only one that's been demonstrated to work in
the real world.  It's a shame, really, but this is not the 'net of 1983.
You can be nice and flexible and accomodating or you can be (modestly)
secure.  Pick one.

- Disposition of quarantined mail is problematic -- unless of course
the intention is to let it accumulate indefinitely, which is a dubious
idea at best.  Eventually, *something* has to be done.  Delivering it
to the user is probably unwise, which you should already know, otherwise
you would have delivered it in the first place.  Simply deleting it means
that you lied to the delivering MTA back when you first accepted it --
and that's very poor operational practice.  Not to mention quite rude.
So what, *exactly*, do you plan to do with it?  When?  How?

- Informing users of quarantined mail is another problem.  Anyone with
any familiarity with user behavior knows that they'll click on anything
and everything in a heartbeat without performing any kind of due diligence.
So that well-crafted spear phish that you should have rejected outright
but quarantined because it scored well enough on your content-screening
system?  Your user is going to pull that right back and open it as soon
as they glance over the "Subject" line quickly in Monday morning's
quarantine report, and you will be 0wned by lunchtime.  Game over, man.

- On top of all of this, there's an even more fundamental strategic problem.
Spam is (among many other things) an attack on the time and attention
of recipients.  Mail system operators who deploy quarantines are
aiding and abetting that attack, because they're chewing up the same
scarce and expensive resources: time and attention.  One of the goals
of a mail defense system is to reduce that time and attention to zero.
Of course, under real-world conditions, that's never possible -- still,
one should always strive to asymptotically approach it.  But quarantines
are clearly a massive step in the wrong direction, for what I trust
are abundantly obvious reasons.

> Implemented appropriately;  an SMTP callout (or "SMTP connect"
> verification) to the connecting IP is a great way to help resolve the
> suspiciousness level of the sending mail server.

No, it's always abusive.  As analysis carried out by the late Bruce
Gingery, myself, and others on spam-l years ago (when Verizon was doing
this) demonstrates, allowing others to cause your operation to generate
outbound SMTP traffic to third parties of *their* choosing is a serious
tactical error.  It furnishes a free, scalable DDoS support service --
which, unfortunately, is not a hypothetical issue.  It also facilitates
third-party bypass of others' security mechanisms, which, depending on
your legal jurisdiction, may be an issue: I would recommend consulting
your attorneys before deploying it, and asking them what exposure you
incur by doing so.  And on top of all that, it has no anti-spam value
whatsoever.  (Note: self-directed callouts, to one's own internal mail
infrastructure, are not abusive.  They're inefficient, and much better
methods nearly always exist for providing the same functionality, but
they're not abusive.)

> Using the name abuse@  for the abuse contact  is not required,  and it is
> likely to be targetted by spammers.

Of course it will be.  So will "postmaster".   So will every other
role and non-role address.  Any reasonably competent operation should
have no problem dealing with that.  IMHO, anyone who cannot manage the
rudimentary task of managing their "abuse" mailbox is wholly unqualified
to run a mail system.  (Or a web server, or a network, or anything else
exposed to the public Internet.)

---rsk