ISP port blocking practice

Fri Oct 23 17:14:30 UTC 2009

Michael Peddemors wrote:
> On October 23, 2009, Steve Bertrand wrote:
>> http://eagle.ca/update/mail/Outlook_Express/index.html
>>
>> ...yes, believe it or not, even with the pictures, they will sometimes
>> still get it wrong ;)
>>
>> Years in planning and implementation, but a good, large-scale learning
>> exercise and the achievement of no port 25 that I'm very proud of.
>>
>> Steve
>>
> 
> Congratulations, it would be nice if everyone got there, and we push all our 
> clients to adopt such a strategy, but it is always surprising how many still 
> fear.. change.. and the phone calls they fear may come from it.

Thanks.

The phone calls is what we 'feared' the most. For most things, I'm able
to come up with hackery/workarounds to enable change with no client
impact, but not in this case.

What we did was go on a massive PR campaign via email and web for nearly
two years while I ran both 587 and 25 in parallel. Also, (for the most
part), we'd have the users make the changes pro-actively during
unrelated calls.

Getting closer to the 'due date', I set up a in-band, on-the-side
network of sensors that monitored for port 25 traffic across the network
segments. The sensors had access to RADIUS and other systems that
automated the task of retrieving the username (or client ident of some
sort) who was using the IP in question during that time period. The
results would then be emailed to me.

Sometimes the support staff would make a few cold-calls here or there to
further knock down the list when things were slow.

Most of the domain hosting and non-resi clients have their own 'techs',
so they were pretty good.

Slowly but surely, I started blocking 25 on segments of the network. At
this point, I'd say that we had about 80% coverage.

On and after doomsday, the call volume wasn't overly bad (I think we had
6 staff at that time). Because we were very prepared (with the
handy-dandy pictorials), calls incoming were exceptionally short: "yep,
you can't send. Read this email we're about to send and you'll be good
to go". We of course impounded into their minds that "oh, you didn't
follow the instructions we've been sending for the last two years" for
good measure.

Collateral damage was minimalistic, but was quickly spotted via the
sensors. Adjustments were made, and here we are. I'd have no fear in
doing it again, now that I know what to expect :)

Although we have only ~10k access users and on top of that ~400 hosted
domains, I do believe that the effort can scale up to any scope, so long
as the proper preparations are made in advance.

I believe renumbering my network twice prior to that helped with keeping
me sensible and realistic in how I needed to prepare though.

Cheers,

Steve