[alexis at panix.com: Panix.com- Some brief comments on the hijacking of our domain]

Mon Jan 17 08:43:59 UTC 2005

----- Forwarded message from Alexis Rosen <alexis at panix.com> -----

X-Original-To: tls at netbsd.org
Delivered-To: tls at netbsd.org
Resent-Message-Id: <200501170842.j0H8gWi21166 at panix5.panix.com>
X-Original-To: tls at panix.com
Delivered-To: tls at rek.tjls.com
Date: Mon, 17 Jan 2005 01:42:04 -0500
From: Alexis Rosen <alexis at panix.com>
To: nanog at merit.edu
Subject: Panix.com- Some brief comments on the hijacking of our domain
User-Agent: Mutt/1.4.2.1i

[Please note: I tried to post this five hours ago. It didn't make it, though
 I resubscribed to nanog-post (and acked the confirmation check) about half
 an hour previously. I'm resending (with light edits) and CCing this to a
 few friends; if any of you get this and see that it's not on nanog yet,
 please resend it for me. Thanks.]

We're still digging out from under here, so I can't say nearly as much as I'd
like. However, I have a few things that really need to be said sooner rather
than later. (A couple of the later points are operational. Skip to "***" if
you don't care who I'm grateful to...)

First, I want to thank Martin Hannigan at Verisign. Whatever I may think of
the (in)action I got from other parties there, he made significant efforts
to get them to move, and the incomplete view of events that I have leads
me to believe that it's his efforts, and the efforts of others at Verisign
that he worked on, that got Melbourne IT to finally get off the dime. This
was a very serious effort on his part, for someone who wasn't his direct
customer, and I'm very appreciative of the concern and the effort.

(This isn't to say that the immense efforts of other parties wasn't also
helpful in this respect.)

Secondly, I want to thank the MANY people here (and elsewhere), most of whom
I don't know and have never had contact with, who devoted time and energy to
this issue. Some I do know, and some of them were especially generous. You
know who you are, but a partial list includes Thor Simon, Perry Metzger,
Steve Bellovin, Bill Manning, and <hm, I don't know if I can say those names>.
Thank you.

Third (here's the "***"), I want to make a plea for those with operational
control over large nameservers to reload their caches or expire out the
panix.com entries from their caches, if they haven't yet picked up the
correct data for our zone. (Note that having correct "NS" records isn't
sufficient if you're caching all types.) The correct zones can be pulled
from 198.7.0.1 or 198.7.0.2, for comparison's sake.

If any of you have hand-copied our data into your DNS, please delete it
so we're not afflicted by odd bits of stale data in the far future, when
this incident is long forgotten.

I noted something very odd earlier today. The A records for the hosts
purporting to be mail.panix.com and mail2.panix.com were changed, with the
last octets switched to ".0", making them unreachable. At the time I was
grateful (because mail was being queued or bounced at the sender side,
rather than bounced- and possibly copied- at the recipient side) but I
didn't have time to try to figure out who had done what. I still don't know
who/what was responsible, but I thank those who are, and just so I have a
fuller understanding, I'd appreciate it if someone who knows what was done
would contact me and fill me in.

Someone here pointed out that we seem to have an SSH daemon running on
port 80. That's intentional. It's on our shell hosts, and it's actually a
clever bit of front-end code that switches web clients to a web server and
ssh clients to the ssh daemon. It's for the benefit of customers who want
to ssh in but are behind dumbass (or rightfully paranoid, take your pick)
firewalls that don't allow out anything but connections to port 80.

Thor and others have been commenting a bit on the fact that *something*
is broken or compromised, either at MelbourneIT, Dotster, or Verisign. I
hope that now that it's Monday morning in Australia, and will be in 12-15
hours here in the US, we can make some progress on figuring out what really
happened. This would start with Verisign, Dotster, and MelbourneIT producing
*all* relevant logs. I'll be discussing that with them tomorrow.

There's a lot more to be said here, but for now we're going to finish
cleaning up the mess, get the registry back to dotster, and try to catch
up on some sleep. Oh, and work with various law enforcement types to try
to catch the bastards responsible for this.

/a
---
Alexis Rosen
President
Public Access Networks Corp. - Panix.com                      alexis at panix.com
Grand Central Server LLC.                        alexis at grandcentralserver.com

----- End forwarded message -----