How our young colleagues are being educated....

William Herrin bill at
Fri Dec 26 07:56:40 UTC 2014

On Thu, Dec 25, 2014 at 7:06 PM, Mike Jones <mike at> wrote:
> As for the content.. a scalable network is one
> you can add hosts to, so what's a non-scalable network? will the
> building collapse if i plug my laptop in?

Hi Mike,

A few starting points for interesting insight:

According to the estimate, it costs about $8000/year (pennies here and
pennies there, they add up) to add a single multihomed network to the
Internet before you even consider the bytes sent and received. There
are around 500,000 such networks. If 10,000,000 such networks were
required, we would have difficulty building routers that could work.

Indeed, in the 90's the Internet's 50,000ish networks caught up to and
nearly exceeded the routers we were capable of building. We came close
to having to triage by cutting networks off the Internet.

That's an example of something that scales poorly.

On the other hand, adding a DNS zone costs $10/year or less. We could
add a billion or a trillion more and it might add a few million
dollars total to the cost of a few root and TLD name servers.

The DNS scales well.

> As I have been following NANOG for years I do notice a lot of mistakes
> or "over-simplifications" that show a clear distinction between the
> theory in the university books and the reality on nanog, and
> demonstrate the lecturers lack of real world exposure. As a simple
> example, in IPv4 the goal is to conserve IP addresses therefore on
> point to point links you use a /30 which only wastes 50% of the
> address space. In the real world - /31's? but a /31 is impossible I
> hear the lecturers say...

In the real world you often assign a /32 to a loopback address on each
router and make all of the serial interfaces borrow that address (ip
unnumbered in Cisco parlance) which wastes no addresses.

With non-point to point links there are other tricks you can play to
avoid wasting more addresses than strictly necessary.

> Amoung the things I have heard so far: MAC Addresses are unique,

Except when they're not. The 802.3 standard is ambiguous about whether
a MAC address should be unique per interface or unique per host. Sun
(now Oracle) took the latter view and assigned the same MAC address to
every Ethernet port on a particular host leading to hideously confused
Ethernet switches.

The ambiguity even creeps into Linux. Unless the behavior is
overridden with a sysctl, Linux will happily answer an arp request on
eth0 for an IP address that lives on eth1.

> IP fragments should be blocked for security reasons,

Not a smart move, IMO. In a stateful firewall (e.g. NAT) let the
firewall reassemble the packets. In a stateless firewall, block the
first fragment only, and only if it's too short for whatever filtering
you intend to apply. Any first fragment that's not an attack will be
at least a few hundred bytes long.

Also, pity the fool who blocks ICMP because he breaks TCP at the same
time. Path MTU discovery requires ICMP destination unreachable
messages to function. TCP will screech to a halt every time it
attempts to send a packet larger than the path MTU until the host
receives the ICMP notification.

> and the OSI model
> only has 7 layers to worry about. All theoretically correct. All
> wrong.

Not exactly. The OSI layers exhibit a basically correct understanding
of packet networks. They just don't stack so neatly as the authors
expected. In particular, we keep finding excuses to stack additional
layer 2's and 3's on top of underlying layer 2's and 3's. We give this
names like "MPLS" and "VPN."

Bill Herrin

William Herrin ................ herrin at  bill at
Owner, Dirtside Systems ......... Web: <>
May I solve your unusual networking challenges?

More information about the NANOG mailing list