The 100 Gbit/s problem in your network

fredrik danerklint fredan-nanog at fredan.se
Tue Feb 12 16:17:50 UTC 2013


> And if you don't have said awesome software, then how do you propose to
> limit the bandwidth need for the cache so you aren't burning more bandwidth
> than your hit rate, which is what everyone is trying to ask you (or more
> accurately, explain to you)?

Without the concept of TLMC, I don't know.

I do think that I need to explain how TLMC works.
(please see the file 'tlmc-20130207-r1.tar.gz' as well).
This is going to be a long answer.


We are trying to get the url:
http://static.tlmc.csp.example/hello_world.html


First the DNS needs to get the IP address of 'static.tlmc.csp.example',
so we have something to connect to. What we would like to have is the
IP address of a cache server at the ISP. The CSP has a 'database' of
which ISP:s around the world do participate in TLMC. This information
is stored in a remark field in the IRR.


We do know of where the origin the DNS request is coming from, so
we answer that request with a CNAME of:

'static.tlmc.csp.example' IN CNAME 
'static.tlmc.csp.example.tlmc.isp.example'

(If an ISP does not participate in TLMC, the CSP would instead answer
with a A/AAAA record).

We now have to ask the DNS server at the ISP for an IP address to
connect to. The ISP is in a good mood today, so we are getting the
anycast address to connect to.

(If the ISP is not in a good mode, called Offline mode in TLMC, the
DNS server at the ISP will answer with a CNAME of:

'static.tlmc.csp.example.tlmc.isp.example' IN CNAME
'kaa.k.se.static.tlmc.csp.example'

This assume that the DNS server was place in Karlskrona, Sweden.
With this the geographic location of where a request is coming is
already built in).

If we have an end-user/residence which have an cache server, this is
the address (the anycast one) its going to listen too. If an end-user
does not have an cache server, the ISP must have one. Probably as close
to the edge as possible.


(Here starts the answer to your question in the beginning):

These two have on thing in comment, though. They have a plug-in in the
Traffic Server called, 'hash_remap' (which I made specifically for
trying to solve the scenario you replied with. And Netflix's).


What the plug-in will do is to change the hostname from
'static.tlmc.csp.example' to a hash-based one. In the example url
giving, this will be:

'b1902023cbb5ff2597718437.tlmc.isp.example'.

The first hash, 'b1902023cbb5ff25', is the combined hash of host and url.
The second hash, '97718437' is the hash of the host only.

With this, the ISP is going to have another DNS request. A hashed based
one. Depending of how much information they are collecting from their
cache servers, they know from which one they should load the content
from in this case. This principle is called consistent hashing
and scales very well.

How many layers of consistent hashing should a ISP be using? Only they
know the answer for this one.


-- 
//fredan

The Last Mile Cache - http://tlmc.fredan.se




More information about the NANOG mailing list