Someone Please Help Me Understand

Sat Apr 2 17:54:40 UTC 2016

Ok, I'm trying to learn, so bear with me.

We are an ISP in Indianapolis that has full routes from 3 different
providers HE.Net in Columbus OH being one.  We also are peered with 2
peering exchanges, including EquinixIX in Chicago.  The problem is
Instagram and Facebook (same company, I know) for our customers seems
very slow.  

This is where I need a way to troubleshoot/understand more.  I did a
traceroute to the IP that is serving the pictures, and it resolves to
the FBCDN servers in Dallas, and is showing packet loss and pings once
it hits Dallas, and are in the 1xxs of ms.

Tracing route to instagram-p3-shv-01-dfw1.fbcdn.net [31.13.66.52]

over a maximum of 30 hops:

  1     4 ms     3 ms     4 ms  10.7.0.1

  2    20 ms    43 ms    42 ms  inmtvlobs-rtr-01.dynamic.pdsconnect.me
[192.69.57.1]

  3    25 ms    47 ms    29 ms
inmtvlmwt-rtr-01.infrastructure.pdsconnect.me [192.69.48.162]

  4    46 ms    32 ms    58 ms
inindyhen-core1.infrastructure.pdsconnect.me [192.69.48.193]

  5    36 ms    53 ms    51 ms  ge2-4.core1.cmh1.he.net [184.105.32.1]

  6    47 ms    41 ms    75 ms  10ge1-2.core1.chi1.he.net
[184.105.222.165]

  7    57 ms    57 ms    53 ms  100ge14-1.core2.chi1.he.net
[184.105.81.97]

  8    57 ms    73 ms    84 ms  100ge12-1.core1.mci3.he.net
[184.105.81.209]

  9    75 ms    73 ms   102 ms  10ge15-6.core1.dal1.he.net
[184.105.222.10]

10    93 ms   103 ms    92 ms  eqix-da1.facebook.com [206.223.118.176]

11   102 ms   101 ms     *     psw01c.dfw1.tfbnw.net [173.252.65.196]

12    92 ms    97 ms   105 ms  msw1aq.01.dfw1.tfbnw.net [204.15.21.89]

13   110 ms     *       98 ms  instagram-p3-shv-01-dfw1.fbcdn.net
[31.13.66.52]

Since I am peered with the route servers in EquinixIX Chicago, shouldn't
the data be coming from there, or at least hit their routers?  In my
trace, it shows HE to Chicago, then to Dallas.  How does FB decide what
IP the content gets displayed from, and is there anything I can do as a
provider?  If it is DNS, I can obviously clear the cache to see if it
gets new IPs.  If I'm not getting FB peering IPs in Chicago, do I need
to peer directly?  Should I get FaceBook involved?

Eric Rogers

PDS Connect

(317) 831-3000 x200 