COVID-19 vs. our Networks

Mon Mar 16 18:58:57 UTC 2020

On this subject, this is worth a read:

https://transition.fcc.gov/pshs/docs/emergency-information/Pandemic_Comms_Impact_Study_%28December%202007%29.pdf
Department of Homeland Security
Pandemic Influenza Impact on Communications Networks Study
Dec 2007

JL

From: NANOG <nanog-bounces at nanog.org> on behalf of Tom Beecher <beecher at beecher.cc>
Date: Monday, March 16, 2020 at 2:10 PM
To: Mike Bolitho <mikebolitho at gmail.com>
Cc: NANOG <nanog at nanog.org>
Subject: Re: COVID-19 vs. our Networks

Mike-

The TSP program provides for priority treatment for only 2 things : provisioning of new capacity, and restoration of capacity. It provides no accommodations for intermittent degradation events upstream.

Source :

DHC Office of Emergency Communications, TSP Program Office, TSP Vendor Handbook.

https://www.cisa.gov/sites/default/files/publications/OEC%20Service%20Vendor%20Handbook%20for%20TSP%2010-23-2017%20FINAL%20508C.pdf<https://urldefense.com/v3/__https:/www.cisa.gov/sites/default/files/publications/OEC*20Service*20Vendor*20Handbook*20for*20TSP*2010-23-2017*20FINAL*20508C.pdf__;JSUlJSUlJSU!!CQl3mcHX2A!XxhMP9AfXmBNjiociWHdq1-r4dwGmRnCRuDMe2bkws8JFS7ZeBDDcSORQWi3sQehSRlv9eI9$>

On Mon, Mar 16, 2020 at 10:42 AM Mike Bolitho <mikebolitho at gmail.com<mailto:mikebolitho at gmail.com>> wrote:
I think there's a bit of a misunderstanding of what I'm trying to say here. We have dual private lines from two Tier I providers. These interconnect all major hospitals and our data centers. We also have a third metro connection that connects things regionally. We have DIA on top of that. I think people are vastly underestimating just how much $aaS there is within the medical field. TeleDoc, translation services, remote radiologists, the way prescriptions get filled, how staffing works, third party providers basically hoteling within our facilities, critical staff VPNed in because the government has locked things down, etc. Then there's things that we don't use but I'm sure other providers do, GoToMeeting, O365, VaaS, etc. There's no practical way to engineer your WAN to facilitate dozens of connections to these services.

This extends beyond just hospitals as well. Fire departments, police departments, water treatment etc. Regardless of whether or not those entities planned well (I think we did), the government should and will step in if critical services are degraded. And for what it's worth, Stephen, I know how things are built within the ISP world. I spent four years there. That doesn't change the fact that we're possibly heading into uncharted waters when it comes to utilization and the impact that will have on $aaS products that are interwoven into every single vertical, including entities that fall under TSP, critical national security and emergency preparedness functions, including those areas related to safety, maintenance of law and order, and public health. It's easy for all you guys to sit here and armchair quarterback other people's planning but when things really start to degrade, all bets are off. If you don't believe that, just look at the news. States are literally shutting down private businesses (restaurants, bars, night clubs, private schools) and banning people from associating in groups of larger than 50.
The opinions expressed here are my own and do not represent my employer or their views.

- Mike Bolitho

On Sun, Mar 15, 2020 at 6:12 PM Stephen Fulton <sf at lists.esoteric.ca<mailto:sf at lists.esoteric.ca>> wrote:
In $dayjob I constantly see the lack of understanding of the difference
between what the Internet is and what a path engineered private circuit
is (eg. pseudowire, wave, whatever).  The latest fight is over SD-WAN
and those who think it will replace MPLS entirely and they won't need
those expensive routers anymore.  But I digress.

Mark's comment and others like it are the correct approach Mike.  If
your private WAN is most critical, then invest in and manage user
complaints about poor Internet service.  ISP's, IXP's and CDN's are not
going to twist themselves into knots to solve your problems, even if
someone calls it an emergency.  Sorry.

Stephen

On 2020-03-15 02:01, Mark Tinka wrote:
>
>
> On 14/Mar/20 19:14, Mike Bolitho wrote:
>
>>     /
>>     /
>>
>> I work for a hospital, we ran into some issues last week due to
>> congestion that was totally outside of our control that was off of our
>> WAN (Thanks Call Of Duty). Now, the issue we ran into was not mission
>> critical at the time but it was still disruptive. As more and more
>> people are driven home during this time, more and more people will be
>> using bandwidth intensive streaming and online gaming products. If
>> more and more TSP coded entities are running into issues, ISPs, IXPs,
>> and CDNs will be forced to act.
>
> Hmmh, if that level of priority is required, I'd probably build my own
> network, and not rely on public infrastructure like the Internet.
>
> Mark.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20200316/fbb83a44/attachment.html>