Charter and Cox contacts
daniel at pyranah.com
daniel at pyranah.com
Tue May 14 14:13:06 UTC 2019
"On 5/13/19 12:11 PM, daniel at pyranah.com wrote:
> Does anyone have contacts at Charter (Spectrum) and Cox? For some reason,
> our IP has been blocked by them and our customers are unable to send email
> via their charter/cox accounts. Thanks
Would you be talking about port 25/tcp outbound? Lots of ISPs will
block port 25 as a rule; you have to call to have it opened. At the
same time, you can request the PTR record you need to keep big mail
receivers happy."
It's the outgoing smtp ports, 587 for charter and 465 for cox.
They are open on my end but our IP address that most of our
customers are natted through is blocked by the email servers.
Daniel Temple
VP of Operations
Pyranah Communications, LLC
828-743-2470 ext.205
Pyranah.com
Like us on Facebook!
-----Original Message-----
From: NANOG <nanog-bounces at nanog.org> On Behalf Of nanog-request at nanog.org
Sent: Tuesday, May 14, 2019 8:00 AM
To: nanog at nanog.org
Subject: NANOG Digest, Vol 136, Issue 14
Send NANOG mailing list submissions to
nanog at nanog.org
To subscribe or unsubscribe via the World Wide Web, visit
https://mailman.nanog.org/mailman/listinfo/nanog
or, via email, send a message with subject or body 'help' to
nanog-request at nanog.org
You can reach the person managing the list at
nanog-owner at nanog.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of NANOG digest..."
Today's Topics:
1. Issues with SIP packets between VZ Fios and NTT (Dovid Bender)
2. Re: Ownership of Routers on Both Ends of Transnational Links
(Pengxiong Zhu)
3. Re: Issues with SIP packets between VZ Fios and NTT
(Brielle Bruns)
4. Re: Ownership of Routers on Both Ends of Transnational Links
(Christopher Morrow)
5. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender)
6. Re: Ownership of Routers on Both Ends of Transnational Links
(Pengxiong Zhu)
7. Re: Issues with SIP packets between VZ Fios and NTT
(Brielle Bruns)
8. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender)
9. Re: Ownership of Routers on Both Ends of Transnational Links
(Christopher Morrow)
10. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender)
11. Re: Issues with SIP packets between VZ Fios and NTT (Jared Mauch)
12. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender)
13. Re: Issues with SIP packets between VZ Fios and NTT (Pete Rohrman)
14. Charter and Cox contacts (daniel at pyranah.com)
15. Mostly name and shame... (bzs at theworld.com)
16. Re: Ownership of Routers on Both Ends of Transnational Links
(Randy Bush)
17. Re: Charter and Cox contacts (Stephen Satchell)
18. RE: FCC Hurricane Michael after-action report (frnkblk at iname.com)
19. Re: FCC Hurricane Michael after-action report (Mike Bolitho)
20. Re: Issues with SIP packets between VZ Fios and NTT (Saku Ytti)
21. Re: NTP for ASBRs? (Mark Tinka)
22. Re: NTP for ASBRs? (colin johnston)
23. Fwd: [afnog] SAFNOG-5 & iWeek 2019 Registrations Open (Mark Tinka)
24. Re: Issues with SIP packets between VZ Fios and NTT (Dovid Bender)
----------------------------------------------------------------------
Message: 1
Date: Mon, 13 May 2019 11:21:12 -0400
From: Dovid Bender <dovid at telecurve.com>
To: NANOG <nanog at nanog.org>
Subject: Issues with SIP packets between VZ Fios and NTT
Message-ID:
<CAM3TTh2qpkWOD2DPy2GU-9hHkKUX8GVExfjYmmq0YEeCyubaKw at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Hi,
Over the last 48 hours we have been getting a lot of alerts of customers
phones losing registrations to us. All the complaints are coming from
customers that are on VZ Fios in the NYC area. Anyone else see anything
strange going on?
TIA.
Dovid
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/37343019/attachment-0001.html>
------------------------------
Message: 2
Date: Sat, 11 May 2019 17:24:27 -0700
From: Pengxiong Zhu <pzhu011 at ucr.edu>
To: "North American Network Operators' Group" <nanog at nanog.org>
Cc: Zhiyun Qian <zhiyunq at cs.ucr.edu>, Zhongjie Wang
<zwang048 at ucr.edu>, Keyu Man <kman001 at ucr.edu>
Subject: Re: Ownership of Routers on Both Ends of Transnational Links
Message-ID:
<CAEnbXKuzZ8ZdQafN3cnn6Q5zo3ydsmiMSJ83mEjhL5wf_Myk9g at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Thanks again for your insightful responses!
The case we discuss above is Chinese ISPs renting routers located outside
China and the IPs belong to other ISPs.
How about the case that the IP belongs to a Chinese ISP and is located in
US(from RTT result), can we say it is very likely or definitely
owned/operated by the Chinese ISP? Why would some ISP try to rent routers
of Chinese ISP in US?
For example, a traceroute from Ohio to an IP in China. Hop 17 and hop 18
should be located in US based on the RTT, and yet they belong to a Chinese
AS(China Telecom). Does this mean that Chinese Telecom is managing these
two hops?
HOST: Loss% Snt Last Avg Best Wrst StDev
6. AS??? 100.65.11.97 0.0% 100 2.0 1.0 0.4 12.6 1.3
7. AS??? 52.93.15.238 0.0% 100 2.4 2.0 1.5 11.4 1.1
8. AS??? 52.93.14.134 0.0% 100 21.9 26.3 4.2 54.4 11.3
9. AS??? 52.93.14.119 0.0% 100 2.6 2.1 1.6 10.8 1.2
10. AS??? 100.91.27.86 0.0% 100 25.8 26.2 25.6 34.9 1.2
11. AS??? 54.239.42.197 0.0% 100 25.5 25.9 25.4 35.8 1.5
12. AS??? 100.91.4.218 0.0% 100 25.9 26.2 25.1 38.3 1.6
13. AS??? 100.91.4.217 0.0% 100 25.4 26.0 25.3 41.4 2.0
14. AS??? 100.91.5.85 0.0% 100 25.3 25.8 25.2 29.1 0.9
15. AS??? 54.239.103.86 0.0% 100 25.6 30.0 25.2 49.1 3.8
16. AS??? 54.239.103.77 0.0% 100 25.3 25.6 25.2 28.1 0.5
17. AS4134 218.30.53.1 0.0% 100 28.0 29.1 25.2 33.1 2.3
18. AS4134 202.97.50.21 0.0% 100 32.4 29.1 25.2 33.5 2.4
19. AS??? ??? 100.0 100 0.0 0.0 0.0 0.0 0.0
20. AS??? ??? 100.0 100 0.0 0.0 0.0 0.0 0.0
21. AS4134 202.97.94.121 0.0% 100 186.8 185.6 181.8 189.8 2.3
22. AS4816 119.147.222.6 0.0% 100 182.6 183.5 182.4 195.8 1.8
23. AS4816 183.2.182.130 0.0% 100 181.7 183.3 181.5 207.0 3.9
24. AS??? ??? 100.0 100 0.0 0.0 0.0 0.0 0.0
25. AS45102 116.251.113.158 0.0% 100 176.7 177.9 176.5 186.7 2.1
26. AS45102 116.251.115.141 0.0% 100 213.2 213.4 213.1 218.5 0.6
Best,
Pengxiong Zhu
Department of Computer Science and Engineering
University of California, Riverside
On Tue, Apr 16, 2019 at 7:37 PM Erik Sundberg <ESundberg at nitelusa.com>
wrote:
> May sure when you are dealing with transnational links to watch the
> latency so you can tell when the link goes international. Just because you
> are going from a US Network provider to China Telecom doesn't mean that
> your not connecting to them in the united states.
>
>
>
> For example a traceroute from Denver to 27.29.128.1 which is an IP in
> China Telecom's network.
>
> It's about 26ms between Denver and Los Angeles. Hop 5 to Hop 6
>
> China Telecom connects to GTT in Los Angeles Hop7/8
>
> On Hop 8 is in the United State and Hop 9 is across the pacific. Because
> the latency goes from 31 ms to 183 ms.
>
>
> Just something to keep in mind.
>
>
>
> Packets Pings
> Host
> Loss% Snt Last Avg Best Wrst StDev
> 1. _gateway
> 0.0% 14 1.0 1.2 1.0 2.8 0.5
> 2. te-0-0-26.ear2.den1.us.nitelusa.net
> 0.0% 14 0.9 1.0 0.8 2.1 0.4
> 3. te-0-0-26.ear1.den1.us.nitelusa.net
> 0.0% 14 1.1 1.6 1.1 2.9 0.7
> 4. te-0-0-1-0.cr1.den1.us.nitelusa.net
> 0.0% 14 1.0 1.0 1.0 1.1 0.0
> 5. ae1-122.cr0-den2.ip4.gtt.net
> 0.0% 14 0.5 1.2 0.3 6.9 2.0
> 6. et-0-0-47.cr3-lax2.ip4.gtt.net
> 0.0% 14 26.5 26.4 26.2 26.7 0.2
> 7. as4134.lax20.ip4.gtt.net
> 0.0% 14 27.7 28.7 26.8 30.1 1.1
> 8. 202.97.50.29
> 0.0% 14 31.4 30.6 26.8 34.1 2.4
> 9. 202.97.41.129
> 0.0% 14 183.3 187.1 183.3 190.8 2.4
> 10. 202.97.94.101
> 0.0% 14 187.9 188.6 186.1 211.2 6.8
> 11. 202.97.94.141
> 0.0% 13 177.8 180.7 177.2 184.2 2.3
> 12. 202.97.67.54
> 0.0% 13 199.5 201.2 197.4 205.1 2.6
> 13. 111.177.110.62
> 0.0% 13 205.9 206.3 205.9 208.2 0.7
> 14. 27.29.128.1
> 0.0% 13 202.6 202.8 202.5 203.9 0.4
>
>
> Erik Sundberg
>
> Sr. Network Engineer
>
> Nitel
>
> 350 N Orleans Street
>
> Suite 1300N
>
> Chicago, Il 60654
>
> Desk: 773-661-5532
>
> Cell: 708-710-7419
>
> NOC: 866-892-0915
>
> Email: esundberg at nitelusa.com
>
> web: www.nitelusa.com
>
> ------------------------------
> *From:* Zhiyun Qian <zhiyunq at cs.ucr.edu>
> *Sent:* Tuesday, April 16, 2019 1:02:36 PM
> *To:* Erik Sundberg
> *Cc:* Pengxiong Zhu; Zhiyun Qian; Zhongjie Wang; Keyu Man
> *Subject:* Re: Ownership of Routers on Both Ends of Transnational Links
>
> Erik,
>
> Thanks a lot for the information! This is extremely helpful. We are
> conducting an analysis on performance/policy-related study on transnational
> links. We are hoping to submit a paper soon. Will be glad to share all the
> details once we have a draft!
>
> Best,
> -Zhiyun
>
>
> On Tue, Apr 16, 2019 at 10:35 AM Erik Sundberg <ESundberg at nitelusa.com>
> wrote:
>
> CPE is usually ran by the customer. Some provider do offer managed routers
> for a fee. Kinda like renting a cable modem from your provider.
>
>
> What are your guys trying to accomplish or find out?
>
> Erik
>
>
>
> Erik Sundberg
> Sr. Network Engineer
> Nitel
> 350 N Orleans Street
> Suite 1300N
> Chicago, Il 60654
> Desk: 773-661-5532
> Cell: 708-710-7419
> NOC: 866-892-0915
> Email: esundberg at nitelusa.com
> web: www.nitelusa.com
>
> ------------------------------
> *From:* Pengxiong Zhu <pzhu011 at ucr.edu>
> *Sent:* Tuesday, April 16, 2019 12:32 PM
> *To:* Erik Sundberg
> *Cc:* Zhiyun Qian; Zhongjie Wang; Keyu Man
> *Subject:* Re: Ownership of Routers on Both Ends of Transnational Links
>
> Thanks a lot!
>
> Are the Customer Devices managed by Telia or the customer?
>
> Best,
> Pengxiong Zhu
> Department of Computer Science and Engineering
> University of California, Riverside
>
>
> On Tue, Apr 16, 2019 at 7:43 AM Erik Sundberg <ESundberg at nitelusa.com>
> wrote:
>
> I hope this helps with the breakdown for telia.
>
>
>
>
> Telia i think is using /31's for there serial blocks now
>
> 62.115.170.56 (Telia Edge Rotuer)
>
> 62.115.170.57 (Customer Device)
>
>
>
> chinaunicom-ic-341501-sjo-b21.c.telia.net.
>
>
> <Customername>-<CircuitID>-<POP>-<router>.c.telia.net
>
>
>
> Customer: ChinaUnicom
>
> Telia Circuit ID's are: ic-123456
>
> POP: SJO (Airport code)
>
> Router: b21
>
> Doamin: c.telia.net "Customer.telia.net"
>
>
>
>
> Erik Sundberg
>
> Sr. Network Engineer
>
> Nitel
>
> 350 N Orleans Street
>
> Suite 1300N
>
> Chicago, Il 60654
>
> Desk: 773-661-5532
>
> Cell: 708-710-7419
>
> NOC: 866-892-0915
>
> Email: esundberg at nitelusa.com
>
> web: www.nitelusa.com
>
> ------------------------------
> *From:* NANOG <nanog-bounces at nanog.org> on behalf of Pengxiong Zhu <
> pzhu011 at ucr.edu>
> *Sent:* Monday, April 15, 2019 11:36:45 PM
> *To:* nanog at nanog.org
> *Cc:* Keyu Man; Zhiyun Qian; Zhongjie Wang
> *Subject:* Ownership of Routers on Both Ends of Transnational Links
>
> Howdy folks,
>
> We are a group of researchers at UC Riverside conducting some measurement
> about transnational networks. In particular, we are interested in studying
> the ownership of routers on the two sides of transnational links.
>
> We have some concrete questions which we hope someone can shed some light
> on. Basically when we send packets from US/Canada to China, through
> traceroute and the RTT of each hop, we can locate the last hop in the US
> before the packets enter China (*there is a large jump of RTT of 100+ms
> from this hop onwards*). Oftentimes the ownership of such routers is
> ambiguous.
>
> These hops whose IPs seem to belong to US or European ISPs (*according to
> BGP info*) but their reverse DNS names have *chinaunicom* in it, which is
> a Chinese ISP.
> AS1299 Telia Company AB
> 62.115.170.57 name = chinaunicom-ic-341501-sjo-b21.c.telia.net.
> 62.115.33.230 name = chinaunicom-ic-302366-las-bb1.c.telia.net.
> 213.248.73.190 name = chinaunicom-ic-127288-sjo-b21.c.telia.net.
>
> AS701 Verizon Business
> 152.179.103.254 name = chinaunicom-gw.customer.alter.net.
>
> While the following routers, they don't have a reverse DNS name at all,
> which seem to be uncommon if they were managed by US or European ISPs but
> quite common for Chinese ISPs.
> AS6453 TATA COMMUNICATIONS (AMERICA) INC
> 63.243.205.90
> 66.110.59.118
>
> Can anyone confirm that these are indeed managed by the Chinese ISPs (even
> though they are physically located in the US according to the traceroute
> and RTT analysis)?
>
>
> Best,
> Pengxiong Zhu
> Department of Computer Science and Engineering
> University of California, Riverside
>
> ------------------------------
>
> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files
> or previous e-mail messages attached to it may contain confidential
> information that is legally privileged. If you are not the intended
> recipient, or a person responsible for delivering it to the intended
> recipient, you are hereby notified that any disclosure, copying,
> distribution or use of any of the information contained in or attached to
> this transmission is STRICTLY PROHIBITED. If you have received this
> transmission in error please notify the sender immediately by replying to
> this e-mail. You must destroy the original transmission and its attachments
> without reading or saving in any manner. Thank you.
>
>
> ------------------------------
>
> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files
> or previous e-mail messages attached to it may contain confidential
> information that is legally privileged. If you are not the intended
> recipient, or a person responsible for delivering it to the intended
> recipient, you are hereby notified that any disclosure, copying,
> distribution or use of any of the information contained in or attached to
> this transmission is STRICTLY PROHIBITED. If you have received this
> transmission in error please notify the sender immediately by replying to
> this e-mail. You must destroy the original transmission and its attachments
> without reading or saving in any manner. Thank you.
>
>
> ------------------------------
>
> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files
> or previous e-mail messages attached to it may contain confidential
> information that is legally privileged. If you are not the intended
> recipient, or a person responsible for delivering it to the intended
> recipient, you are hereby notified that any disclosure, copying,
> distribution or use of any of the information contained in or attached to
> this transmission is STRICTLY PROHIBITED. If you have received this
> transmission in error please notify the sender immediately by replying to
> this e-mail. You must destroy the original transmission and its attachments
> without reading or saving in any manner. Thank you.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190511/4d6a3b8d/attachment-0001.html>
------------------------------
Message: 3
Date: Mon, 13 May 2019 09:38:53 -0600
From: Brielle Bruns <bruns at 2mbit.com>
To: nanog at nanog.org
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID: <98cf5423-4167-2242-62f7-782a9c500c18 at 2mbit.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
On 5/13/2019 9:21 AM, Dovid Bender wrote:
> Hi,
>
> Over the last 48 hours we have been getting a lot of alerts of customers
> phones losing registrations to us. All the complaints are coming from
> customers that are on VZ Fios in the NYC area. Anyone else see anything
> strange going on?
>
While you are diagnosing, might check to make sure that the SIP ALG is
disabled on all of their routers too.
--
Brielle Bruns
The Summit Open Source Development Group
http://www.sosdg.org / http://www.ahbl.org
------------------------------
Message: 4
Date: Mon, 13 May 2019 11:51:58 -0400
From: Christopher Morrow <morrowc.lists at gmail.com>
To: Pengxiong Zhu <pzhu011 at ucr.edu>
Cc: "North American Network Operators' Group" <nanog at nanog.org>, Keyu
Man <kman001 at ucr.edu>, Zhiyun Qian <zhiyunq at cs.ucr.edu>, Zhongjie
Wang <zwang048 at ucr.edu>
Subject: Re: Ownership of Routers on Both Ends of Transnational Links
Message-ID:
<CAL9jLabwtcfU=ndpRyO2WFxmZzmrxE0JZk2TYTFwjxxdH+1TBw at mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
On Mon, May 13, 2019 at 11:38 AM Pengxiong Zhu <pzhu011 at ucr.edu> wrote:
>
> Thanks again for your insightful responses!
>
> The case we discuss above is Chinese ISPs renting routers located outside China and the IPs belong to other ISPs.
>
I think you are using all of the wrong verbs here... 'renting' does
not make sense here, I'm unclear on what you actually mean, please try
again with a different verb OR more clarifying text.
\
------------------------------
Message: 5
Date: Mon, 13 May 2019 12:20:10 -0400
From: Dovid Bender <dovid at telecurve.com>
To: Brielle Bruns <bruns at 2mbit.com>
Cc: NANOG <nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID:
<CAM3TTh2WCwQcWBjjNPHXQbWAsowXgarev61ovOTA3AZ3D5-cfQ at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Thought of that. Customers have their own CPE's. So far the only thing
mutual here is that it's NTT -> VZ. Here is what I found so far looking at
two Polycom phones using non standard ports (e.g. not 5060)
1) PhoneA tries to register multiple extensions and for each request we
send a 401. We expect to get back a REGISTER request with a no-once but we
don't. This happens for a while and then magically it starts working.
2) PhoneB tries to register the time time as PhoneA and has no issues.
At first I thought it was something possibly with the SIP call-ID but I
ruled that out since in the same SIP DIALOG it was not working then it
started. Also the seems to be per phone each phone is behind NAT and the
traffic is coming from a different NAT'd port. Seems like there is some
device in the middle that is randomly dropping traffic on specific sessions.
On Mon, May 13, 2019 at 11:40 AM Brielle Bruns <bruns at 2mbit.com> wrote:
> On 5/13/2019 9:21 AM, Dovid Bender wrote:
> > Hi,
> >
> > Over the last 48 hours we have been getting a lot of alerts of customers
> > phones losing registrations to us. All the complaints are coming from
> > customers that are on VZ Fios in the NYC area. Anyone else see anything
> > strange going on?
> >
>
>
> While you are diagnosing, might check to make sure that the SIP ALG is
> disabled on all of their routers too.
>
>
>
> --
> Brielle Bruns
> The Summit Open Source Development Group
> http://www.sosdg.org / http://www.ahbl.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/ea4a26ee/attachment-0001.html>
------------------------------
Message: 6
Date: Mon, 13 May 2019 09:31:23 -0700
From: Pengxiong Zhu <pzhu011 at ucr.edu>
To: Christopher Morrow <morrowc.lists at gmail.com>
Cc: "North American Network Operators' Group" <nanog at nanog.org>, Keyu
Man <kman001 at ucr.edu>, Zhiyun Qian <zhiyunq at cs.ucr.edu>, Zhongjie
Wang <zwang048 at ucr.edu>
Subject: Re: Ownership of Routers on Both Ends of Transnational Links
Message-ID:
<CAEnbXKswsW41UmSW4djXhRGdTWX=FyspNq8i9nPAvHDY68bSSg at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Sorry for the confusion. I mean the IPs belong to non-Chinese ISPs but are
actually controlled/managed by Chinese ISPs.
Best,
Pengxiong Zhu
Department of Computer Science and Engineering
University of California, Riverside
On Mon, May 13, 2019 at 8:52 AM Christopher Morrow <morrowc.lists at gmail.com>
wrote:
> On Mon, May 13, 2019 at 11:38 AM Pengxiong Zhu <pzhu011 at ucr.edu> wrote:
> >
> > Thanks again for your insightful responses!
> >
> > The case we discuss above is Chinese ISPs renting routers located
> outside China and the IPs belong to other ISPs.
> >
>
> I think you are using all of the wrong verbs here... 'renting' does
> not make sense here, I'm unclear on what you actually mean, please try
> again with a different verb OR more clarifying text.
> \
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/d31cb249/attachment-0001.html>
------------------------------
Message: 7
Date: Mon, 13 May 2019 10:48:17 -0600
From: Brielle Bruns <bruns at 2mbit.com>
To: NANOG <nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID: <361d0ac7-c356-dbbf-b40b-d75536da8080 at 2mbit.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
On 5/13/2019 10:20 AM, Dovid Bender wrote:
> Thought of that. Customers have their own CPE's. So far the only thing
> mutual here is that it's NTT -> VZ. Here is what I found so far looking
> at two Polycom phones using non standard ports (e.g. not 5060)
> 1) PhoneA tries to register multiple extensions and for each request we
> send a 401. We expect to get back a REGISTER request with a no-once but
> we don't. This happens for a while and then magically it starts working.
> 2) PhoneB tries to register the time time as PhoneA and has no issues.
>
> At first I thought it was something possibly with the SIP call-ID but I
> ruled that out since in the same SIP DIALOG it was not working then it
> started. Also the seems to be per phone each phone is behind NAT and the
> traffic is coming from a different NAT'd port. Seems like there is some
> device in the middle that is randomly dropping traffic on specific sessions.
Are you using TLS encrypted SIP or just plain ol' cleartext?
If its encrypted, I'd look at possibly there being a MTU/MSS issue
somewhere along the path possibly?
--
Brielle Bruns
The Summit Open Source Development Group
http://www.sosdg.org / http://www.ahbl.org
------------------------------
Message: 8
Date: Mon, 13 May 2019 13:04:45 -0400
From: Dovid Bender <dovid at telecurve.com>
To: Brielle Bruns <bruns at 2mbit.com>
Cc: NANOG <nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID:
<CAM3TTh3QT9ezBe=u4GeeMsK11Wb+Gcbfg15jg4pBB=gMyXPkeQ at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Good ol UDP encrypted.
On Mon, May 13, 2019 at 12:49 PM Brielle Bruns <bruns at 2mbit.com> wrote:
>
> On 5/13/2019 10:20 AM, Dovid Bender wrote:
> > Thought of that. Customers have their own CPE's. So far the only thing
> > mutual here is that it's NTT -> VZ. Here is what I found so far looking
> > at two Polycom phones using non standard ports (e.g. not 5060)
> > 1) PhoneA tries to register multiple extensions and for each request we
> > send a 401. We expect to get back a REGISTER request with a no-once but
> > we don't. This happens for a while and then magically it starts working.
> > 2) PhoneB tries to register the time time as PhoneA and has no issues.
> >
> > At first I thought it was something possibly with the SIP call-ID but I
> > ruled that out since in the same SIP DIALOG it was not working then it
> > started. Also the seems to be per phone each phone is behind NAT and the
> > traffic is coming from a different NAT'd port. Seems like there is some
> > device in the middle that is randomly dropping traffic on specific
> sessions.
>
>
> Are you using TLS encrypted SIP or just plain ol' cleartext?
>
> If its encrypted, I'd look at possibly there being a MTU/MSS issue
> somewhere along the path possibly?
>
>
> --
> Brielle Bruns
> The Summit Open Source Development Group
> http://www.sosdg.org / http://www.ahbl.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/6d84c2d2/attachment-0001.html>
------------------------------
Message: 9
Date: Mon, 13 May 2019 13:40:11 -0400
From: Christopher Morrow <morrowc.lists at gmail.com>
To: Pengxiong Zhu <pzhu011 at ucr.edu>
Cc: "North American Network Operators' Group" <nanog at nanog.org>, Keyu
Man <kman001 at ucr.edu>, Zhiyun Qian <zhiyunq at cs.ucr.edu>, Zhongjie
Wang <zwang048 at ucr.edu>
Subject: Re: Ownership of Routers on Both Ends of Transnational Links
Message-ID:
<CAL9jLaYfTGuK165y9tGuJPJfSBciZxEAw=Vp4QXnzc0=g9W5fQ at mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
On Mon, May 13, 2019 at 12:31 PM Pengxiong Zhu <pzhu011 at ucr.edu> wrote:
>
> Sorry for the confusion. I mean the IPs belong to non-Chinese ISPs but are actually controlled/managed by Chinese ISPs.
>
this is, as I think was said earlier, normal practice.
Sometimes you accept a /31 from your "provider" or "peer", sometimes
they accept yours...
sometimes this is because of seasons/reasons/etc, sometimes because
it's how folk denote who's paying for the link in between.
Those ips are not useful as a signal, which I think was also said
previously in this thread.
> Best,
> Pengxiong Zhu
> Department of Computer Science and Engineering
> University of California, Riverside
>
>
> On Mon, May 13, 2019 at 8:52 AM Christopher Morrow <morrowc.lists at gmail.com> wrote:
>>
>> On Mon, May 13, 2019 at 11:38 AM Pengxiong Zhu <pzhu011 at ucr.edu> wrote:
>> >
>> > Thanks again for your insightful responses!
>> >
>> > The case we discuss above is Chinese ISPs renting routers located outside China and the IPs belong to other ISPs.
>> >
>>
>> I think you are using all of the wrong verbs here... 'renting' does
>> not make sense here, I'm unclear on what you actually mean, please try
>> again with a different verb OR more clarifying text.
>> \
------------------------------
Message: 10
Date: Mon, 13 May 2019 14:32:37 -0400
From: Dovid Bender <dovid at telecurve.com>
To: Brielle Bruns <bruns at 2mbit.com>
Cc: NANOG <nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID:
<CAM3TTh3JiP8CeVuB+N8Z7Z4sy6MyxfNRdr4P9CWPhYxwjqApEA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
FYI: More than one person reached out to me off list. The issue is clearly
with VZ. Traces by the others were done and NTT was not in the mix. The
only common denominator was 401 SIP packets hitting VZ Fios IP's in the NY
area.
On Mon, May 13, 2019 at 1:04 PM Dovid Bender <dovid at telecurve.com> wrote:
> Good ol UDP encrypted.
>
>
>
> On Mon, May 13, 2019 at 12:49 PM Brielle Bruns <bruns at 2mbit.com> wrote:
>
>>
>> On 5/13/2019 10:20 AM, Dovid Bender wrote:
>> > Thought of that. Customers have their own CPE's. So far the only thing
>> > mutual here is that it's NTT -> VZ. Here is what I found so far looking
>> > at two Polycom phones using non standard ports (e.g. not 5060)
>> > 1) PhoneA tries to register multiple extensions and for each request we
>> > send a 401. We expect to get back a REGISTER request with a no-once but
>> > we don't. This happens for a while and then magically it starts working.
>> > 2) PhoneB tries to register the time time as PhoneA and has no issues.
>> >
>> > At first I thought it was something possibly with the SIP call-ID but I
>> > ruled that out since in the same SIP DIALOG it was not working then it
>> > started. Also the seems to be per phone each phone is behind NAT and
>> the
>> > traffic is coming from a different NAT'd port. Seems like there is some
>> > device in the middle that is randomly dropping traffic on specific
>> sessions.
>>
>>
>> Are you using TLS encrypted SIP or just plain ol' cleartext?
>>
>> If its encrypted, I'd look at possibly there being a MTU/MSS issue
>> somewhere along the path possibly?
>>
>>
>> --
>> Brielle Bruns
>> The Summit Open Source Development Group
>> http://www.sosdg.org / http://www.ahbl.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/585bb7a9/attachment-0001.html>
------------------------------
Message: 11
Date: Mon, 13 May 2019 14:43:45 -0400
From: Jared Mauch <jared at puck.nether.net>
To: Dovid Bender <dovid at telecurve.com>
Cc: Brielle Bruns <bruns at 2mbit.com>, NANOG <nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID: <3B38B91F-EF60-4838-BA72-DF8EA3691749 at puck.nether.net>
Content-Type: text/plain; charset="utf-8"
This matches my experience with running SIP on networks. Slowly over the years it became more unreliable as “helper” ALGs were in the path.
Eventually we moved some devices off 5060 to alleviate the problem.
Sent from my iCar
> On May 13, 2019, at 2:32 PM, Dovid Bender <dovid at telecurve.com> wrote:
>
> FYI: More than one person reached out to me off list. The issue is clearly with VZ. Traces by the others were done and NTT was not in the mix. The only common denominator was 401 SIP packets hitting VZ Fios IP's in the NY area.
>
>
>
>> On Mon, May 13, 2019 at 1:04 PM Dovid Bender <dovid at telecurve.com> wrote:
>> Good ol UDP encrypted.
>>
>>
>>
>>> On Mon, May 13, 2019 at 12:49 PM Brielle Bruns <bruns at 2mbit.com> wrote:
>>>
>>> On 5/13/2019 10:20 AM, Dovid Bender wrote:
>>> > Thought of that. Customers have their own CPE's. So far the only thing
>>> > mutual here is that it's NTT -> VZ. Here is what I found so far looking
>>> > at two Polycom phones using non standard ports (e.g. not 5060)
>>> > 1) PhoneA tries to register multiple extensions and for each request we
>>> > send a 401. We expect to get back a REGISTER request with a no-once but
>>> > we don't. This happens for a while and then magically it starts working.
>>> > 2) PhoneB tries to register the time time as PhoneA and has no issues.
>>> >
>>> > At first I thought it was something possibly with the SIP call-ID but I
>>> > ruled that out since in the same SIP DIALOG it was not working then it
>>> > started. Also the seems to be per phone each phone is behind NAT and the
>>> > traffic is coming from a different NAT'd port. Seems like there is some
>>> > device in the middle that is randomly dropping traffic on specific sessions.
>>>
>>>
>>> Are you using TLS encrypted SIP or just plain ol' cleartext?
>>>
>>> If its encrypted, I'd look at possibly there being a MTU/MSS issue
>>> somewhere along the path possibly?
>>>
>>>
>>> --
>>> Brielle Bruns
>>> The Summit Open Source Development Group
>>> http://www.sosdg.org / http://www.ahbl.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/b0a29b6a/attachment-0001.html>
------------------------------
Message: 12
Date: Mon, 13 May 2019 14:44:40 -0400
From: Dovid Bender <dovid at telecurve.com>
To: Jared Mauch <jared at puck.nether.net>
Cc: Brielle Bruns <bruns at 2mbit.com>, NANOG <nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID:
<CAM3TTh3rn1Ms+cs51Bc8=_oSyx8SNBu=T=OtxWCnav1WE3b7nQ at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
In our case we are not using 5060. The issue seems exclusive to VZ.
On Mon, May 13, 2019 at 2:43 PM Jared Mauch <jared at puck.nether.net> wrote:
> This matches my experience with running SIP on networks. Slowly over the
> years it became more unreliable as “helper” ALGs were in the path.
>
> Eventually we moved some devices off 5060 to alleviate the problem.
>
> Sent from my iCar
>
> On May 13, 2019, at 2:32 PM, Dovid Bender <dovid at telecurve.com> wrote:
>
> FYI: More than one person reached out to me off list. The issue is clearly
> with VZ. Traces by the others were done and NTT was not in the mix. The
> only common denominator was 401 SIP packets hitting VZ Fios IP's in the NY
> area.
>
>
>
> On Mon, May 13, 2019 at 1:04 PM Dovid Bender <dovid at telecurve.com> wrote:
>
>> Good ol UDP encrypted.
>>
>>
>>
>> On Mon, May 13, 2019 at 12:49 PM Brielle Bruns <bruns at 2mbit.com> wrote:
>>
>>>
>>> On 5/13/2019 10:20 AM, Dovid Bender wrote:
>>> > Thought of that. Customers have their own CPE's. So far the only thing
>>> > mutual here is that it's NTT -> VZ. Here is what I found so far
>>> looking
>>> > at two Polycom phones using non standard ports (e.g. not 5060)
>>> > 1) PhoneA tries to register multiple extensions and for each request
>>> we
>>> > send a 401. We expect to get back a REGISTER request with a no-once
>>> but
>>> > we don't. This happens for a while and then magically it starts
>>> working.
>>> > 2) PhoneB tries to register the time time as PhoneA and has no issues.
>>> >
>>> > At first I thought it was something possibly with the SIP call-ID but
>>> I
>>> > ruled that out since in the same SIP DIALOG it was not working then it
>>> > started. Also the seems to be per phone each phone is behind NAT and
>>> the
>>> > traffic is coming from a different NAT'd port. Seems like there is
>>> some
>>> > device in the middle that is randomly dropping traffic on specific
>>> sessions.
>>>
>>>
>>> Are you using TLS encrypted SIP or just plain ol' cleartext?
>>>
>>> If its encrypted, I'd look at possibly there being a MTU/MSS issue
>>> somewhere along the path possibly?
>>>
>>>
>>> --
>>> Brielle Bruns
>>> The Summit Open Source Development Group
>>> http://www.sosdg.org / http://www.ahbl.org
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/5ae2701b/attachment-0001.html>
------------------------------
Message: 13
Date: Mon, 13 May 2019 13:52:11 -0400
From: Pete Rohrman <prohrman at stage2networks.com>
To: <nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID: <688fc03b-8ab8-3380-7652-dadb15202d23 at stage2networks.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Dovid Bender,
I'm seeing the same sort of thing. Polycom phones. Multiple
customers getting to me from Verizon in NYC area. I'm seeing phones
register for a while, then drop off, then I see them trying to re-reg
resulting in your 401 below.
Call me. 212 497 8015. Let's look at this.
Pete
Pete Rohrman
Stage2 Support
212 497 8000, Opt. 2
On 5/13/19 12:20 PM, Dovid Bender wrote:
> Thought of that. Customers have their own CPE's. So far the only thing
> mutual here is that it's NTT -> VZ. Here is what I found so far
> looking at two Polycom phones using non standard ports (e.g. not 5060)
> 1) PhoneA tries to register multiple extensions and for each request
> we send a 401. We expect to get back a REGISTER request with a no-once
> but we don't. This happens for a while and then magically it starts
> working.
> 2) PhoneB tries to register the time time as PhoneA and has no issues.
>
> At first I thought it was something possibly with the SIP call-ID but
> I ruled that out since in the same SIP DIALOG it was not working then
> it started. Also the seems to be per phone each phone is behind NAT
> and the traffic is coming from a different NAT'd port. Seems like
> there is some device in the middle that is randomly dropping traffic
> on specific sessions.
>
>
>
>
>
> On Mon, May 13, 2019 at 11:40 AM Brielle Bruns <bruns at 2mbit.com
> <mailto:bruns at 2mbit.com>> wrote:
>
> On 5/13/2019 9:21 AM, Dovid Bender wrote:
> > Hi,
> >
> > Over the last 48 hours we have been getting a lot of alerts of
> customers
> > phones losing registrations to us. All the complaints are coming
> from
> > customers that are on VZ Fios in the NYC area. Anyone else see
> anything
> > strange going on?
> >
>
>
> While you are diagnosing, might check to make sure that the SIP
> ALG is
> disabled on all of their routers too.
>
>
>
> --
> Brielle Bruns
> The Summit Open Source Development Group
> http://www.sosdg.org / http://www.ahbl.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/adc79d9d/attachment-0001.html>
------------------------------
Message: 14
Date: Mon, 13 May 2019 15:11:03 -0400
From: <daniel at pyranah.com>
To: <nanog at nanog.org>
Subject: Charter and Cox contacts
Message-ID: <[email protected]>
Content-Type: text/plain; charset="utf-8"
Does anyone have contacts at Charter (Spectrum) and Cox? For some reason,
our IP has been blocked by them and our customers are unable to send email
via their charter/cox accounts. Thanks
Daniel Temple
VP of Operations
Pyranah Communications, LLC
828-743-2470 ext.205
<http://www.pyranah.com/> Pyranah.com
<https://www.facebook.com/cashiersvalleyelectronics/> Like us on Facebook!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/c0972bf8/attachment-0001.html>
------------------------------
Message: 15
Date: Mon, 13 May 2019 17:39:39 -0400
From: bzs at theworld.com
To: nanog at nanog.org
Cc: abuse at outlook.com
Subject: Mostly name and shame...
Message-ID: <23769.58395.683601.559454 at gargle.gargle.HOWL>
Content-Type: text/plain; charset=us-ascii
Why has OUTLOOK.COM allowed daily dictionary spammers to operate from
their net, FOR YEARS?
It can't be that hard to detect and block.
2019-05-13T17:00:18.194103-04:00 pcls6 sendmail[14128]: NOUSER: proctor5 relay=mail-eopbgr740053.outbound.protection.outlook.com [40.107.74.53]
2019-05-13T17:00:18.444848-04:00 pcls6 sendmail[14128]: NOUSER: proctor4 relay=mail-eopbgr740053.outbound.protection.outlook.com [40.107.74.53]
2019-05-13T17:00:18.698977-04:00 pcls6 sendmail[14128]: NOUSER: proctor2 relay=mail-eopbgr740053.outbound.protection.outlook.com [40.107.74.53]
2019-05-13T17:00:18.952548-04:00 pcls6 sendmail[14128]: NOUSER: proctor10 relay=mail-eopbgr740053.outbound.protection.outlook.com [40.107.74.53]
2019-05-13T17:10:27.523597-04:00 pcls6 sendmail[19471]: NOUSER: proctor6 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43]
2019-05-13T17:10:27.775984-04:00 pcls6 sendmail[19471]: NOUSER: proctor5 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43]
2019-05-13T17:10:28.029744-04:00 pcls6 sendmail[19471]: NOUSER: proctor4 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43]
2019-05-13T17:10:28.283016-04:00 pcls6 sendmail[19471]: NOUSER: proctor2 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43]
2019-05-13T17:10:28.537106-04:00 pcls6 sendmail[19471]: NOUSER: proctor10 relay=mail-eopbgr810043.outbound.protection.outlook.com [40.107.81.43]
2019-05-13T17:30:47.045677-04:00 pcls6 sendmail[31621]: NOUSER: proctor6 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72]
2019-05-13T17:30:47.299131-04:00 pcls6 sendmail[31621]: NOUSER: proctor5 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72]
2019-05-13T17:30:47.552492-04:00 pcls6 sendmail[31621]: NOUSER: proctor4 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72]
2019-05-13T17:30:47.804233-04:00 pcls6 sendmail[31621]: NOUSER: proctor2 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72]
2019-05-13T17:30:48.056635-04:00 pcls6 sendmail[31621]: NOUSER: proctor10 relay=mail-eopbgr810072.outbound.protection.outlook.com [40.107.81.72]
2019-05-13T17:35:05.867715-04:00 pcls6 sendmail[1352]: NOUSER: proctor9 relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127]
2019-05-13T17:35:06.120021-04:00 pcls6 sendmail[1352]: NOUSER: proctor7 relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127]
2019-05-13T17:35:06.372603-04:00 pcls6 sendmail[1352]: NOUSER: proctor6 relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127]
2019-05-13T17:35:06.627583-04:00 pcls6 sendmail[1352]: NOUSER: proctor5 relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127]
2019-05-13T17:35:06.885218-04:00 pcls6 sendmail[1352]: NOUSER: proctor relay=mail-eopbgr50127.outbound.protection.outlook.com [40.107.5.127]
etc etc etc etc.
--
-Barry Shein
Software Tool & Die | bzs at TheWorld.com | http://www.TheWorld.com
Purveyors to the Trade | Voice: +1 617-STD-WRLD | 800-THE-WRLD
The World: Since 1989 | A Public Information Utility | *oo*
------------------------------
Message: 16
Date: Mon, 13 May 2019 14:53:01 -0700
From: Randy Bush <randy at psg.com>
To: Christopher Morrow <morrowc.lists at gmail.com>
Cc: Pengxiong Zhu <pzhu011 at ucr.edu>, Keyu Man <kman001 at ucr.edu>, North
American Network Operators' Group <nanog at nanog.org>, Zhiyun Qian
<zhiyunq at cs.ucr.edu>, Zhongjie Wang <zwang048 at ucr.edu>
Subject: Re: Ownership of Routers on Both Ends of Transnational Links
Message-ID: <m236li58ea.wl-randy at psg.com>
Content-Type: text/plain; charset=US-ASCII
i suspect the OP is down the rabbit hole of what is known as
"anti-aliasing," trying to find out whether IP address A on some router
is actually on the same router as IP address B, and what AS(s) those IPs
are in. your point is that an inter-as link may have IPs from either of
the providers. yup. and, because it is an INTER-as link, it does not
really belong to one or t'other.
this particular rabbit digs deep holes. an early entrance to the burrow
is the classic from the uw crew
inproceedings{Spring:2002:MIT:633025.633039,
author = {Spring, Neil and Mahajan, Ratul and Wetherall, David},
title = {Measuring ISP Topologies with Rocketfuel},
booktitle = {Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications},
series = {SIGCOMM '02},
year = {2002},
isbn = {1-58113-570-X},
location = {Pittsburgh, Pennsylvania, USA},
pages = {133--145},
numpages = {13},
url = {http://doi.acm.org/10.1145/633025.633039},
doi = {10.1145/633025.633039},
acmid = {633039},
publisher = {ACM},
address = {New York, NY, USA},
}
randy
------------------------------
Message: 17
Date: Mon, 13 May 2019 15:29:43 -0700
From: Stephen Satchell <list at satchell.net>
To: nanog at nanog.org
Subject: Re: Charter and Cox contacts
Message-ID: <1d87ab31-ab93-d69b-be66-c29bf97dedb8 at satchell.net>
Content-Type: text/plain; charset=windows-1252
On 5/13/19 12:11 PM, daniel at pyranah.com wrote:
> Does anyone have contacts at Charter (Spectrum) and Cox? For some reason,
> our IP has been blocked by them and our customers are unable to send email
> via their charter/cox accounts. Thanks
Would you be talking about port 25/tcp outbound? Lots of ISPs will
block port 25 as a rule; you have to call to have it opened. At the
same time, you can request the PTR record you need to keep big mail
receivers happy.
------------------------------
Message: 18
Date: Mon, 13 May 2019 23:48:02 -0500
From: <frnkblk at iname.com>
To: "'Mel Beckman'" <mel at beckman.org>, "Mike Bolitho"
<mikebolitho at gmail.com>
Cc: <nanog at nanog.org>
Subject: RE: FCC Hurricane Michael after-action report
Message-ID: <[email protected]>
Content-Type: text/plain; charset="utf-8"
This webinar may be of some interest to those in this group:
https://www.fcc.gov/small-rural-communications-provider-network-resiliency-webinar
Here’s some additional color commentary on the FCC’s concerns:
https://urgentcomm.com/2019/05/10/backhaul-problems-disjointed-recovery-efforts-key-causes-of-unacceptable-extended-wireless-outage-after-hurricane-michael-fcc-report-says/
"“Uniti Fiber (Uniti) provides backhaul services to Verizon Wireless in Bay and Gulf Counties. Uniti indicates it experienced at least 33 separate fiber cuts during the recovery effort. These fiber cuts included damage to sections that already had been repaired. Commenters attributed fiber cuts to debris-removal crews, power-company restorations, and returning homeowners clearing their property.”
One of my takeaways from that article was that burying fiber underground could likely have avoided many/most of these fiber cuts, though I’m not familiar enough with the terrain to know how feasible that is.
Frank
From: NANOG <nanog-bounces at nanog.org> On Behalf Of Mel Beckman
Sent: Saturday, May 11, 2019 9:52 AM
To: Mike Bolitho <mikebolitho at gmail.com>
Cc: nanog at nanog.org
Subject: Re: FCC Hurricane Michael after-action report
This is what I tell outage complainers during natural disasters, such as the fires in California that recently took out a lot of power and communications:
“Stop whining about how long it is taking to repair your Internet, your cell phone service, or your cable TV. You didn’t pay anything extra to recover from natural disasters, and none of us in the field are getting paid anything extra to restore your services.
No, we don’t know how long it will take. It takes what it takes. That you don’t get instant gratification doesn’t make us incompetent. It makes you ungrateful.
It’s a natural disaster. These are not scheduled. Your outage is nobody’s fault. We don’t have a duty to mitigate all conceivable failures.
It takes time to repair. We’re not cheating you, or loafing around. We don’t owe you any special attention because of your status or reputation.
So quit whining and be thankful you’re alive, and hopefully you haven’t lost too much. Maybe pitch in and help those who have.“
I also send this to ignorant journalists and grandstanding politicians.
-mel via cell
On May 11, 2019, at 4:29 AM, Mike Bolitho <mikebolitho at gmail.com <mailto:mikebolitho at gmail.com> > wrote:
Trying not to get political, here goes...
Something important to keep in mind: The current administration has been getting slammed for their lack of response in the aftermath of Michael since the hurricane hit. A lot of that criticism revolves around communications infrastructure and FEMA's lack of assistance. The current administration has, time and time again, used federal agencies (specifically their presidential appointees) to defend the administration's actions or inactions. I have read the full report and it is more or less a thinly veiled hit piece. I'm not going to link them here (they are easy enough to find via Google) but there are several very good articles written by reputable tech journalists that go into greater detail responding to the report. Worth checking out.
I say all of that because most of us like to hate on telecom companies (many times rightly so) but I don't think they are entirely to blame here. There's nothing Verizon or AT&T can do if their backhaul is cut by a tree or some third party clean up crew. The report is a gross oversimplification of how telecommunication infrastructure works. I think anyone here that has ever worked a storm like this can attest to the complexity and difficulty you run into during recovery. Hanlon's Razor and all but this is the FCC and I would hope they would know better.
Speaking specifically to point 51, it's impossible to coordinate between the thousands of crews working to clean things up and repair physical infrastructure after a massive storm like this. Many of the people doing physical cleanup are volunteers that are fully independent of any governing body or company. It is not a telco's responsibility to know when and where those crews are working. Further, even if those crews we're calling in and letting each telco know exactly where they were, what does that provide other than an impossibly large and fluid dataset to parse for any meaningful information.
- Mike Bolitho
On Thu, May 9, 2019, 4:43 PM Sean Donelan wrote:
The FCC has released its report and analysis of Hurricane Michael impact
on communications: preparation, effect and recovery.
https://www.fcc.gov/document/fcc-releases-report-communication-impacts-hurricane-michael-0
Conclusions and Recommendations
51. Backhaul outages loomed large as an impediment to communications
recovery. Uncoordinated post-storm recovery efforts between and among
communications, utility, and debris removal teams created unnecessary
delays to a speedy return to service. Customers who had communications
service restored – only to lose it again almost immediately because of a
fiber cut – provide a clear example of how better cross-sector
coordination could have improved the restoration process.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/f13f13d1/attachment-0001.html>
------------------------------
Message: 19
Date: Mon, 13 May 2019 22:20:22 -0700
From: Mike Bolitho <mikebolitho at gmail.com>
Cc: nanog at nanog.org
Subject: Re: FCC Hurricane Michael after-action report
Message-ID:
<CACoNLryTaFJYDMUSL0duWxSKr913r6X=XrBN6d2c_K-NozYRbA at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
In Florida, especially the panhandle, it's not possible to bury it. The
water table is way too high.
On Mon, May 13, 2019, 9:47 PM <frnkblk at iname.com> wrote:
> This webinar may be of some interest to those in this group:
>
>
> https://www.fcc.gov/small-rural-communications-provider-network-resiliency-webinar
>
>
>
> Here’s some additional color commentary on the FCC’s concerns:
>
>
> https://urgentcomm.com/2019/05/10/backhaul-problems-disjointed-recovery-efforts-key-causes-of-unacceptable-extended-wireless-outage-after-hurricane-michael-fcc-report-says/
>
> "“Uniti Fiber (Uniti) provides backhaul services to Verizon Wireless in
> Bay and Gulf Counties. Uniti indicates it experienced at least 33 separate
> fiber cuts during the recovery effort. These fiber cuts included damage to
> sections that already had been repaired. Commenters attributed fiber cuts
> to debris-removal crews, power-company restorations, and returning
> homeowners clearing their property.”
>
> One of my takeaways from that article was that burying fiber underground
> could likely have avoided many/most of these fiber cuts, though I’m not
> familiar enough with the terrain to know how feasible that is.
>
> Frank
>
>
>
> *From:* NANOG <nanog-bounces at nanog.org> *On Behalf Of *Mel Beckman
> *Sent:* Saturday, May 11, 2019 9:52 AM
> *To:* Mike Bolitho <mikebolitho at gmail.com>
> *Cc:* nanog at nanog.org
> *Subject:* Re: FCC Hurricane Michael after-action report
>
>
>
> This is what I tell outage complainers during natural disasters, such as
> the fires in California that recently took out a lot of power and
> communications:
>
>
>
> “Stop whining about how long it is taking to repair your Internet, your
> cell phone service, or your cable TV. You didn’t pay anything extra to
> recover from natural disasters, and none of us in the field are getting
> paid anything extra to restore your services.
>
>
>
> No, we don’t know how long it will take. It takes what it takes. That you
> don’t get instant gratification doesn’t make us incompetent. It makes you
> ungrateful.
>
>
>
> It’s a natural disaster. These are not scheduled. Your outage is nobody’s
> fault. We don’t have a duty to mitigate all conceivable failures.
>
>
>
> It takes time to repair. We’re not cheating you, or loafing around. We
> don’t owe you any special attention because of your status or reputation.
>
>
>
> So quit whining and be thankful you’re alive, and hopefully you haven’t
> lost too much. Maybe pitch in and help those who have.“
>
>
>
> I also send this to ignorant journalists and grandstanding politicians.
>
> -mel via cell
>
>
> On May 11, 2019, at 4:29 AM, Mike Bolitho <mikebolitho at gmail.com> wrote:
>
> Trying not to get political, here goes...
>
>
>
> Something important to keep in mind: The current administration has been
> getting slammed for their lack of response in the aftermath of Michael
> since the hurricane hit. A lot of that criticism revolves around
> communications infrastructure and FEMA's lack of assistance. The current
> administration has, time and time again, used federal agencies
> (specifically their presidential appointees) to defend the administration's
> actions or inactions. I have read the full report and it is more or less a
> thinly veiled hit piece. I'm not going to link them here (they are easy
> enough to find via Google) but there are several very good articles written
> by reputable tech journalists that go into greater detail responding to the
> report. Worth checking out.
>
>
>
> I say all of that because most of us like to hate on telecom companies
> (many times rightly so) but I don't think they are entirely to blame here.
> There's nothing Verizon or AT&T can do if their backhaul is cut by a tree
> or some third party clean up crew. The report is a gross oversimplification
> of how telecommunication infrastructure works. I think anyone here that has
> ever worked a storm like this can attest to the complexity and difficulty
> you run into during recovery. Hanlon's Razor and all but this is the FCC
> and I would hope they would know better.
>
>
>
> Speaking specifically to point 51, it's impossible to coordinate between
> the thousands of crews working to clean things up and repair physical
> infrastructure after a massive storm like this. Many of the people doing
> physical cleanup are volunteers that are fully independent of any governing
> body or company. It is not a telco's responsibility to know when and where
> those crews are working. Further, even if those crews we're calling in and
> letting each telco know exactly where they were, what does that provide
> other than an impossibly large and fluid dataset to parse for any
> meaningful information.
>
>
>
> - Mike Bolitho
>
>
>
> On Thu, May 9, 2019, 4:43 PM Sean Donelan wrote:
>
>
> The FCC has released its report and analysis of Hurricane Michael impact
> on communications: preparation, effect and recovery.
>
>
>
> https://www.fcc.gov/document/fcc-releases-report-communication-impacts-hurricane-michael-0
>
> Conclusions and Recommendations
>
> 51. Backhaul outages loomed large as an impediment to communications
> recovery. Uncoordinated post-storm recovery efforts between and among
> communications, utility, and debris removal teams created unnecessary
> delays to a speedy return to service. Customers who had communications
> service restored – only to lose it again almost immediately because of a
> fiber cut – provide a clear example of how better cross-sector
> coordination could have improved the restoration process.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190513/9f6d6901/attachment-0001.html>
------------------------------
Message: 20
Date: Tue, 14 May 2019 09:03:30 +0300
From: Saku Ytti <saku at ytti.fi>
To: prohrman at stage2networks.com
Cc: "nanog at nanog.org" <nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID:
<CAAeewD_nOWcRVKf7_OM9Gs2NA4BvFvBFU36pMtpKmiEGV3SuqA at mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Can someone try to recreate the problem with TCP/5060. Or do iperf
test on equivalent ports with UDP+TCP, to determine if the problem is
related specifically to UDP.
Most networks have some form of limits to even transit traffic, UDP is
most typical L4 to have policers.
On Tue, 14 May 2019 at 00:12, Pete Rohrman <prohrman at stage2networks.com> wrote:
>
> Dovid Bender,
>
> I'm seeing the same sort of thing. Polycom phones. Multiple customers getting to me from Verizon in NYC area. I'm seeing phones register for a while, then drop off, then I see them trying to re-reg resulting in your 401 below.
>
> Call me. 212 497 8015. Let's look at this.
>
> Pete
>
> Pete Rohrman
> Stage2 Support
> 212 497 8000, Opt. 2
>
> On 5/13/19 12:20 PM, Dovid Bender wrote:
>
> Thought of that. Customers have their own CPE's. So far the only thing mutual here is that it's NTT -> VZ. Here is what I found so far looking at two Polycom phones using non standard ports (e.g. not 5060)
> 1) PhoneA tries to register multiple extensions and for each request we send a 401. We expect to get back a REGISTER request with a no-once but we don't. This happens for a while and then magically it starts working.
> 2) PhoneB tries to register the time time as PhoneA and has no issues.
>
> At first I thought it was something possibly with the SIP call-ID but I ruled that out since in the same SIP DIALOG it was not working then it started. Also the seems to be per phone each phone is behind NAT and the traffic is coming from a different NAT'd port. Seems like there is some device in the middle that is randomly dropping traffic on specific sessions.
>
>
>
>
>
> On Mon, May 13, 2019 at 11:40 AM Brielle Bruns <bruns at 2mbit.com> wrote:
>>
>> On 5/13/2019 9:21 AM, Dovid Bender wrote:
>> > Hi,
>> >
>> > Over the last 48 hours we have been getting a lot of alerts of customers
>> > phones losing registrations to us. All the complaints are coming from
>> > customers that are on VZ Fios in the NYC area. Anyone else see anything
>> > strange going on?
>> >
>>
>>
>> While you are diagnosing, might check to make sure that the SIP ALG is
>> disabled on all of their routers too.
>>
>>
>>
>> --
>> Brielle Bruns
>> The Summit Open Source Development Group
>> http://www.sosdg.org / http://www.ahbl.org
--
++ytti
------------------------------
Message: 21
Date: Tue, 14 May 2019 09:41:55 +0200
From: Mark Tinka <mark.tinka at seacom.mu>
To: nanog at nanog.org
Subject: Re: NTP for ASBRs?
Message-ID: <0c74f5cb-9299-e6f9-b0e6-46863ca96fdd at seacom.mu>
Content-Type: text/plain; charset=utf-8
On 9/May/19 02:47, Bryan Holloway wrote:
>
>
> Hawai'i and Arizona can add/subtract without looking at the damn
> calendar. I'm just sayin' I'd like to see more of that.
Well, 2 months ago, the EU parliament voted to scrap daylight saving
time from 2021. This would also apply to the UK if it chooses to remain
in the EU, or during the extended transition period that Theresa May is
currently working.
It's now up to the various EU member states to decide whether they want
to remain permanently in winter or permanently in summer.
Of course, the UK government aren't necessarily amused.
Mark.
------------------------------
Message: 22
Date: Tue, 14 May 2019 09:10:16 +0100
From: colin johnston <colinj at gt86car.org.uk>
To: Mark Tinka <mark.tinka at seacom.mu>
Cc: nanog at nanog.org
Subject: Re: NTP for ASBRs?
Message-ID: <DD2322BD-6FA2-4F84-BB58-A33B81B9DCEA at gt86car.org.uk>
Content-Type: text/plain; charset=us-ascii
> On 14 May 2019, at 08:41, Mark Tinka <mark.tinka at seacom.mu> wrote:
>
>
>
> On 9/May/19 02:47, Bryan Holloway wrote:
>
>>
>>
>> Hawai'i and Arizona can add/subtract without looking at the damn
>> calendar. I'm just sayin' I'd like to see more of that.
>
> Well, 2 months ago, the EU parliament voted to scrap daylight saving
> time from 2021. This would also apply to the UK if it chooses to remain
> in the EU, or during the extended transition period that Theresa May is
> currently working.
>
> It's now up to the various EU member states to decide whether they want
> to remain permanently in winter or permanently in summer.
>
> Of course, the UK government aren't necessarily amused.
>
> Mark.
>
Dst Time works great for Scotland as allows kids to go to school during lighter hours.
It has been proved to save road deaths
Col
------------------------------
Message: 23
Date: Tue, 14 May 2019 11:31:57 +0200
From: Mark Tinka <mark.tinka at seacom.mu>
To: "nanog at nanog.org" <nanog at nanog.org>
Subject: Fwd: [afnog] SAFNOG-5 & iWeek 2019 Registrations Open
Message-ID: <a58f9917-1c39-1dbf-bf23-bb993719c81f at seacom.mu>
Content-Type: text/plain; charset="utf-8"
FYI.
Mark.
-------- Forwarded Message --------
Subject: [afnog] SAFNOG-5 & iWeek 2019 Registrations Open
Date: Mon, 13 May 2019 11:06:53 +0000
From: Portia Rabonda <portia.rabonda at safnog.net>
To: afnog at afnog.org <afnog at afnog.org>
Greetings All,
It gives us great pleasure to announce that the SAFNOG-5 & iWeek 2019
event registration is now open!
SAFNOG, in collaboration with iWeek, is proud to announce that this
year’s event registration is free of charge. SAFNOG-5 and iWeek 2019
will be held between the 26th- 28th August, 2019, in Johannesburg, South
Africa.
SAFNOG-5 seats are limited to 120 people, to reserve your spot, register
through our website: http://safnog.org/. Please take note to select
“SAFNOG-5” as the main event of attendance to secure your free seat.
Information about the event programme, the venue, travel information and
other important updates will be made available on http://safnog.org/ in
due time.
We are officially 105 days from the big showdown in Jozi!
We look forward to seeing you there!
Regards,
Portia Rabonda on Behalf of SAFNOG MC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190514/ba3116d6/attachment-0001.html>
-------------- next part --------------
_______________________________________________
afnog mailing list
https://www.afnog.org/mailman/listinfo/afnog
------------------------------
Message: 24
Date: Tue, 14 May 2019 07:48:25 -0400
From: Dovid Bender <dovid at telecurve.com>
To: Saku Ytti <saku at ytti.fi>
Cc: Peter Rohrman <prohrman at stage2networks.com>, "nanog at nanog.org"
<nanog at nanog.org>
Subject: Re: Issues with SIP packets between VZ Fios and NTT
Message-ID:
<CAM3TTh2+4ZXjgYLXrFMxazudrmkOEE9W8DDk+3QWRJ2NtNaeJw at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
It's not strictly UDP. I spoke with someone yesterday that was re-producing
it with curl.
On Tue, May 14, 2019 at 2:04 AM Saku Ytti <saku at ytti.fi> wrote:
> Can someone try to recreate the problem with TCP/5060. Or do iperf
> test on equivalent ports with UDP+TCP, to determine if the problem is
> related specifically to UDP.
>
> Most networks have some form of limits to even transit traffic, UDP is
> most typical L4 to have policers.
>
> On Tue, 14 May 2019 at 00:12, Pete Rohrman <prohrman at stage2networks.com>
> wrote:
> >
> > Dovid Bender,
> >
> > I'm seeing the same sort of thing. Polycom phones. Multiple
> customers getting to me from Verizon in NYC area. I'm seeing phones
> register for a while, then drop off, then I see them trying to re-reg
> resulting in your 401 below.
> >
> > Call me. 212 497 8015. Let's look at this.
> >
> > Pete
> >
> > Pete Rohrman
> > Stage2 Support
> > 212 497 8000, Opt. 2
> >
> > On 5/13/19 12:20 PM, Dovid Bender wrote:
> >
> > Thought of that. Customers have their own CPE's. So far the only thing
> mutual here is that it's NTT -> VZ. Here is what I found so far looking at
> two Polycom phones using non standard ports (e.g. not 5060)
> > 1) PhoneA tries to register multiple extensions and for each request we
> send a 401. We expect to get back a REGISTER request with a no-once but we
> don't. This happens for a while and then magically it starts working.
> > 2) PhoneB tries to register the time time as PhoneA and has no issues.
> >
> > At first I thought it was something possibly with the SIP call-ID but I
> ruled that out since in the same SIP DIALOG it was not working then it
> started. Also the seems to be per phone each phone is behind NAT and the
> traffic is coming from a different NAT'd port. Seems like there is some
> device in the middle that is randomly dropping traffic on specific sessions.
> >
> >
> >
> >
> >
> > On Mon, May 13, 2019 at 11:40 AM Brielle Bruns <bruns at 2mbit.com> wrote:
> >>
> >> On 5/13/2019 9:21 AM, Dovid Bender wrote:
> >> > Hi,
> >> >
> >> > Over the last 48 hours we have been getting a lot of alerts of
> customers
> >> > phones losing registrations to us. All the complaints are coming from
> >> > customers that are on VZ Fios in the NYC area. Anyone else see
> anything
> >> > strange going on?
> >> >
> >>
> >>
> >> While you are diagnosing, might check to make sure that the SIP ALG is
> >> disabled on all of their routers too.
> >>
> >>
> >>
> >> --
> >> Brielle Bruns
> >> The Summit Open Source Development Group
> >> http://www.sosdg.org / http://www.ahbl.org
>
>
>
> --
> ++ytti
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20190514/060b4add/attachment-0001.html>
End of NANOG Digest, Vol 136, Issue 14
**************************************
More information about the NANOG
mailing list