Rogers Outage Canada
Jon Sands
fohdeesha at gmail.com
Mon Jul 11 09:40:53 UTC 2022
Given the outage was so bad it was disrupting select E911 services
nationwide for something like 24+ hours, it's great to see such an in
depth analysis and plan of action to prevent such things in the future.
bravo rogers
On 7/10/2022 2:55 PM, L F wrote:
> fyi - see BOLD.
>
> A Message from Rogers President and CEO
>
>
>
> Dear Valued Customer,
>
> As you know, we experienced a service outage across the Rogers, Fido,
> Chatr and Cityfone wireless networks on Friday.
>
> I am reaching out to share that our services have been restored, and
> our networks and systems are close to fully operational. Our technical
> teams are continuing to monitor for any remaining intermittent issues.
> I also want to outline an action plan we are putting in place to
> address what happened.
>
> I want to share what we know about what happened on Friday. We now
> believe *we’ve narrowed the cause to a network system failure
> following a maintenance update in our core network, which caused some
> of our routers to malfunction. We disconnected the specific equipment
> and redirected traffic,*which allowed our network and services to come
> back online over time as we managed traffic volumes returning to
> normal levels.
>
>
>
>
> On Sat, Jul 9, 2022 at 9:09 PM L F <liz.fazekas at gmail.com> wrote:
>
>
> Lest we ALL MOVE ON….
>
>
> Yes he said
>
> “RETARD” =
>
> The delay in processing the current status of a Situation…
>
>
> Lets move on to bigger n scarier Post mortems:
>
>
> Was This or Was this NOT
>
> A CYBERATTACK ???
>
> Done.
>
> Lets deal with REAL threats not PC references.
>
> Time to evaluate n re eval Our NTS!!
>
> Veni vidi vici 2022
>
>
>
>
> On Sat, Jul 9, 2022 at 5:01 PM Eric Kuhnke <eric.kuhnke at gmail.com>
> wrote:
>
> Can we have a discussion with the list admins about a list
> member appending a threat of violence to their outbound
> emails? Whether serious or not.
>
> Does this person need directions to some local mental
> healthcare resources?
>
>
> On Sat, 9 Jul 2022 at 08:48, Keith Medcalf
> <kmedcalf at dessus.com> wrote:
>
>
> >I can't either, but the reality right now seems to be
> that 911 calls are
> >failing for anyone on a Rogers cellphone.
>
> This is par for the course. These people chose to deal
> with Rogers despite knowing the consequences. It is like
> if you bought a Rogers Snowblower and it did not work.
> That would mean that people who bought the Rogers
> Snowblower will not be using it to get rid of the snow
> that is preventing them from leaving their house.
>
> Mutatis mutandis when Rogers is down things that are
> Rogers dependent will not work.
>
> Some people are so retarded it is astonishing!
>
> --
> (CAUTION) You are advised that if you attack my person or
> property, you will be put down in accordance with the
> provisions of section 34 & 35 of the Criminal Code
> respectively. If you are brandishing (or in possession)
> of a weapon then lethal force will be applied to your
> person in accordance with the law. This means that your
> misadventures may end in your death. Consider yourself
> cautioned and govern your actions appropriately.
>
> >-----Original Message-----
> >From: NANOG <nanog-bounces+kmedcalf=dessus.com at nanog.org>
> On Behalf Of
> >Eric Kuhnke
> >Sent: Friday, 8 July, 2022 13:34
> >To: jim deleskie <deleskie at gmail.com>
> >Cc: NANOG list <nanog at nanog.org>
> >Subject: Re: Rogers Outage Canada
> >
> >
> >I have seen anecdotal reports that the mobile network is
> in a half broken
> >state that phones remain registered to, so a 911 call
> will attempt and
> >then fail.
> >
> >
> >This is unlike what would happen if you had a US/Canada
> cellphone with
> >battery power but no SIM card in it that would search for
> any available
> >network in RF range for a 911 call if needed.
> >
> >
> >On Fri, 8 Jul 2022 at 12:31, jim deleskie <deleskie at gmail.com
> ><mailto:deleskie at gmail.com> > wrote:
> >
> >
> > i cant see BGP taking out SS7.
> >
> > -jim
> >
> > On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004
> ><greenjosh6499 at gmail.com <mailto:greenjosh6499 at gmail.com>
> > wrote:
> >
> >
> > According to Cloudflare Radar
> ><https://radar.cloudflare.com/asn/812?date_filter=last_24_hours>
> , Rogers
> >BGP announcements spiked massively to levels 536,777%
> higher than normal
> >(343,601 vs 64 normally) just minutes before the outage.
> I would not be
> >surprised if this happened to be the culprit.
> >
> > Regards,
> > Josh Green
> >
> > On Fri, Jul 8, 2022 at 2:19 PM Andrew
> Paolucci via NANOG
> ><nanog at nanog.org <mailto:nanog at nanog.org> > wrote:
> >
> >
> > In the early hours of the morning
> around 2-3am my modem
> >got hit with a configuration update that caused a DHCP
> release that
> >wasn't renewed for about two hours, after rollback the
> connection was
> >fine for 3 hours before this network wide outage.
> >
> >
> > Maybe a failed night time update
> was attempted again
> >during office hours, I've heard daytime guys are still
> WFH and night
> >shift is in building.
> >
> >
> > I expect we'll never get a real
> explanation. Rogers is
> >notorious for withholding any type of helpful or
> technical information.
> >
> >
> > Sent from my inoperable Rogers
> Mobile via emergency eSIM.
> >
> >
> > Regards,
> >
> > Andrew Paolucci
> > -------- Original Message --------
> > On Jul. 8, 2022, 1:48 p.m., Jay
> Hennigan < jay at west.net
> ><mailto:jay at west.net> > wrote:
> >
> >
> > On 7/8/22 07:44, Robert
> DeVita wrote: > Does anyone
> >have information on a widespread Rogers outage in Canada.
> I > have
> >customers with multiple sites down. There's discussion on
> the Outages
> >mailing list. Seems widespread, affecting all services,
> mobile, voice,
> >Internet. No cause or ETR posted yet. -- Jay Hennigan -
> jay at west.net
> ><mailto:jay at west.net> Network Engineering - CCIE #7880
> 503 897-8550 -
> >WB6RDV
> >
> >
> >
> > --
> >
> > Josh Green.
>
>
>
>
>
--
Jon Sands
MFI Labs
https://fohdeesha.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20220711/687b071a/attachment.html>
More information about the NANOG
mailing list