FCC Outage Reports ..(.was Verizon outage in Southern California?)

Fri Oct 21 04:52:01 UTC 2005

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thinking out loud.

I guess some sort of trust model would help similar to what nsp-sec has
in place (not sure its current state).

It could be nice if there was some sort of a consensus among this
consortium to distribute executive health metrics with the help of some
secure trusted monitoring mechanism or maybe push model to a central
database of some sort.

Like to hear more thoughts as well.

regards,
/virendra

Wallace Keith wrote:
> I wasn't thinking in terms of  automatic monitoring, that would open up
> a can of worms security wise.
>  Just looking at some way of getting the manual reporting (that is still
> taking place to the FCC) back in the (semi?)public domain. Due to
> terrorism concerns, that information is no longer available online. I'm
> pretty sure the LEC's and IXC's like it that way also, as they no longer
> have to air their dirty laundry. I was able to get some information
> under the Freedom of Information act for an outage that affected me
> directly , but that takes days or weeks. As close to real-time
> information as possible is what's needed to assess and respond to a
> major outage, i.e. routing voice/data via different carriers, being able
> to explain to end users why their email or phone calls didn't go through
> , etc. and eliminating the need to open tons of trouble tickets during a
> major event.  One master ticket - such as fiber cut affect xxx OC48's
> would suffice.
> Not sure how this can be balanced against DHS perceived needs
> though...any suggestions?
> 
> -----Original Message-----
> From: Vicky Rode [mailto:vickyr at socal.rr.com] 
> Sent: Wednesday, October 19, 2005 5:45 PM
> To: Wallace Keith
> Subject: Re: Verizon outage in Southern California?
> 
> I wonder how would Telcos, ISPs and GOV agencies feel about a third
> party polling their devices, not to mention security.
> 
> 
> I think netcarft comes close to what you're suggesting.
> 
> 
> regards,
> /virendra
> 
> 
> 
> Wallace Keith wrote:
> 
>>>All this speculation!!
>>>Remember the good old days when you could see faxes of FCC outage 
>>>reports online?
>>>Was sure nice to know what was going on, before the FCC took these 
>>>offline (due to DHS?) It would really by nice to have some sort of an 
>>>online clearing house, and gain some visibility again into overall 
>>>network status. This will become even more important as things 
>>>continue to converge. DACS and DC Power failures tend to affect 
>>>multiple services and in the case of power,  multiple carriers that 
>>>are colo'd in the CO.
>>>-Keith
>>>
>>>-----Original Message-----
>>>From: owner-nanog at merit.edu [mailto:owner-nanog at merit.edu] On Behalf 
>>>Of Vicky Rode
>>>Sent: Wednesday, October 19, 2005 1:29 PM
>>>To: wb8foz at nrk.com
>>>Cc: nanog list
>>>Subject: Re: Verizon outage in Southern California?
>>>
>>>
>>>I wonder what ever happened to redundancy? I guess 5 9s (dunno what 
>>>the going number is) got blown out of the water for them.
>>>
>>>
>>>
>>>regards,
>>>/virendra
>>>
>>>David Lesher wrote:
>>>
>>>
>>>>>Speaking on Deep Background, the Press Secretary whispered:
>>>>>
>>>>>
>>>>>
>>>>>>I'm not completely familiar with the telco jargon.
>>>>>>Does Tandem mean the same as a local central office, where POTS 
>>>>>>lines terminate at the switch? Long Beach has a population of 
>>>>>>470,000. The C/Os I know of are:
>>>>>
>>>>>
>>>>>
>>>>>A "Central Office" switch talks to subscribers aka end-users. 
>>>>>On its backside, it talks to other CO's and tandems. Time was, that 
>>>>>was also VF copper pairs, but it's long since all
>>>>>DS1 and up.....
>>>>>
>>>>>A tandem is a switch that talks not to subs, but only to CO's. In 
>>>>>days
>>>
>>>
>>>>>of old, when a {dialup} call went to the other side of town, chances 
>>>>>are it went you-yourCO-downtown tandem-joesCO-joe. {copper all the 
>>>>>way...}.
>>>>>
>>>>>A tandem was always housed in large CO building, but might have been 
>>>>>ATT's vice the operationg company, etc...
>>>>>
>>>>>But ESS's and ""classless switching"" and massive expansion of the 
>>>>>plant really muddled the picture. An ESS could be both a CO switch 
>>>>>[for multiple prefixes and even multiple NPA's..] AND act like a 
>>>>>tandem.. And oh, the actual "line cards" can be remoted 100 miles 
>>>>>away
>>>
>>>
>>>>>in a horz. phonebooth box alongside the road in Smallville....
>>>>>with DS1's/OC coming back. 
>>>>>
>>>>>My guess is a DACS, a cross-connect point that is an software-driven 
>>>>>patch panel, lost its marbles. [engineering term of art.....] A DACS 
>>>>>could have dozen->MANY dozen DS1/DS3/OC-n going hither and yon. Some 
>>>>>will be leased circuits. Others will be the CO trunks going from one 
>>>>>switch to another. It may/may not have muxes internal, so that what 
>>>>>arrives on a DS1 leaves in a OC96..
>>>>>
>>>>>I note it went down at 2:20 AM. That SCREAMS software
> 
> upgrade/cutover.
> 
>>>
>>>>>What's to bet GEE, no...VZEEE, was doing just that and there was a 
>>>>>major ohshit.
>>>>>
>>>>>Sean noted a long while back that somehow, DACS crashes always seem 
>>>>>to
>>>
>>>
>>>>>take hours to recover. Maybe the backups are on Kansas City standard 
>>>>>tapes, I donno.. but this sounds like that..
>>>>>
>>>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (MingW32)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFDWHPxpbZvCIJx1bcRAv/DAJwPg4J5CNSZc6z0kTv3UwFKHBKq7QCePH2M
As5MkIaZrrH0N+XnxT7oVAI=
=G/S3
-----END PGP SIGNATURE-----