FYI Netflix is down

Tony McCrory tony.mccrory at gmail.com
Mon Jul 2 18:53:32 UTC 2012


On 2 July 2012 19:20, Cameron Byrne <cb.list6 at gmail.com> wrote:

>
> Make your chaos animal go after sites and regions instead of individual
> VMs.
>
> CB
>

>From a previous post mortem
http://techblog.netflix.com/2011_04_01_archive.html

"
Create More Failures
Currently, Netflix uses a service called "Chaos
Monkey<http://techblog.netflix.com/2010/12/5-lessons-weve-learned-using-aws.html>"
to simulate service failure. Basically, Chaos Monkey is a service that
kills other services. We run this service because we want engineering teams
to be used to a constant level of failure in the cloud. Services should
automatically recover without any manual intervention. We don't however,
simulate what happens when an entire AZ goes down and therefore we haven't
engineered our systems to automatically deal with those sorts of failures.
Internally we are having discussions about doing that and people are
already starting to call this service "Chaos Gorilla".
*"*

It would seem the Gorilla hasn't quite matured.

Tony



More information about the NANOG mailing list