FYI Netflix is down

Tony McCrory tony.mccrory at
Mon Jul 2 18:53:32 UTC 2012

On 2 July 2012 19:20, Cameron Byrne <cb.list6 at> wrote:

> Make your chaos animal go after sites and regions instead of individual
> VMs.
> CB

>From a previous post mortem

Create More Failures
Currently, Netflix uses a service called "Chaos
to simulate service failure. Basically, Chaos Monkey is a service that
kills other services. We run this service because we want engineering teams
to be used to a constant level of failure in the cloud. Services should
automatically recover without any manual intervention. We don't however,
simulate what happens when an entire AZ goes down and therefore we haven't
engineered our systems to automatically deal with those sorts of failures.
Internally we are having discussions about doing that and people are
already starting to call this service "Chaos Gorilla".

It would seem the Gorilla hasn't quite matured.


More information about the NANOG mailing list