[Paper] B4: Experience with a Globally-Deployed Software Defined

Sun Aug 18 01:08:31 UTC 2013

> On Sat, Aug 17, 2013 at 2:32 PM, Avi Freedman <freedman at freedman.net> wrote:
> 
> > No, people never use *flow controllers* for anything.
>
> > People have been doing SDN since before Google was around.
> > OK, so it was horrible expect scripts but it worked.
>
> Not really.

Note I am talking about flow controllers in my first point.

(And I was trying to be funny to match Todd's tone, though
 I guess it's dangerous to try to copy the master)

Re: flow controllers -

The idea of centralized decision makers doing something (typically
per flow) has been proposed, in my experience, by those with little
operational experience or those with extraordinarily constrained
topoligies, types of traffic, and usually external filtering to
constrain the types of traffic one could face.

Because...  There have been no proposals that I have seen (or
that those who are at the Major Vendors who follow it more 
closely tell me about when I ask a few times/year) to actually
deal with the every-packet-is-a-flow problem we saw first with
7206VXRs and that remain a real possibility for Internet-
connected networks.  Distributing flow controllers and making
them hierarchical doesn't seem to help in the architectures
that I've seen proposed.

So it seems to be of use only for very tiny networks or for
very constrained and filtered or non Internet-connected topologies.

I'd be interested to be shown otherwise.

> Automatic reconfiguration of routers is not what a software-defined network is.
>
> It is  one of the things (but not all of the things)  that SDN provides.
> 
> A software defined network is one where the forwarding behavior can be
> completely defined
> in software running outside of the devices that perform the forwarding.

That said, I wince every time someone starts talking about (not suggesting
you are here but many do) making the routing engineer or designer in a box
that sits on the bottom or besides the network.

Those who have experience and/or run larger infrastructure usually say
words like "of course we have to worry about feedback loops" but many don't.

I think innovation is great but I don't think there are that many shops
that are better off writing their own control pane (centralized, distribtued,
whatever) right now.

It's worth remembering that Google is a software company.  They are far ahead 
in software defined everything. 

> You can write expect scripts all day; but you cannot turn your basic switch
> into a Load balancer  or stateful firewall with one.
> or decide in real time exactly which destination Ethernet ports a packet
>  coming in a certain port is going to touch,  without having structured
> VLANs and  static MAC tables on the switches ahead of time.
> 
> Changing device configurations with expect scripts is a limited tiny subset
> of what SDN is.

True, but the number of production environments that are going to be more
stable or scalable by having people build their own control logic is pretty
small in my experience.  

And being able to debug and reach out to a community of operators with
a common set of experience of what to do, not to do, and how to debug
is extraordinarily valuable for production networks.

When I look at most of the non-Google big guys, SDN means pushing the
vendors for better control plane instrumentation and ability to program
(but more on the instrumentation side as where the gaps have been), and
potentially to get some cross-provider way of doing the above.

+ having merchant silicon one can get/use for cheap, typically for more
constrained topologies, doing pretty dumb switching and/or routing stuff.

> -JH

Where I see the delta a lot given the customer conversations I have is
in the magic provisioning of cloud network infrastructures.

New school SDN is that everything is a tunnel, magic software maps things,
commercial providers doing this uniformly have to aggressively rate-
limit their clients, and performance for content delivery is limited 
because the hypervisors must be briding and can't do PCI passthrough or
SR/IOV.

Old school SDN (not really that old school) is API-based provisioning
of network devices with vendor support (let's say Juniper) to do 
filtering, VLANs, and shaping and tunnels where needed.  

It'll definitely be interesting to see where things go over the next
few years.

I know tens of companies who have run away from cloud providers with
new(er) school SDN-ish infrastructures for the simplicity of just
having some high performance dedicated machines/hypervisors with
dead simple switching infrastructure.

Anyway, innovation is great but I just see few companies with the
understanding to go build their own control plane software to 
connect to the Internet with.  And those vendors who do build it
will get borged by one of the routing/switching vendors and things
will become product features, differentiated by providers, most
likely.  (Though I hope not)

Avi