Quality of User Experience (was RE: image stream routers)

Sat Sep 17 07:54:23 UTC 2005

[let me preface this by saying that if you don't know this already, i do 
happen to work for a router vendor]

from the perspective of ANY router, "quality of end-user experience" is 
not something which fits into layers 1-7 - its a layer 8-10 thing. 
however, having said that, certainly routers "doing the wrong thing" can 
definitely negatively impact end-user experience.

i think its best to answer this by what 'role' various routers have, and 
what their primary function should be.  that ultimately determines what 
the right 'boxes' are for given 'roles' - and if you put the wrong box 
in the wrong category, that can negatively impact service in a way that 
customers think your service sucks.

no doubt there are more roles than this & we can get more & more 
specific - but its my AU$0.02 worth:

note that i'm deliberately not getting into whether it should be IPv4, 
IPv6, MPLS, all of the above, none of the above .. thats up to what 
service you have, how you provision it and how you traffic engineer it.

1. Core router
	- generally consist of interface speeds of OC12 upwards.
	- move packets from A to B with minimal additional latency
	  and minimal jitter
	- should be capable of implementing ACLs with no
	  performance degredation but primary role is to push packets

	- just about mandatory these days that can handle interfaces
	  pushing maximum packets/sec with minimum packet size so
	  as to be able to withstand DDoS attacks - either at it, or
	  through it

2. Transit or peering-facing router
	- interface speeds of >OC3, probably decent GbE density
	  desirable
	- mandatory implementation of ACLs
	- mandatory full-feature BGP features & widgets
	- mandatory implementation of uRPF or similar
	- ideally be capable of traffic 'accounting' mechanisms
	  (e.g. packet-sampling, netflow etc)

3. customer-facing router (FR/ATM/..)
	- decent system-density for customer connections
	- GbE uplink interface(s)
	- mandatory implementation of ACLs
	- mandatory full-feature BGP features & widgets
	- mandatory implementation of uRPF or similar
	- ideally be capable of traffic 'accounting' mechanisms
	  (e.g. packet-sampling, netflow, anomoly detection etc)
	- ideally be able to implement 'better' queueing mechanisms
	  than just standard FIFO.  e.g. low-latency queueing for
	  voice traffic, fair-queueing for fairness, deep(er) buffers
	  to attempt to minimize packet drop due to speed-mismatch

4. broadband aggregation router (e.g. LNS)
	- handle large numbers of logical sessions from central
	  configuration/policy (e.g. tie into RADIUS server(s))
	- GbE uplink interface(s)
	- mandatory implementation of ACLs
	- mandatory implementation of uRPF or similar
	- ideally be capable of traffic 'accounting' mechanisms
	  (e.g. packet-sampling, netflow, anomoly detection etc)
	- ideally be able to implement 'better' queueing mechanisms
	  than just standard FIFO.  e.g. low-latency queueing for
	  voice traffic, fair-queueing for fairness, deep(er) buffers
	  to attempt to minimize packet drop due to speed-mismatch
	- sufficient control-plane CPU to handle large # of connection
	  establishments/sec (e.g. connection to LAC being lost)

5. customer-premises router (CPE)
	- generally low-speed (<30Mbps)
	- end-users love ones with built-in NAT, DHCP, firewall,
	  wireless, probably VoIP, ...
	- low-cost
	- minimal CPU - no need to handle DoS attack because WAN
	  bandwidth is exhausted before PPS limit of CPU is hit

going through these, i'd say "ASIC based" or multiple-distributed-CPU is 
  what you want for (1).  anything less than that means you're likely to 
have reduced customer satisfaction.

(2), (3) & (4) generally are a mix of s/w and h/w-based routers - 
architectures vary quite greatly but with silicon developments in the 
last few years, most semi-recent products are typically a combination of 
h/w and s/w with (ideally) the work split 90/10.  or 99/1.  or 100/0 in 
an ideal world.

(5) can stay software.

cheers,

lincoln.

Christopher J. Wolff wrote:
> Thanks for the thoughtful response.
> 
> One of the network architecture issues I'm always trying to gauge and get my
> arms around is what I'll call, "Quality of user experience."  In other
> words, what mix of network hardware, software, customer support, and
> management will create a perception that the network is performing at
> maximum efficiency.
> 
> Although the perception of network performance is entirely subjective there
> are some factors that I'm sure we can all agree contribute to overall
> satisfaction...i.e.
> 
> -WAN link latency.
> -Packet Loss.
> -Consistency in packet generation/serialization (A packet always enters
> interface A and leaves interface B in .5 ms)
> 
> So, if all other elements (software, customer support, and management) are
> equal, what router hardware architecture will contribute to a positive or
> negative user experience?  In other words, if the routing device between my
> workstation and server is a Juniper M7 instead of Pentium IV running
> unix-flavor-of-the-day, how will that affect the quality of user experience?
> 
> Thank you,
> Christopher
> 
> -----Original Message-----
> From: owner-nanog at merit.edu [mailto:owner-nanog at merit.edu] On Behalf Of
> Lincoln Dale
> Sent: Friday, September 16, 2005 11:18 PM
> To: Christopher J. Wolff
> Cc: nanog at merit.edu
> Subject: Re: image stream routers
> 
> 
> Christopher J. Wolff wrote:
>> I'd be interested to know the relative pros and cons of switching packets
> in
>> software (Imagestream) versus handing them off to a dedicated ASIC (Cisco,
>> Juniper)
> 
> [without having looked at Imagestream in any way, shape or form..]
> 
> it would be _unlikely_ that any router vendor that wants to support >OC3 
> could do so with the 'standard' (non-modified) linux IP stack.  if they 
> are modifying the 'standard' linux IP stack then its very unlikely that 
> one could do so without having to publish the source-code to it.  (i.e. 
> as per GPL).
> 
> 'standard' linux on standard hardware isn't capable of much more than 
> 100K PPS.  sure - some folks have a few hundred packets/sec - but these 
> are minimalist versus the demonstrated performance of ASIC-based 
> forwarding, typically 30M-50M PPS.
> 
> one advantage of software is programmability.  if there is a bug you can 
> fix it.
> if there is a bug in an ASIC, it may or may not be possible to fix it - 
> it depends on awful lot on how the ASIC is built (whether its 100% fixed 
> functionality or supports limited programmability in various stages of 
> the forwarding pipeline).
> it may be that its not fixable but that the ASIC allows 
> software-workarounds - in essence, 'fixing' something by diverting it to 
> a (slower) software-path.
> 
> note that there is a correction to make here: not all routers _ARE_ 
> ASIC-based for forwarding.  in fact, most of the Cisco /router/ product 
> portfolio isn't hardware-forwarding based.  generally speaking it isn't 
> necessary - UNTIL you get to the point of having interface speeds & 
> number-of-interfaces which exceed the capabilities of general-purpose 
> processors.  that is, typically somewhere between 100K PPS and 1M PPS.
> 
> 
> cheers,
> 
> lincoln.
> 
> 