Newbie network upgrade question, apologies in advance to NANOG

Thu Jul 3 20:20:32 UTC 2003

This isn't the exact same situation, but in a similar vein...I'll offer this real-world experience.

I once worked for a client that had 2 full t1's, and a 768k t1 in a multilink bundle.  The fragmentation and reordering done by PPP was keeping the CPU load at %80+, and the jitter was terrible.  Breaking the bundle and dedicating the 768k for videoconferencing and doing per-packet load sharing on the 2 x t1 for data fixed all the probelms they were having.  (It also increased the data throughput for backups significantly.)

-Ejay

	-----Original Message----- 
	From: Vandy Hamidi [mailto:vandy.hamidi at markettools.com] 
	Sent: Wed 7/2/2003 3:35 PM 
	To: prue 
	Cc: nanog at merit.edu 
	Subject: RE: Newbie network upgrade question, apologies in advance to NANOG

	I would agree only under certain limited situations.
	Per packet load balancing COULD increase jitter, and if you're running VOIP (or similar protocols) could degrade performance.  It could also affect TCP performance (on OSes not SACK enabled) as well.
	This would only really happen if you're T1's are near capacity (~above 80% or so).  Near when queues start causing noticeable delays.

	If were talking about 2 identically configured T1's, on the same router, through the same loop provider, connected to one ISP--I highly doubt a situation where packet reordering would arise.  It's not impossible, but unlikely as all the circuits would be utilized the same, thus queue delays should be similar across the board.

	I've done this on a private network with 4 T1's and never had a problem.  We were pushing 100GB database dumps across it and performance did quadruple over the single T1.

	        -=Vandy=-

	-----Original Message-----
	From: prue [mailto:prue at usc.edu]
	Sent: Wednesday, July 02, 2003 1:21 PM
	To: Vandy Hamidi
	Cc: nanog at merit.edu
	Subject: RE: Newbie network upgrade question, apologies in advance to
	NANOG

	Vandy,

	>Also, you may want to set your border router (the one with the serials to your
	>ISP) to route "per packet" as opposed to allowing the routes to cache.  This
	>will distribute the bandwidtch evenly across your T1's.  If you don't, then a
	>single high traffic session or destination can consume an uneven amount of
	>bandwidth on one of your lines.  You can ask your ISP to do this as well for
	>incoming packets.

	That is not such a good idea generally.  If you do this then you get packet
	reordering.  This can be detrimental to TCP performance.

	Walt