100G QSFP28 DAC cables - experience

Thu Sep 14 20:29:25 UTC 2017

They're pretty fragile assemblies too,  I ruined about 30 of them lacing them in,  they need fish-paper around each cable so you don't crush the conductors when lacing. 

-----Original Message-----
From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of Brant Ian Stevens
Sent: Thursday, September 14, 2017 2:05 PM
To: Tyler Conrad
Cc: NANOG
Subject: Re: 100G QSFP28 DAC cables - experience

+1 on this...  I'd go so far as to say skip the copper, and just go with 
active-optical for short-run interconnects.

> Tyler Conrad <mailto:tyler at tgconrad.com>
> September 14, 2017 at 2:12 PM
> We're using a mix as well, some QSFP28 AOC, others DAC. One thing that you
> need to keep in mind about the DACs is going to be the bend radius. These
> things are girthy af, so make sure to either overestimate your runs
> slightly, or buy one to test first.
>
> Hugo Slabbert <mailto:hugo at slabnet.com>
> September 14, 2017 at 12:54 PM
>
> On Wed 2017-Sep-06 09:17:39 +0200, Jiri Prochazka <jiri at cdn77.com> wrote:
>
>
> We're deploying a decent chunk of 100G QSFP28 at the moment, but it's 
> a mix of:
>
> - a handful of 100G QSFP28 copper DACs for some switch peerlinks
> - a bit >100x 100G QSFP28 AOC for interswitch links
> - a lot more 100G QSFP28 -> 4x25G SFP28 copper breakouts
>
> We're only a few weeks in at this point, so mileage may vary in the 
> long run etc.
>
> The copper peerlinks are mostly 1M with some 3M.  We've had no issues 
> with them so far.
>
> The AOC interswitch links vary more in length, but some of those are 
> >10M (hence AOC rather than copper).  We've faced no issues with 
> those.  Granted, there is BGP with BFD running across those, so those 
> should help in terms of liveness checks and such.
>
> I mention that because where we _have_ had issues are on the 100G -> 
> 4x25G copper breakouts.  Those are for 25G edge connectivity.  It's a 
> decent sample size with a bit north of 600x 25G ports.  The trouble 
> we've had there have been with some links showing link up on the 
> switch and server side but actually failing to pass any traffic, so we 
> need to stuff some >L1 liveness checks on there to ensure those links 
> are good while we sort out the root issue.  It is not yet clear if 
> this is a cable fault, driver issue, or something firmware-ish on the 
> NICs.
>
> Also, fun fact: 25G only made its way into the 802.3ad bonding mode 
> driver in the Linux kernel in March this year[1].
>
> Jiri Prochazka <mailto:jiri at cdn77.com>
> September 6, 2017 at 3:17 AM
> Hi folks,
>
> I'm wondering if anyone have (either positive or negative) experience 
> with 100G QSFP28 DAC cables?
>
> Is there anyone who is using 100G DAC in large scale and would 
> recommend it (which means there are no issues compared to SR4 links)?
>
> I'm thinking about cables with lenght up to 1m, not more.
>
> We have had quite bad experience with 10G DAC in the past - but I do 
> not want to be slave of the past.
>
>
>
>
> Thank you for your thoughts!
>
>
>
> Jiri
>

-- 

-- 
Regards,
--
Brant I. Stevens, Principal & Consulting Architect
branto at argentiumsolutions.com
d:212.931.8566, x101. m:917.673.6536. f:917.525.4759.
http://argentiumsolutions.com