100G QSFP28 DAC cables - experience

Hugo Slabbert hugo at slabnet.com
Thu Sep 14 16:54:31 UTC 2017

On Wed 2017-Sep-06 09:17:39 +0200, Jiri Prochazka <jiri at cdn77.com> wrote:

>Hi folks,
>I'm wondering if anyone have (either positive or negative) experience 
>with 100G QSFP28 DAC cables?
>Is there anyone who is using 100G DAC in large scale and would 
>recommend it (which means there are no issues compared to SR4 links)?
>I'm thinking about cables with lenght up to 1m, not more.
>We have had quite bad experience with 10G DAC in the past - but I do 
>not want to be slave of the past.
>Thank you for your thoughts!

We're deploying a decent chunk of 100G QSFP28 at the moment, but it's a mix 

- a handful of 100G QSFP28 copper DACs for some switch peerlinks
- a bit >100x 100G QSFP28 AOC for interswitch links
- a lot more 100G QSFP28 -> 4x25G SFP28 copper breakouts

We're only a few weeks in at this point, so mileage may vary in the long 
run etc.

The copper peerlinks are mostly 1M with some 3M.  We've had no issues with 
them so far.

The AOC interswitch links vary more in length, but some of those are >10M 
(hence AOC rather than copper).  We've faced no issues with those.  
Granted, there is BGP with BFD running across those, so those should help 
in terms of liveness checks and such.

I mention that because where we _have_ had issues are on the 100G -> 4x25G 
copper breakouts.  Those are for 25G edge connectivity.  It's a decent 
sample size with a bit north of 600x 25G ports.  The trouble we've had 
there have been with some links showing link up on the switch and server 
side but actually failing to pass any traffic, so we need to stuff some >L1 
liveness checks on there to ensure those links are good while we sort out 
the root issue.  It is not yet clear if this is a cable fault, driver 
issue, or something firmware-ish on the NICs.

Also, fun fact: 25G only made its way into the 802.3ad bonding mode driver 
in the Linux kernel in March this year[1].

Hugo Slabbert       | email, xmpp/jabber: hugo at slabnet.com
pgp key: B178313E   | also on Signal

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20170914/a6866460/attachment.sig>

More information about the NANOG mailing list