<div dir="ltr">I'm being sloppy with my verbiage, it's just been a long time since I thought about this in detail, sorry. <div><br><div>The MAC layer hands bits to the Media Independent Interface, which connects the MAC to the PHY. The PHY converts the digital 1/0 into the form required by the media transmission type; the 'what goes over the wire' L1 stuff. The method of encoding will always add SOME number of bits as overhead. Ex, 64b/66b means that for every 64 bits of data to transmit, 2 bits are added, so 66 actual bits are transmitted. This encoding overhead is what I meant when I said 'ethernet control frame juju'. This starts getting into the weeds on symbol/baud rates and stuff as well, which I dont want to do now cause I'm even rustier there. </div></div><div><br></div><div>When FEC is enabled, the number of overhead bits added to the transmission increases. For 400G-FR4 for example, you start with 256b/257b , which is doubled to 512b/514b for ($reason I cannot remember), then RS-FEC(544,514) is applied, adding 30 more bits for FEC. Following the example, this means 544 bits are transmitted for every 512 bits of payload data. So , more overhead. Those additional bits can correct up to 15 corrupted bits of the payload. </div><div><br></div><div>All of these overhead bits are added in the PHY on the way out, and removed on the way in. So you'll never see them on a packet capture unless you're using something that's actually grabbing the bits off the wire. </div><div><br></div><div>( Pretty sure this is right, anyone please correct me if I munged any of it up.)</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 18, 2024 at 2:45 PM Aaron Gould <<a href="mailto:aaron1@gvtc.com">aaron1@gvtc.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><u></u>
<div>
<p>Thanks. What "all the ethernet control frame juju" might you be
referring to? I don't recall Ethernet, in and of itself, just
sending stuff back and forth. Does anyone know if this FEC stuff
I see concurring is actually contained in Ethernet Frames? If so,
please send a link to show the ethernet frame structure as it
pertains to this 400g fec stuff. If so, I'd really like to know
the header format, etc.<br>
</p>
<p>-Aaron<br>
</p>
<div>On 4/18/2024 1:17 PM, Tom Beecher
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">FEC is occurring at the PHY , below the PCS.
<div><br>
</div>
<div>Even if you're not sending any traffic, all the ethernet
control frame juju is still going back and forth, which FEC
may have to correct.</div>
<div><br>
</div>
<div>I *think* (but not 100% sure) that for anything that by
spec requires FEC, there is a default RS-FEC type that will be
used, which *may* be able to be changed by the device. Could
be fixed though, I honestly cannot remember. </div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Apr 18, 2024 at
1:35 PM Aaron Gould <<a href="mailto:aaron1@gvtc.com" target="_blank">aaron1@gvtc.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<pre>Not to belabor this, but so interesting... I need a FEC-for-Dummies or FEC-for-IP/Ethernet-Engineers...
Shown below, my 400g interface with NO config at all... Interface has no traffic at all, no packets at all.... BUT, lots of FEC hits. Interesting this FEC-thing. I'd love to have a fiber splitter and see if wireshark could read it and show me what FEC looks like...but something tells me i would need a 400g sniffer to read it, lol
It's like FEC (fec119 in this case) is this automatic thing running between interfaces (hardware i guess), with no protocols and nothing needed at all in order to function.
-Aaron
</pre>
<pre>{master}
me@mx960> show configuration interfaces et-7/1/4 | display set
{master}
me@mx960>
{master}
me@mx960> clear interfaces statistics et-7/1/4
{master}
me@mx960> show interfaces et-7/1/4 | grep packet
Input packets : 0
Output packets: 0
{master}
me@mx960> show interfaces et-7/1/4 | grep "put rate"
Input rate : 0 bps (0 pps)
Output rate : 0 bps (0 pps)
{master}
me@mx960> show interfaces et-7/1/4 | grep rror
Link-level type: Ethernet, MTU: 1514, MRU: 1522, Speed: 400Gbps, BPDU Error: None, Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
Bit errors 0
Errored blocks 0
Ethernet FEC statistics Errors
FEC Corrected Errors 28209
FEC Uncorrected Errors 0
FEC Corrected Errors Rate 2347
FEC Uncorrected Errors Rate 0
{master}
me@mx960> show interfaces et-7/1/4 | grep packet
Input packets : 0
Output packets: 0
{master}
me@mx960> show interfaces et-7/1/4 | grep "put rate"
Input rate : 0 bps (0 pps)
Output rate : 0 bps (0 pps)
{master}
me@mx960> show interfaces et-7/1/4 | grep rror
Link-level type: Ethernet, MTU: 1514, MRU: 1522, Speed: 400Gbps, BPDU Error: None, Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
Bit errors 0
Errored blocks 0
Ethernet FEC statistics Errors
FEC Corrected Errors 45153
FEC Uncorrected Errors 0
FEC Corrected Errors Rate 29
FEC Uncorrected Errors Rate 0
{master}
me@mx960> show interfaces et-7/1/4 | grep packet
Input packets : 0
Output packets: 0
{master}
me@mx960> show interfaces et-7/1/4 | grep "put rate"
Input rate : 0 bps (0 pps)
Output rate : 0 bps (0 pps)
{master}
me@mx960> show interfaces et-7/1/4 | grep rror
Link-level type: Ethernet, MTU: 1514, MRU: 1522, Speed: 400Gbps, BPDU Error: None, Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
Bit errors 0
Errored blocks 0
Ethernet FEC statistics Errors
FEC Corrected Errors 57339
FEC Uncorrected Errors 0
FEC Corrected Errors Rate 2378
FEC Uncorrected Errors Rate 0
{master}
me@mx960>
</pre>
<p><br>
</p>
<div>On 4/18/2024 7:13 AM, Mark Tinka wrote:<br>
</div>
<blockquote type="cite"> <br>
<br>
On 4/17/24 23:24, Aaron Gould wrote: <br>
<br>
<blockquote type="cite">Well JTAC just said that it seems
ok, and that 400g is going to show 4x more than 100g
"This is due to having to synchronize much more to
support higher data." <br>
<br>
</blockquote>
<br>
We've seen the same between Juniper and Arista boxes in
the same rack running at 100G, despite cleaning fibres,
swapping optics, moving ports, moving line cards, e.t.c.
TAC said it's a non-issue, and to be expected, and shared
the same KB's. <br>
<br>
It's a bit disconcerting when you plot the data on your
NMS, but it's not material. <br>
<br>
Mark. <br>
</blockquote>
<pre cols="72">--
-Aaron</pre>
</div>
</blockquote>
</div>
</blockquote>
<pre cols="72">--
-Aaron</pre>
</div>
</blockquote></div>