Linux router network cards

Sun Oct 25 16:05:50 UTC 2020

Not true, Intel Xeon E5 v3/4 have many more lanes than that. For example,
this one has 40
https://ark.intel.com/content/www/us/en/ark/products/91754/intel-xeon-processor-e5-2680-v4-35m-cache-2-40-ghz.html

Put two in the system and thats 80 lanes of PCIe Gen3.

Even newer procs in the bronze/silver/gold/platinum lineup have more lanes.

On Sat, Oct 24, 2020 at 8:57 AM Keith Medcalf <kmedcalf at dessus.com> wrote:

>
> And do not use an Intel CPU.
>
> Intel only has 4x PCIe lanes that are shared out into whatever
> configuration they claim to have and are totally unsuitable for use in a
> computer that actually has to be able to do high-speed I/O.
>
> --
> Be decisive.  Make a decision, right or wrong.  The road of life is paved
> with flat squirrels who could not make a decision.
>
> >-----Original Message-----
> >From: NANOG <nanog-bounces+kmedcalf=dessus.com at nanog.org> On Behalf Of
> >Eric Kuhnke
> >Sent: Saturday, 24 October, 2020 06:22
> >To: Jared Geiger <jared at compuwizz.net>
> >Cc: NANOG <nanog at nanog.org>
> >Subject: Re: Linux router network cards
> >
> >In addition to Jared's advice, I would recommend calculating PCI-Express
> >bandwidth bus points for whatever platform one is using.
> >
> >
> >For instance using the Intel X710-DA4, which could be capable in a
> >maximal scenario of 80Gbps of traffic, ensure it's in at least a PCI-E
> >3.0 x4 slot. And calculate the total number of PCI-E 3.0 x1 (or PCI-E 4.0
> >if a very new system) lanes that exist and are connected to the CPU. Big
> >difference between some options for Ryzen and Threadripper vs Intel CPUs,
> >towards the lower end of the cost range.
> >
> >
> >With recent Linux kernels if you have an Intel 510 or 710 series two or
> >four port card in a slot that can't support its full capability, you'll
> >get a warning in dmesg at boot time.
> >
> >
> >
> >On Thu, Oct 22, 2020 at 9:30 PM Jared Geiger <jared at compuwizz.net
> ><mailto:jared at compuwizz.net> > wrote:
> >
> >
> >       I use DANOS with Intel XL710 10G NICs in DPDK mode for linux based
> >routing.
> >
> >       If you're doing routing protocols, allocate 2 CPU cores to the
> >control plane and then a CPU core per 10G/1G interface for the dataplane,
> >plus an extra core for good measure. So for a 4 x 10G router taking in
> >full routes, 2 cores for control plane, 5 cores for the dataplane. Those
> >cores should be Intel Xeon E5-2600v3/4 or newer and faster the clocks,
> >the better.
> >
> >       Similar CPU core allocations if you choose TNSR.
> >
> >       On Thu, Oct 22, 2020 at 3:21 PM Jean St-Laurent via NANOG
> ><nanog at nanog.org <mailto:nanog at nanog.org> > wrote:
> >
> >
> >               Chelsio cards are probably what you are looking for.
> >
> >               https://www.chelsio.com/terminator-6-asic/
> >
> >               It's closer to an asic than a traditional nic as the
> >router/firewall rules
> >               are pushed directly into the hardware.
> >
> >               I don't know how good they are with linux and they seem to
> be
> >compatible.
> >               https://www.chelsio.com/linux/
> >
> >               You will need to mess around a bit and fiddle here and
> there.
> >If you don't
> >               mind using FreeBSD instead of linux, you could achieve a
> >smoother and more
> >               integrated experience.
> >
> >               Jean
> >
> >               -----Original Message-----
> >               From: NANOG <nanog-bounces+jean=ddostest.me at nanog.org
> ><mailto:ddostest.me at nanog.org> > On Behalf Of micah
> >               anderson
> >               Sent: Thursday, October 22, 2020 5:31 PM
> >               To: Philip Loenneker <Philip.Loenneker at tasmanet.com.au
> ><mailto:Philip.Loenneker at tasmanet.com.au> >; NANOG
> >               <nanog at nanog.org <mailto:nanog at nanog.org> >
> >               Subject: RE: Linux router network cards
> >
> >
> >               Thanks for the reply.
> >
> >               Philip Loenneker <Philip.Loenneker at tasmanet.com.au
> ><mailto:Philip.Loenneker at tasmanet.com.au> > writes:
> >               > Take a look at the Mellanox ConnectX 5 series of cards.
> They
> >handle
> >               > DPDK, PVRDMA (basically SR-IOV that allows live migration
> >between
> >               > hosts), and can even process packets within the NIC for
> some
> >
> >               From what I can tell, SR-IOV/PVRDMA aren't really useful
> for
> >me in building
> >               a router that wont be doing any virtualization.
> >
> >               If the card can do DPDK, can it do XDP?
> >
> >               > The slidedeck for the presentation is here:
> >               > https://www.ausnog.net/sites/default/files/ausnog-
> >2019/presentations/1
> >               > .9_Rhod_Brown_AusNOG2019.pdf
> >               >
> >               > It's heavily targeting virtualised workloads but some of
> the
> >feature sets
> >               apply to bare-metal uses too.
> >
> >               Yeah, this wont be a virtualized environment, just a router
> >passing packets,
> >               dropping them, handling bgp and collecting flows.
> >
> >               --
> >                       micah
> >
> >
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20201025/64655a10/attachment.html>