10G CPE w/VXLAN - vendors?

Wed Jun 14 20:06:31 UTC 2023

The redundant links to the customer site that traverse independent underlay carriers, and in some cases, equal-cost paths that we want to load-balance across, are the hard part.  I’m not going to trust STP for that, and we aim for <3sec failover where we do have redundant paths.  ERPS can handle the failover, but not the load-balancing.  Any L2-over-L3 encapsulation protocol can handle the failover + ECMP features, but I need to do it at ~10G (~20G if ECMP) wire speed.

We provide IaaS services to our customers, which is why we’re stretching VLANs to them in the first place.  Viewed from the IaaS perspective, this is a bunch of DC-DC connections… but relative to the overall network, the customer-prem devices fall into the traditional “CPE” category.  (Most customers either just plug in bare fiber, or they connect to an intermediate carrier’s CPE.)

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN logo]]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png at 01D99ECB.C1CEDE50]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/[email protected]>

From: Joe Freeman <joe at netbyjoe.com>
Sent: Wednesday, June 14, 2023 2:16 PM
To: Adam Thompson <athompson at merlin.mb.ca>; nanog <nanog at nanog.org>
Subject: Re: 10G CPE w/VXLAN - vendors?

I think you’re probably overthinking this a bit.

Why do you need to extend your vxlan/evpn to the customer premise? There are a number of 1G/10G even 100G CPE demarc devices out there that push/pop tags, even q-in-q, or 802.1ad. Assuming you have some type of aggregation node you bring these back to, tie those tags to the appropriate EVPN instance at the aggregation point. Don’t extend anything but a management tag and an S-tag essentially to the device at the customer premise.

You can even put that management tagged vlan in it’s own L3 segment, or a larger L3 network and impose security. This way you’re not exposing your whole service infrastructure to a bad actor that might unplug your cpe device and plug into your network directly.

From: NANOG <nanog-bounces+joe=netbyjoe.com at nanog.org<mailto:nanog-bounces+joe=netbyjoe.com at nanog.org>> on behalf of Adam Thompson <athompson at merlin.mb.ca<mailto:athompson at merlin.mb.ca>>
Date: Wednesday, June 14, 2023 at 2:52 PM
To: nanog <nanog at nanog.org<mailto:nanog at nanog.org>>
Subject: 10G CPE w/VXLAN - vendors?
Hello, all.
I’m having difficulty finding vendors, never mind products, that fit my need.

We have a small but growing number of L2 (bridged) customers that have diverse fiber paths available, and, naturally, want to make use of them.
We have a solution for this: we extend the edge of our EVPN VXLAN fabric right to the customer premise.  The customer-prem device needs 4x10G SFP+ cages (2 redundant paths, plus LAG to customer), and the switches we currently use, Arista 7020Rs, are quite expensive if I’m deploying one one per customer.  (Nice switches, but overkill here – I don’t need 40/100G, and I don’t need 24 SFP+ ports.  And they still take forever to ship.)

We use RFC7438 §6.3 “vlan-aware-bundle” mode, not §6.1 “vlan-based” mode, which limits our choices somewhat.  I might be willing to entertain spinning up a separate VXLAN mesh using RFC7438 §6.1 (“vlan-based”) and static VTEPs if it saves me a lot of pain.

However, I’m having trouble finding small & cheaper 1U (or even desktop/wallmount) devices that have 4 SFP+ cages, and can do VXLAN, in the first place.
Who even makes CPE gear with SFP+ ports?  (Other than Mikrotik CRS309-1G-8S+IN / CRS317-1G-16S+RM, which are nice, but our policy requires vendor support contracts, so… no-go.)

Vendors?  Model#s, if you happen to know any?

Reply here or privately, whatever floats your boat – any pointers appreciated!

Adam Thompson
Consultant, Infrastructure Services
[[MERLIN logo]]
100 - 135 Innovation Drive
Winnipeg, MB R3T 6A8
(204) 977-6824 or 1-800-430-6404 (MB only)
https://www.merlin.mb.ca<https://www.merlin.mb.ca/>
[cid:image002.png at 01D99EC2.B891B0A0]Chat with me on Teams<https://teams.microsoft.com/l/chat/0/[email protected]>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20230614/00404d9b/attachment.html>