<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Dan,
<div class=""><br class="">
</div>
<div class="">One point you didn’t touch on is that IPSec is integrated into IPv6, typically hardware-accelerated on the NIC, enabling device-to-device VPNs, mitigates most of the dynamic issues associated with network-to-network IPSec over IPv4. </div>
<div class=""><br class="">
</div>
<div class="">Yes, I realize IPv4 is hanging around longer than most expect, but in some cases I think you can make a case for deploying IPv6 just on the VPN benefits alone. With no public-facing services, IPv6 is already deployed in most LANs as a direct result
of its use by modern OSes for inter-LAN communication. All you typically need to do is enable IPv6 at the gateway.</div>
<div class=""><br class="">
</div>
<div class=""> -mel</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Feb 11, 2022, at 10:33 AM, Dan Sneddon <<a href="mailto:sneddon@gmail.com" class="">sneddon@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="auto" class="">
<div dir="ltr" class=""></div>
<div dir="ltr" class="">Thank you Joy for de-lurking. I actually was not familiar with ZeroTier, and this is a space that I thought I was quite familiar with, so I’m glad you brought it to everyone’s attention. I will look further at ZeroTier, it looks very
interesting.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">I am also a very long-time lurker (although I was a NANOG list admin ~10 years ago) who is emerging to join this conversation.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">I have recently been doing some work to evaluate and develop VPN solutions for connecting multiple data center cloud environments, including low-power small edge sites, and I have some thoughts about the current state of the art to share.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">Until recently a very strong proponent of IPSEC. I liked the way IPSEC was placed within the OSI model directly at layer 3, unlike some of the VPN technologies which operate above or below layer 3. However I do not believe that IPSEC
is future-proof, for the following two reasons:</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It is very much a static set-it-and-forget-it technology, but that doesn’t work in a dynamically changing environment.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">2) IPSEC does not always lend itself to hardware offloading in the way some other technologies do. Some NICs do support hardware acceleration for IPSEC, but this does not always integrate well with kernel or user space when you are integrating
virtual network functions (VNFs) like routers/firewalls/load-balancers.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">Wireguard works well in dynamic environments. TLS using something like OpenSSL does as well. Both provide key advantages, particularly on top of Linux.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">* Support for hardware offloads such as TCP segmentation provide vast improvements in performance on higher-end x86 hardware. Some recent testing I have been shown proves that TCP segmentation offload can provide more than a 5X speedup
compared to other HW offloads without TCP segmentation (from 5Gb/s to above 25Gb/s in some tests).</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">* With the right encryption algorithm CPU acceleration for cryptography reduces CPU load and increases performance.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">* Integration with kernel routing provides the ability to integrate with dynamic routing such as BGP daemons (e.g. FRRouting, etc.).</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">* In recent Linux kernels eBPF/XDP provide a hardware interface to the kernel which accelerates network throughput to near line-rate, while minimizing CPU impact.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">This may not apply to William Herrin’s (OP) use case of a VPN appliance for 100mbps to 1gbps speeds, but it is something to keep in mind for building higher performance solutions or for planning for increasing bandwidth in the future.
For the 100mbps+ use case I have had success building appliances using OpenVPN on top of certain ARM based platforms like Marvell Armada, or single-board computers with Intel CPUs with AES-NI acceleration. I am currently looking at implementing Wireguard on
the same platforms. For a simple low-power ARM router appliance the Turris Omnia has been a great fully open platform running a custom LEDE/OpenWRT OS. The Turris Mox provides a modular hardware platform for expandability, albeit with slightly less performance.
Both of these platforms are developed by the engineers at CZ.nic, the TLD registrar for the Czech Republic.</div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class=""><a href="https://secure.nic.cz/files/Turris-web/Omnia/Omnia2020_datasheet.pdf" class="">https://secure.nic.cz/files/Turris-web/Omnia/Omnia2020_datasheet.pdf</a></div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class=""><a href="https://www.turris.com/en/mox/overview/" class="">https://www.turris.com/en/mox/overview/</a></div>
<div dir="ltr" class=""><br class="">
</div>
<div dir="ltr" class="">-Dan Sneddon</div>
<div dir="ltr" class=""><br class="">
<blockquote type="cite" class="">On Feb 10, 2022, at 10:51 AM, <a href="mailto:joy@cleverhack.com" class="">
joy@cleverhack.com</a> wrote:<br class="">
<br class="">
</blockquote>
</div>
<blockquote type="cite" class="">
<div dir="ltr" class=""><span class="">Hello NANOG,</span><br class="">
<span class=""></span><br class="">
<span class="">My name is Joy Larkin and I'm actually a long-time years-long lurker on the NANOG list (I have v odd hobbies) and I am also ZeroTier's Head of Marketing. I know I'm not supposed to be too promotional on here, but I'd love to see some of you pick
up ZT.</span><br class="">
<span class=""></span><br class="">
<span class="">Our founder, Adam Ierymenko just did a talk at Networking Field Day 27, here are two of the recordings from that session:</span><br class="">
<span class=""></span><br class="">
<span class="">* ZeroTier The Planetary Data Center</span><br class="">
<span class=""> * <a href="https://www.youtube.com/watch?v=T2BbrqpnMAE" class="">
https://www.youtube.com/watch?v=T2BbrqpnMAE</a></span><br class="">
<span class=""></span><br class="">
<span class="">* ZeroTier Technical Deep Dive</span><br class="">
<span class=""> * <a href="https://www.youtube.com/watch?v=VhQ30bVF3_s" class="">
https://www.youtube.com/watch?v=VhQ30bVF3_s</a></span><br class="">
<span class=""></span><br class="">
<span class="">If you have questions, let me know - you can reach me at <a href="mailto:joy.larkin@zerotier.com" class="">
joy.larkin@zerotier.com</a></span><br class="">
<span class=""></span><br class="">
<span class="">Best,</span><br class="">
<span class="">-Joy</span><br class="">
<span class=""></span><br class="">
<span class="">On 2022-02-10 10:12, Mike Lyon wrote:</span><br class="">
<blockquote type="cite" class=""><span class="">How about running ZeroTier on those Linux boxes and call it a day?</span><br class="">
</blockquote>
<blockquote type="cite" class=""><span class=""><a href="https://www.zerotier.com/" class="">https://www.zerotier.com/</a></span><br class="">
</blockquote>
<blockquote type="cite" class=""><span class="">-Mike</span><br class="">
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">On Feb 10, 2022, at 10:07, David Guo via NANOG <<a href="mailto:nanog@nanog.org" class="">nanog@nanog.org</a>></span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">wrote:</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class=""></span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">You may try WireGuard and use ddns</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">From: NANOG <<a href="mailto:nanog-bounces+david=xtom.com@nanog.org" class="">nanog-bounces+david=xtom.com@nanog.org</a>> On Behalf Of</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">William Herrin</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">Sent: Friday, February 11, 2022 2:02 AM</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">To: <a href="mailto:nanog@nanog.org" class="">
nanog@nanog.org</a></span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">Subject: VPN recommendations?</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">Hi folks,</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">Do you have any recommendations for VPN appliances? Specifically: I</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">need to build a site to site VPNs at speeds between 100mpbs and 1</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">gbit where all but one of the sites are behind an IPv4 NAT gateway</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">with dynamic public IP addresses.</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">Normally I'd throw OpenVPN on a couple of Linux boxes and be happy</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">but my customer insists on a network appliance. Site to site VPNs</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">using IPSec and static IP addresses on the plaintext side are a dime</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">a dozen but traversing NAT and dynamic IP addresses (and</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">automatically re-establishing when the service goes out and comes</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">back up with different addresses) is a hard requirement.</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">Thanks in advance,</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">Bill Herrin</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">--</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class="">William Herrin</span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class=""><a href="mailto:bill@herrin.us" class="">bill@herrin.us</a></span><br class="">
</blockquote>
</blockquote>
<blockquote type="cite" class="">
<blockquote type="cite" class=""><span class=""><a href="https://bill.herrin.us/" class="">https://bill.herrin.us/</a></span><br class="">
</blockquote>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>