<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div dir="ltr"></div><div dir="ltr">Thank you Joy for de-lurking. I actually was not familiar with ZeroTier, and this is a space that I thought I was quite familiar with, so I’m glad you brought it to everyone’s attention. I will look further at ZeroTier, it looks very interesting.</div><div dir="ltr"><br></div><div dir="ltr">I am also a very long-time lurker (although I was a NANOG list admin ~10 years ago) who is emerging to join this conversation.</div><div dir="ltr"><br></div><div dir="ltr">I have recently been doing some work to evaluate and develop VPN solutions for connecting multiple data center cloud environments, including low-power small edge sites, and I have some thoughts about the current state of the art to share.</div><div dir="ltr"><br></div><div dir="ltr">Until recently a very strong proponent of IPSEC. I liked the way IPSEC was placed within the OSI model directly at layer 3, unlike some of the VPN technologies which operate above or below layer 3. However I do not believe that IPSEC is future-proof, for the following two reasons:</div><div dir="ltr"><br></div><div dir="ltr">1) IPSEC does not lend itself to dynamic routing or dynamic configuration. It is very much a static set-it-and-forget-it technology, but that doesn’t work in a dynamically changing environment.</div><div dir="ltr"><br></div><div dir="ltr">2) IPSEC does not always lend itself to hardware offloading in the way some other technologies do. Some NICs do support hardware acceleration for IPSEC, but this does not always integrate well with kernel or user space when you are integrating virtual network functions (VNFs) like routers/firewalls/load-balancers.</div><div dir="ltr"><br></div><div dir="ltr">Wireguard works well in dynamic environments. TLS using something like OpenSSL does as well. Both provide key advantages, particularly on top of Linux.</div><div dir="ltr"><br></div><div dir="ltr">* Support for hardware offloads such as TCP segmentation provide vast improvements in performance on higher-end x86 hardware. Some recent testing I have been shown proves that TCP segmentation offload can provide more than a 5X speedup compared to other HW offloads without TCP segmentation (from 5Gb/s to above 25Gb/s in some tests).</div><div dir="ltr"><br></div><div dir="ltr">* With the right encryption algorithm CPU acceleration for cryptography reduces CPU load and increases performance.</div><div dir="ltr"><br></div><div dir="ltr">* Integration with kernel routing provides the ability to integrate with dynamic routing such as BGP daemons (e.g. FRRouting, etc.).</div><div dir="ltr"><br></div><div dir="ltr">* In recent Linux kernels eBPF/XDP provide a hardware interface to the kernel which accelerates network throughput to near line-rate, while minimizing CPU impact.</div><div dir="ltr"><br></div><div dir="ltr">This may not apply to William Herrin’s (OP) use case of a VPN appliance for 100mbps to 1gbps speeds, but it is something to keep in mind for building higher performance solutions or for planning for increasing bandwidth in the future. For the 100mbps+ use case I have had success building appliances using OpenVPN on top of certain ARM based platforms like Marvell Armada, or single-board computers with Intel CPUs with AES-NI acceleration. I am currently looking at implementing Wireguard on the same platforms. For a simple low-power ARM router appliance the Turris Omnia has been a great fully open platform running a custom LEDE/OpenWRT OS. The Turris Mox provides a modular hardware platform for expandability, albeit with slightly less performance. Both of these platforms are developed by the engineers at CZ.nic, the TLD registrar for the Czech Republic.</div><div dir="ltr"><br></div><div dir="ltr"><a href="https://secure.nic.cz/files/Turris-web/Omnia/Omnia2020_datasheet.pdf">https://secure.nic.cz/files/Turris-web/Omnia/Omnia2020_datasheet.pdf</a></div><div dir="ltr"><br></div><div dir="ltr"><a href="https://www.turris.com/en/mox/overview/">https://www.turris.com/en/mox/overview/</a></div><div dir="ltr"><br></div><div dir="ltr">-Dan Sneddon</div><div dir="ltr"><br><blockquote type="cite">On Feb 10, 2022, at 10:51 AM, joy@cleverhack.com wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><span>Hello NANOG,</span><br><span></span><br><span>My name is Joy Larkin and I'm actually a long-time years-long lurker on the NANOG list (I have v odd hobbies) and I am also ZeroTier's Head of Marketing. I know I'm not supposed to be too promotional on here, but I'd love to see some of you pick up ZT.</span><br><span></span><br><span>Our founder, Adam Ierymenko just did a talk at Networking Field Day 27, here are two of the recordings from that session:</span><br><span></span><br><span>* ZeroTier The Planetary Data Center</span><br><span> * https://www.youtube.com/watch?v=T2BbrqpnMAE</span><br><span></span><br><span>* ZeroTier Technical Deep Dive</span><br><span> * https://www.youtube.com/watch?v=VhQ30bVF3_s</span><br><span></span><br><span>If you have questions, let me know - you can reach me at joy.larkin@zerotier.com</span><br><span></span><br><span>Best,</span><br><span>-Joy</span><br><span></span><br><span>On 2022-02-10 10:12, Mike Lyon wrote:</span><br><blockquote type="cite"><span>How about running ZeroTier on those Linux boxes and call it a day?</span><br></blockquote><blockquote type="cite"><span>https://www.zerotier.com/</span><br></blockquote><blockquote type="cite"><span>-Mike</span><br></blockquote><blockquote type="cite"><blockquote type="cite"><span>On Feb 10, 2022, at 10:07, David Guo via NANOG <nanog@nanog.org></span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>wrote:</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span></span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>You may try WireGuard and use ddns</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>From: NANOG <nanog-bounces+david=xtom.com@nanog.org> On Behalf Of</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>William Herrin</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>Sent: Friday, February 11, 2022 2:02 AM</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>To: nanog@nanog.org</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>Subject: VPN recommendations?</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>Hi folks,</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>Do you have any recommendations for VPN appliances? Specifically: I</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>need to build a site to site VPNs at speeds between 100mpbs and 1</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>gbit where all but one of the sites are behind an IPv4 NAT gateway</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>with dynamic public IP addresses.</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>Normally I'd throw OpenVPN on a couple of Linux boxes and be happy</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>but my customer insists on a network appliance. Site to site VPNs</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>using IPSec and static IP addresses on the plaintext side are a dime</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>a dozen but traversing NAT and dynamic IP addresses (and</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>automatically re-establishing when the service goes out and comes</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>back up with different addresses) is a hard requirement.</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>Thanks in advance,</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>Bill Herrin</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>--</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>William Herrin</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>bill@herrin.us</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>https://bill.herrin.us/</span><br></blockquote></blockquote></div></blockquote></body></html>