Anycast but for egress

Vimal j.vimal at gmail.com
Wed Jul 28 01:15:12 UTC 2021


AWS Global Accelerator gives anycast IPs that's good for ingress, but my
original question was about having predictable egress IPs.

It looks like having a few EIPs/a contiguous network block is the way to go.

Thanks!

On Tue, Jul 27, 2021 at 4:30 PM Andras Toth <diosbejgli at gmail.com> wrote:

> Since you mentioned AWS, have you tried AWS Global Accelerator? You get a
> pair of globally anycasted static IPs.
> https://aws.amazon.com/global-accelerator/
>
> Another alternative is to request a contiguous IP range of EIPs (/28 or
> /24 etc) that you can use for your EC2 instances or VPC resources.
>
> Andras
>
> On 28 Jul 2021, at 09:19, Daniel Corbe <daniel at corbe.net> wrote:
>
> 
>
> On Jul 27, 2021, at 17:20, Vimal <j.vimal at gmail.com> wrote:
>
>
> Hi all, great replies. :) Let me clarify my initial question, and then
> respond one by one:
>
>
> My intention is to run a web-crawling service on a public cloud. This
> service is geographically distributed, and therefore will run in multiple
> regions around the world inside AWS... this means there will be multiple
> AWS VPCs, each with their own NAT gateway, and traffic destined to websites
> that we crawl will appear to come from this NAT gateway's IP address.
>
>
> The reason I want a predictable IP is to communicate this IP to website
> owners so they can allow access from these IPs into their networks.  I
> chose IP as an example; it can also be a subnet, but what I don't want to
> provide is a list of 100 different IP addresses without any predictability.
>
>
> I understand that this is not perfect, and would frankly not be my
> preferred approach to solve the problem.... but we've had requests of this
> nature from websites to create an allowlist of a limited number of
> predictable IPs so it doesn't trip their IDSs/other systems they might
> have... so we're trying to see how well it would work in practice.  For the
> moment, let's set aside the issue as to whether AWS will even let me
> advertise the same IP on all my VPC NAT gateways, and just look at whether
> it's technically feasible.  My gut feeling is that this wouldn't work well
> in practice, but I wanted to ask the experts here...
>
>
> Also, pointers on what the best practices for solving this issue are most
> welcome, so I can reference those who ask for IP addresses to this
> discussion and follow recommendations here.
>
>
> Onto the responses:
>
>
> @owen at delong.com and @woody at pch.net athompson at merlin.mb.ca
>
> Because there’s no good/reliable way to get the replies back to the
> correct initiating host.
>
>
> When my clients make connections outbound to anycast addresses, the
> destination is more-or-less stable, and the replies come back to the
> client's unique IP, so anycast works in that direction.  The guarantees are
> not present in the reverse direction.
>
>
> Yes, this makes sense as the destination can be anywhere around the world,
> and that routing is asymmetric as others mentioned.  However, if the
> destination service is "close" (in the routing metric sense) to the
> initiating host, anycast return IP ought to work well, right?  I understand
> this is a very important caveat and impractical to implement correctly in
> the real world.
>
>
> We use our IGP (IS-IS) for our Anycast services. We find it to be very
>
> basic, and as such, very predictable.
>
>
> This is interesting... I wonder whether Anycast will still have some
> failure modes and break TCP connections if routing (configuration) were to
> change?  I checked the PDF linked by Bill Woodcock... while the methodology
> is the same from 20y ago, would the data still be the same (order of
> magnitude)? :)
>
>
> https://www.pch.net/resources/Tutorials/anycast/Anycast-v10.pdf (p38)
>
> "Limited operational data shows underlying instability to be on
>
> the order of one flow per ten thousand per hour of duration."
>
>
> @daniel at corbe.net, @matt at netfire.net,
>
> Unless you’re twisting knobs, egress traffic should already exit your
> network at the closest possible egress point to its origin.  Is your
> intention to carry the traffic for longer than that?
>
> No, but I hope my intention is more clear in this email.  It's to have a
> predictable egress IP to simplify firewall rules.
>
>
> thanks all!!
>
>
>
> On Tue, Jul 27, 2021 at 12:25 PM Adam Thompson <athompson at merlin.mb.ca>
> wrote:
>
> Without any sarcasm: to make it harder to block.
>
> If, say, Google, always crawled your site from 8.8.1.2 (random made-up
> example) then you would see a not-insignificant number of hosts and
> networks null-routing that IP.  I have no idea why someone would do so, but
> I've seen it done many times.  Mostly by people who don't understand how
> un-special they are on the internet.  Also it would trigger IDS/IPS systems
> all over the place, having gobs and gobs of connections coming from a
> single IP.
>
>
> That's setting aside the technical issues involved; routing is often
> asymmetric, i.e. the return packet takes a different path than the inbound
> packet.  So it would, as Owen implied, be nearly impossible to ensure the
> reply packets got back to the correct TCP stack.  As an example, I'm
> multi-homed and use path-prepending, so if a packet claiming to be from
> 8.8.8.8 arrived on one of my commercial links, I would send the reply out
> the cheapest link, which in my case is a flat-rate R&E network (that has a
> path to Google), thus ensuring the reply does not get to the originating
> anycast node.
>
>
> When my clients make connections outbound to anycast addresses, the
> destination is more-or-less stable, and the replies come back to the
> client's unique IP, so anycast works in that direction.  The guarantees are
> not present in the reverse direction.
>
>
> The logical extremity of this is that it would be nearly impossible for
> two anycast addresses to establish a TCP connection to each other.  (In
> general.  There will be lots of local cases where it does happen to work,
> by coincidence.)
>
>
> You'll find that even anycast nodes do not make connections outbound using
> their anycast address, pretty much for these reasons.
>
>
> -Adam
>
>
> Adam Thompson
>
> Consultant, Infrastructure Services
>
> <Outlook-1593169877.png>
>
> 100 - 135 Innovation Drive
>
> Winnipeg, MB, R3T 6A8
>
> (204) 977-6824 or 1-800-430-6404 (MB only)
>
> athompson at merlin.mb.ca
>
> www.merlin.mb.ca
>
> From: NANOG <nanog-bounces+athompson=merlin.mb.ca at nanog.org> on behalf of
> Vimal <j.vimal at gmail.com>
>
> Sent: July 27, 2021 12:54
>
> To: nanog at nanog.org <nanog at nanog.org>
>
> Subject: Anycast but for egress
>
>
> (Unsure if this is the right forum to ask this question, but here goes:)
>
>
> From what I understand, IP Anycast can be used to steer traffic into a
> server that's close to the client.
>
>
> I am curious if anyone here has/encountered a setup where they use anycast
> IP on their gateways... to have a predictable egress IP for their traffic,
> regardless of where they are located?
>
>
> For example, a search engine crawler could in principle have the same IP
> advertised all over the world, but it looks like they don't...  I wonder
> why?
>
>
> --
>
> Vimal
>
>
>
>
> --
>
> Vimal
>
>
> If I were in your shoes, I’d pick a VPS provider that assigns external,
> globally routable IPs to their customers.  Linode, Vultr, Digital Ocean,
> etc.
>
> You may be able to do something on AWS with Elastic IPs but I don’t know
> enough about Amazon’s infrastructure to give you a qualified answer.
>
>
>
>

-- 
Vimal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20210727/44c31af9/attachment.html>


More information about the NANOG mailing list