AWS S3 DNS load balancer

Tue Jun 15 13:46:17 UTC 2021

The IP addresses for S3 do not change very often, and are region specific (as you would expect).

You are correct that this can cause problems for clients that never re-resolve (eg Java networkaddress.cache.ttl=-1)

You may be interested in the (periodically updated) list of AWS IP ranges by using their IP ranges JSON API. Refer to:
* https://ip-ranges.amazonaws.com/ip-ranges.json
* https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html

To get all S3 IP ranges currently in use:
"""
curl -sf 'https://ip-ranges.amazonaws.com/ip-ranges.json' \
| jq '.prefixes | map(select(.service == "S3"))'
"""

To get all S3 IP ranges in your region:
"""
 curl -sf 'https://ip-ranges.amazonaws.com/ip-ranges.json' \
| jq '.prefixes | map(select(.service == "S3" and .region == "eu-central-1"))'
"""

These ranges are not (to my knowledge) queryable via DNS.

In terms of this as a general behaviour, it is not uncommon. If I remember correctly this is how Route53 weighted records are implemented. So at least anyone using that feature of Route53 would be doing the same.

Met vriendelijke groeten,

Toby Lorne

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Tuesday, June 15th, 2021 at 13:37, Deepak Jain <deepak at ai.net> wrote:

> They seem to do something a little unusual where every DNS request provides a different IP out of a small pool with those IPs not changing very frequently. (I’m talking specifically about S3 not Route5x or whatever the DNS product is).
>
> Basically like round robin, but instead of providing all of the IPs they are only offering one. This eliminates options for the client DNS resolvers, but may make some things more deterministic.
>
> Is this a “normal” or expected solution or just some local hackery?
>
> Thanks in advance,
>
> DJ