Intermittent incorrect DNS resolution?

Wed Jan 16 22:13:02 UTC 2013

On Wed, Jan 16, 2013 at 2:00 PM, Erik Levinson
<erik.levinson at uberflip.com> wrote:
> Hi everyone,
>
> I'm having an unusual DNS problem and would appreciate feedback.
>
> For the zones in question, primary DNS is provided by GoDaddy and
> secondary DNS by DNS Made Easy. Over a week ago we made changes to
> several A records (including wildcards on two different zones), all
> already having a TTL no greater than one hour.
>
> The new IPs on those A records have taken many millions of requests
> since the changes. Occasionally, a small amount of traffic appears at
> the old IPs that those A records had. This is HTTP traffic. Packet
> captures of this traffic show various Host headers.
>
> Attempting to resolve those various Host headers from various networks
> in Canada against various random private and public resolvers and
> against the authoritative NSs all yield correct results (i.e. new IPs).
>
> However, both GoDaddy and DNS Made Easy use anycast, which makes it less
> likely that I can see the entire picture of what's happening.
>
> I suspect that somewhere, one of their servers has the wrong data, or
> some resolver is misbehaving, but based on the
> pattern/traffic/volume/randomization of hostnames, the resolver theory is
> less likely. I haven't analyzed the source IPs yet to see if they're in a
> particular set of countries.
>
> I've opened a ticket with DNS Made Easy and they replied very quickly
> suggesting the problem is not with them. I've opened a ticket with
> GoDaddy and...well, it's GoDaddy, so I don't expect much (no response yet).
>
> Any ideas? Can folks try resolving eriktest.uberflip.com and post
> here with details only if it resolves to an IP starting with 76.9 (old IPs)?
>
>
> Thanks
>
> Erik

The other likely cause of this is local cacheing nameservers somewhere
at some ISP or major site, that do not respect TTL values for some
reason.

This is sadly a common problem - not statistically, most nameservers
do the right thing, but if you run big sites and flip things, there's
always a long tail of people whose nameservers just didn't get it.

-- 
-george william herbert
george.herbert at gmail.com