Google DNS intermittent ServFail for Disney subdomain

Fri Oct 20 17:00:07 UTC 2017

None of the NS records/delegations are in agreement.  com delegations
don't agree with authoritative in disney.com, and disney.com's
delegations don't agree with studio.disney.com's NSen.

On Fri, Oct 20, 2017 at 7:35 AM, Christopher Morrow
<morrowc.lists at gmail.com> wrote:
> On Fri, Oct 20, 2017 at 1:10 AM, David Sotnick <sotnickd-nanog at ddv.com>
> wrote:
>
>> Well well, it looks like a Direct Connect circuit to Google was leaking the
>> route to this DMZ 153.7.233.0/24 back to Google via BGP.
>>
>> Return traffic from Google (for only some fraction of DNS queries) was
>> passing back across this leaked route, and being dropped on this Direct
>> Connect peering point at Disney.
>>
>> Gotta love it when a problem is solved, by the OP, within an hour of
>> resorting to mailing the NANOG community.
>>
>>
>
> This shows some issues as well, I think?
> http://dnsviz.net/d/studio.disney.com/servers/
>
> $  dig NS disney.com
>
> ;; ANSWER SECTION:
> disney.com. 4676 IN NS huey11.disney.com.
> disney.com. 4676 IN NS huey.disney.com.
> disney.com. 4676 IN NS Orns02.dig.com.
> disney.com. 4676 IN NS Orns01.dig.com.
> disney.com. 4676 IN NS Sens02.dig.com.
> disney.com. 4676 IN NS Sens01.dig.com.
>
> $ dig NS studio.disney.com @huey11.disney.com.
> ;; AUTHORITY SECTION:
> studio.disney.com. 600 IN NS wallyb.pixar.com.
> studio.disney.com. 600 IN NS andre.pixar.com.
> studio.disney.com. 600 IN NS cliff.studio.disney.com.
> studio.disney.com. 600 IN NS norm.studio.disney.com.
>
> $ for d in $(dig +short NS disney.com); do dig +short SOA disney.com @$d;
> done
> huey.disney.com. root.huey.disney.com. 2017102000 3600 900 3600000 3600
> huey.disney.com. root.huey.disney.com. 2017102000 3600 900 3600000 3600
> huey.disney.com. root.huey.disney.com. 2017102000 3600 900 3600000 3600
> huey.disney.com. root.huey.disney.com. 2017102000 3600 900 3600000 3600
> huey.disney.com. root.huey.disney.com. 2017102000 3600 900 3600000 3600
> huey.disney.com. root.huey.disney.com. 2017102000 3600 900 3600000 3600
>
> $ for d in $(dig +short NS studio.disney.com); do dig +short SOA
> studio.disney.com @$d; done
> cliff.studio.disney.com. admin.studio.disney.com. 2017101904 10800 3600
> 604800 86400
> cliff.studio.disney.com. admin.studio.disney.com. 2017101904 10800 3600
> 604800 86400
> cliff.studio.disney.com. admin.studio.disney.com. 2017101904 10800 3600
> 604800 86400
> cliff.studio.disney.com. admin.studio.disney.com. 2017101904 10800 3600
> 604800 86400
> cliff.studio.disney.com. admin.studio.disney.com. 2017101904 10800 3600
> 604800 86400
>
> it looks like the second-level and third-level don't agree with each other
> on whom should be the NS for the third-level?
>
> that shouldn't be fatal, but is something to cleanup.
>
>
> Thanks all, nothing to see here!
>>
>> -David
>>
>> On Thu, Oct 19, 2017 at 8:41 PM, David Sotnick <sotnickd-nanog at ddv.com>
>> wrote:
>>
>> > Hi Nanog,
>> >
>> > I am principal network engineer for sister-studio to Disney Studios. They
>> > have been struggling with DNS issues since Thursday 12th October.
>> >
>> > By all accounts it appears as though *some* of the Google DNS resolvers
>> > cannot reach the authoritative nameservers for "studio.disney.com".
>> >
>> > This is causing ~20-30% of all DNS requests against Google Public DNS
>> > 8.8.8.8 / 8.8.4.4 to fail for requests in this subdomain.
>> >
>> > The name servers reside in 153.7.233.0/24.
>> >
>> > Might someone be able to *connect me* with someone at Google to assist my
>> > poor colleagues who are banging their heads against a brick wall here.
>> >
>> > Thank you,
>> > David
>> >
>>

-- 

"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler