[EXTERNAL] Re: Help with removing DNS shinkhole FP from Charter/Spectrum

Tue Apr 23 20:24:32 UTC 2024

Hi Kenneth,

We have been working internally and with our third-party domain reputation source to get your domain removed from their malware list.

Jim

From: NANOG <nanog-bounces+jim.rampley=charter.com at nanog.org> on behalf of Validin Axon <axon at validin.com>
Date: Tuesday, April 23, 2024 at 2:15 PM
To: Tom Beecher <beecher at beecher.cc>
Cc: NANOG <nanog at nanog.org>
Subject: [EXTERNAL] Re: Help with removing DNS shinkhole FP from Charter/Spectrum

CAUTION: The e-mail below is from an external source. Please exercise caution before opening attachments, clicking links, or following guidance.

Tom,

Thank you for this! It is very interesting that the behavior is intermittent. A friend of mine who tested it this weekend saw correct answers on IPv6 and incorrect answers on IPv4.

Kenneth

On Tue, Apr 23, 2024 at 2:56 PM Tom Beecher <beecher at beecher.cc<mailto:beecher at beecher.cc>> wrote:
Validin, made an interesting observation on this. I am also a Spectrum residential customer,  none of their equipment, run my own DNS server (pihole).

My DHCP Assigned DNS servers are

2001:1998:f00:1::1
2001:1998:f00:2::1

bash-3.2$ dig -x 2001:1998:f00:1::1 +short
dns-cac-lb-01.rr.com<http://dns-cac-lb-01.rr.com>.
bash-3.2$ dig -x 2001:1998:f00:2::1 +short
dns-cac-lb-02.rr.com<http://dns-cac-lb-02.rr.com>.
bash-3.2$

bash-3.2$ dig dns-cac-lb-01.rr.com<http://dns-cac-lb-01.rr.com> +short
209.18.47.61
bash-3.2$ dig dns-cac-lb-02.rr.com<http://dns-cac-lb-02.rr.com> +short
209.18.47.62
bash-3.2$

bash-3.2$ dig @209.18.47.61<http://209.18.47.61> validin.com<http://validin.com> +short
157.245.112.183
137.184.54.107
bash-3.2$ dig @209.18.47.62<http://209.18.47.62> validin.com<http://validin.com> +short
157.245.112.183
137.184.54.107
bash-3.2$

bash-3.2$ dig @2001:1998:f00:1::1 validin.com<http://validin.com> +short
127.0.0.54
bash-3.2$

bash-3.2$ dig @2001:1998:f00:2::1 validin.com<http://validin.com> +short
127.0.0.54
bash-3.2$

Same servers on V4 were returning correct info, but on V6 were not.

However, a few minutes later :

bash-3.2$ dig @2001:1998:f00:1::1 validin.com<http://validin.com> +short
157.245.112.183
137.184.54.107
bash-3.2$ dig @2001:1998:f00:2::1 validin.com<http://validin.com> +short
157.245.112.183
137.184.54.107
bash-3.2$

Deltas :

bash-3.2$ dig @2001:1998:f00:1::1  validin.com<http://validin.com>

; <<>> DiG 9.10.6 <<>> @2001:1998:f00:1::1 validin.com<http://validin.com>
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42329
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;validin.com<http://validin.com>.                   IN      A

;; ANSWER SECTION:
validin.com<http://validin.com>.            60      IN      A       127.0.0.54

;; Query time: 37 msec
;; SERVER: 2001:1998:f00:1::1#53(2001:1998:f00:1::1)
;; WHEN: Tue Apr 23 13:50:03 EDT 2024
;; MSG SIZE  rcvd: 45

bash-3.2$

bash-3.2$ dig @2001:1998:f00:1::1 validin.com<http://validin.com>

; <<>> DiG 9.10.6 <<>> @2001:1998:f00:1::1 validin.com<http://validin.com>
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9667
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;validin.com<http://validin.com>.                   IN      A

;; ANSWER SECTION:
validin.com<http://validin.com>.            600     IN      A       157.245.112.183
validin.com<http://validin.com>.            600     IN      A       137.184.54.107

;; Query time: 157 msec
;; SERVER: 2001:1998:f00:1::1#53(2001:1998:f00:1::1)
;; WHEN: Tue Apr 23 14:19:20 EDT 2024
;; MSG SIZE  rcvd: 72

bash-3.2$

Seems like quite possibly they are intermittently caching bunk data from something.

On Tue, Apr 23, 2024 at 1:39 PM Validin Axon <axon at validin.com<mailto:axon at validin.com>> wrote:
Hi Jason,

> I suspect what’s happened is an incorrect assumption that DNS is even the issue here. Because you mentioned Spectrum Shield, I suspect it is not.

I appreciate the response and links. However, I've been told repeatedly by Spectrum that they're not blocking with Spectrum Shield. Despite these assurances, I've filled out a removal request through their published removal process several times, and the response I received stated that we're not being blocked. This check agrees with that:
https://www.spectrum.net/support/forms/verify_url_security

"Security Shield Is Not Blocking This Site
The URL provided is not being blocked by Spectrum Security Shield
The URL you entered should be accessible."
Further, checking Spectrum DNS servers on the Spectrum network show that my company's main domain and all subdomains resolve to 127.0.0.54. So, if CujoAI/Spectrum Shield are not using DNS query responses to control access, then it's not CujoAI/Spectrum Shield that is responsible for the incorrect DNS response. Using a different recursive resolve, I can resolve our domains just fine. I can also resolve other domains that point to the same IPs as the sinkholed domain just fine. However, many people use the Spectrum default DNS servers and cannot access my website because of this.

> You should contact Charter/Spectrum to have them investigate what their system might be blocking this content.

I have tried, for months, including spending many hours on chat and phone support, to reach someone within Spectrum support who is capable of both understanding and directing me to someone who can fix the problem, but it hasn't happened yet. I've asked to talk to someone on the DNS team and was given a flat "No." I've posted here hoping that someone in the ISP-connected world knows SOMEONE at Spectrum, Akamai, or whichever company is actually responsible for the Spectrum DNS servers who can provide a remediation path.

Regards,

Kenneth

On Tue, Apr 23, 2024 at 12:59 PM 'Livingood, Jason' via axon <axon at validin.com<mailto:axon at validin.com>> wrote:
> However, there's no correction process for Spectrum's DNS sinkhole
> But back to the topic: someone mentioned to me that Spectrum may not be the direct providers for the DNS services they provide to their customers. If anyone knows anything about how I might discover and reach out to the people responsible, please let me know.

I suspect what’s happened is an incorrect assumption that DNS is even the issue here. Because you mentioned Spectrum Shield, I suspect it is not.

Spectrum Shield (https://www.spectrum.com/resources/internet-wifi/benefits-of-spectrum-security-shield) is a customer-managed security protection service built into their gateways (I assume you can turn it off). The malware and content detection engine behind that is very likely run by CujoAI (https://cujo.com/) and it does not use DNS query/response exchanges as the control mechanism (in part to counter-act DNS-changing malware or malware using its own DoH channel for example).

You should contact Charter/Spectrum to have them investigate what their system might be blocking this content.

Comcast (where I work) runs a similar system (https://www.xfinity.com/support/articles/using-xfinity-xfi-advanced-security) and maintains a site to report these sorts of issues (https://www.xfinity.com/support/articles/report-blocked-website).

Jason

The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nanog.org/pipermail/nanog/attachments/20240423/9204914e/attachment.html>