Frontier SoCal FIOS contact? - gateway proxy arp issue

Thu Aug 24 02:11:30 UTC 2017

So yesterday I started seeing some arp warnings in my server logs:

Aug 23 16:09:29 lisa /bsd: arp info overwritten for 96.251.22.154 by
f0:1c:2d:8d:0e:cf on em2
Aug 23 16:12:24 lisa /bsd: arp info overwritten for 96.251.22.154 by
f0:1c:2d:8d:0e:cf on em2
Aug 23 16:21:28 lisa /bsd: arp info overwritten for 96.251.22.154 by
00:25:90:da:ea:f9 on em2

f0:1c:2d:8d:0e:cf is the MAC address of 96.251.22.1,
L100.LSANCA-VFTTP-55.gni.frontiernet.net, my FIOS gateway. It seems that for
some reason proxy arp has been enabled on the router providing my gateway,
and it is arp'ing for my static IP addresses as shown below:

0c:c4:7a:b3:ca:54 - MAC of lisa.pbhware.com (96.251.22.156)
00:25:90:da:ea:f9 - MAC of bart.pbhware.com (96.251.22.154)
f0:1c:2d:8d:0e:cf - MAC of L100.LSANCA-VFTTP-55.gni.frontiernet.net
(96.251.22.1)

On 96.251.22.156:

$ ping 96.251.22.154

16:12:24.416146 0c:c4:7a:b3:ca:54 Broadcast arp 42: arp who-has
bart.pbhware.com tell lisa.pbhware.com

16:12:24.416405 00:25:90:da:ea:f9 0c:c4:7a:b3:ca:54 arp 60: arp reply
bart.pbhware.com is-at 00:25:90:da:ea:f9

16:12:24.419522 f0:1c:2d:8d:0e:cf 0c:c4:7a:b3:ca:54 arp 60: arp reply
bart.pbhware.com is-at f0:1c:2d:8d:0e:cf

Another example for an IP address (96.251.22.158/static-5.pbhware.com) not
currently in use:

$ ping 96.251.22.158

16:26:15.784494 0c:c4:7a:b3:ca:54 Broadcast arp 42: arp who-has
static-5.pbhware.com tell lisa.pbhware.com

16:26:15.787624 f0:1c:2d:8d:0e:cf 0c:c4:7a:b3:ca:54 arp 60: arp reply
static-5.pbhware.com is-at f0:1c:2d:8d:0e:cf

16:26:15.787677 0c:c4:7a:b3:ca:54 f0:1c:2d:8d:0e:cf ip 98: lisa.pbhware.com
> static-5.pbhware.com: icmp: echo request

The gateway starts arp'ing looking for the real owner of that IP so it can
proxy the traffic:

16:28:07.074869 f0:1c:2d:8d:0e:cf Broadcast arp 60: arp who-has
static-5.pbhware.com tell L100.LSANCA-VFTTP-55.gni.frontiernet.net

When I googled it, I found a number of complaints about this dating back
years regarding Verizon FIOS, where proxy arp deployments seemed to be a
standard practice in some areas, but for the decade I have had business FIOS
this has never happened before. So I contacted frontier technical support to
ask about it; unfortunately, the interaction was less than successful :(.
While the level 1 support person was very helpful and didn't take very long
to convince to escalate the issue, the first level 2 support person didn't
quite seem to understand the concept, as demonstrated by the following
excerpts from the chat:

"turning off the ARP is done from the device that is receiving the request"
"that is something he would need to turn off on his router, because we do
not support 3rd party equipment"
"sorry but we cannot make the change that he is requesting, it needs to be
done on his device"
"I understand what you are saying but the ARP request needs to be turned off
through his equipment it is what is allowing signal in, just like a Ping can
be turned off in the router even if something else is requesting the OPING"

Technically, that last comment is true - if my system did not make an arp
request, the frontier router would indeed not respond with a proxy arp
reply. Of course, my systems would have a really hard time talking to each
other if they weren't running arp 8-/.

The last thing he had to say was "its not, on a router if you turn off Ping
requests it will not get pinged, same thing with ARP requests" and I decided
to stop wasting my time on him. Sadly, the level 1 tech informed me that the
level 2 tech was actually working with his lead while responding to my
issue. As I mentioned, the level 1 tech was very helpful, he offered to try
to reach a different level 2 tech. The second level 2 tech refused to have
anything to do with me unless I hooked up the original actiontech router
that came with the Fios service 10 years ago, so I wrote off the official
tech-support channel for now.

So, long story short, are there any fios employees hanging out here that
could possibly get me in contact with someone who understands the concept of
proxy arp and would be able to determine why it suddenly was enabled on the
gateway for my service yesterday and hopefully get it turned back off? That
would be much appreciated.

In the worst case I suppose I can work around this mess with arp inspection
on the switch or static arp entries on the servers s, but I'd rather avoid
being kludgy.

Thanks much.