Ingress filtering on transits, peers, and IX ports

Tue Oct 20 00:38:10 UTC 2020

Thanks to the folks who responded to my messages on and off-list.  A 
couple of folks have asked me to summarize the responses that I 
received.

* Static ACL is currently the best way to protect a multi-homed network. 
  Loose RPF may be used if bogon filtering is more important, but it does 
not provide anti-spoofing security.

* Protect your infrastructure subnets with the ingress ACL [BCP 84 sec 
3.2].  Loopbacks and point-to-point circuits can benefit from this.  In 
the draft ACL, for example, I permit ICMP and traceroute over UDP, and 
block all else.

* Do an egress ACL also, to prevent clutter from reaching the rest of 
the 'Net.  Permit only your aggregate and customer prefixes going 
outbound.

* As I worked through putting the ACLs together, I found that if one 
implements an egress ACL, then customer prefixes must be enumerated 
anyway.  Once those are in an object group, it's easy to add an entry to 
the ingress ACL permitting traffic destined to customer PI space and 
aggregate space.  Seems better than just permitting all traffic in.

Our ACLs, both v4 and v6, now look like the following:

Ingress

* Deny to and from bogon networks, where bogon is either source or dest
* Permit to and from WAN PtP subnets
* For IPv6, also permit link-local IPs (fe80::/10)
* Deny to and from multicast ranges 224.0.0.0/4 and ff00::/8
* Permit ICMP / traceroute over UDP to infrastructure
* Deny all other traffic to infrastructure
* Permit from customer PI / PA space
* Deny from originated aggregate space
* Permit all traffic to customer PI / PA space
* Permit all traffic to aggregate space
* Deny any any

Egress

* Deny to and from bogon networks
* Permit to and from WAN PtP subnets
* For IPv6, also permit link-local IPs
* Deny to and from multicast range
* Permit all traffic from customer PI / PA space
* Permit all traffic from aggregate space
* Deny any any

We have started implementing the ACLs by blocking the bogon traffic 
only.  The other deny rules are set up as permit rules for now with 
logging turned on.  I'll review matching traffic before I switch the 
rules to deny.

Future work also includes automating the updates to the object groups 
via IRR.

BTW, Team Cymru didn't have any guidance around IPv6 bogons, so I put 
together the below object group based on the IANA IPv6 allocation list: 
https://www.iana.org/assignments/ipv6-unicast-address-assignments/ipv6-unicast-address-assignments.xhtml. 
  Obviously this is only for space not yet allocated to RIRs.

object-group network ipv6 IPV6-BOGON
   description Invalid IPV6 networks
   ::/3
   4000::/3
   6000::/3
   8000::/3
   a000::/3
   c000::/3
   e000::/4
   f000::/5
   f800::/6
   fc00::/7
   fe00::/9
   fec0::/10
exit

Thanks,

-Brian

On 2020-10-14 17:43, Brian Knight wrote:
> So I have put together what I think is a reasonable and complete ACL.
> From my time in the enterprise world, I know that a good ingress ACL
> filters out traffic sourcing from:
> 
> * Bogon blocks, like 0.0.0.0/8, 127.0.0.0/8, RFC1918 space, etc
> (well-documented in
> https://team-cymru.com/community-services/bogon-reference/bogon-reference-http/)
> * RIR-assigned blocks I am announcing to the rest of the world
> 
> However, I recognized a SP-specific case where we could receive
> legitimate traffic sourcing from our own IP blocks: customers running
> multi-homed BGP where we have assigned PA space to them.  So I added
> "permit" statements for traffic sourcing from these blocks.
> 
> Also, we have direct peering links that are numbered within our
> assigned prefixes.  So we can use the same ACL with these peer
> interfaces and continue to have BGP work, I added "permit" statements
> for these point-to-point subnets.
> 
> So the order of the statements is:
> 
> * Permit where source is direct peer PtP networks
> * Permit where source is BGP customer PA prefix
> * Deny where source is bogon
> * Deny where source is our advertised prefixes
> * Permit all other traffic
> 
> I considered BGP customer PI prefixes to be out of scope for ingress
> filtering, since the customer is likely to be multi-homing.  Should we
> consider filtering them?
> 
> The Team Cymru Secure IOS Template
> [https://www.cymru.com/Documents/secure-ios-template.html] also
> references an ICMP fragment drop entry on the ingress ACL.  I think
> that's good for an enterprise network, but as an SP, I'm very hesitant
> to include this.  Is this included in anyone else's transit / peer /
> IX ACL?
> 
> Is there anything else that I'm not thinking of?
> 
> Thanks,
> 
> -Brian
> 
> 
> On 2020-10-14 09:25, Brian Knight via NANOG wrote:
>> Hi Marcos,
>> 
>> Thanks for your reply.  But I am looking for guidance on traffic
>> filtering, not BGP prefix filtering.
>> 
>> I have looked at BCP 84, and it's a good overview of the methods
>> available to an ISP.  My questions are more operational in nature and
>> are not covered by the document.  Of the choices presented in BCP 84,
>> what do folks really use?  If it's an ACL, what challenges have there
>> been with updates?  etc.
>> 
>> -Brian
>> 
>> 
>> On 2020-10-13 18:52, Marcos Manoni wrote:
>>> Hi, Brian
>>> 
>>> Check RFC3704/BCP84 Ingress Filtering for Multihomed Networks 
>>> (Updated
>>> by: RFC8704 Enhanced Feasible-Path uRPF).
>>> 
>>>     Ingress Access Lists require typically manual maintenance, but 
>>> are
>>>     the most bulletproof when done properly; typically, ingress 
>>> access
>>>     lists are best fit between the edge and the ISP when the
>>>     configuration is not too dynamic if strict RPF is not an option,
>>>     between ISPs if the number of used prefixes is low, or as an
>>>     additional layer of protection
>>> 
>>> 
>>> Ingress filters Best Practices
>>> https://www.ripe.net/support/training/material/bgp-operations-and-security-training-course/BGP-Slides-Single.pdf
>>> *Don’t accept BOGON ASNs
>>> *Don’t accept BOGON prefixes
>>> *Don’t accept your own prefix
>>> *Don’t accept default (unless you requested it)
>>> *Don’t accept prefixes that are too specific
>>> *Don’t accept if AS Path is too long
>>> *Create filters based on Internet Routing Registries
>>> 
>>> And also BGP Best Current Practices by Philip Smith
>>> http://www.bgp4all.com.au/pfs/_media/workshops/05-bgp-bcp.pdf
>>> 
>>> Regards.
>>> 
>>> El mar., 13 oct. 2020 a las 19:52, Brian Knight via NANOG
>>> (<nanog at nanog.org>) escribió:
>>>> 
>>>> Hi Mel,
>>>> 
>>>> My understanding of uRPF is:
>>>> 
>>>> * Strict mode will permit a packet only if there is a route for the 
>>>> source IP in the RIB, and that route points to the interface where 
>>>> the packet was received
>>>> 
>>>> * Loose mode will permit a packet if there is a route for the source 
>>>> IP in the RIB.  It does not matter where the route is pointed.
>>>> 
>>>> Strict mode won't work for us, because with our multi-homed transits 
>>>> and IX peers, we will almost certainly drop a legitimate packet 
>>>> because the best route is through another transit.
>>>> 
>>>> Loose mode won't work for us, because all of our own prefixes are in 
>>>> our RIB, and thus the uRPF check on a transit would never block 
>>>> anything.
>>>> 
>>>> Or am I missing something?
>>>> 
>>>> Thanks,
>>>> 
>>>> -Brian
>>>> 
>>>> On 2020-10-13 17:22, Mel Beckman wrote:
>>>> 
>>>> You can also use Unicast Reverse Path Forwarding. RPF is more 
>>>> efficient than ACLs, and has the added advantage of not requiring 
>>>> maintenance. In a nutshell, if your router has a route to a prefix 
>>>> in its local RIB, then incoming packets from a border interface 
>>>> having a matching source IP will be dropped.
>>>> 
>>>> RPF has knobs and dials to make it work for various ISP 
>>>> environments. Implement it carefully (as is be standing next to the 
>>>> router involved :)
>>>> 
>>>> Here's a Cisco brief on the topic:
>>>> 
>>>> 
>>>> https://tools.cisco.com/security/center/resources/unicast_reverse_path_forwarding
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> I think all router vendors support this feature. Here's a similar 
>>>> article by Juniper:
>>>> 
>>>> https://www.juniper.net/documentation/en_US/junos/topics/task/configuration/interfaces-configuring-unicast-rpf.html
>>>> 
>>>> 
>>>> -mel beckman
>>>> 
>>>> On Oct 13, 2020, at 3:15 PM, Brian Knight via NANOG 
>>>> <nanog at nanog.org> wrote:
>>>> 
>>>> We recently received an email notice from a group of security 
>>>> researchers who are looking at the feasibility of attacks using 
>>>> spoofed traffic.  Their methodology, in broad strokes, was to send 
>>>> traffic to our DNS servers with a source IP that looked like it came 
>>>> from our network.  Their attacks were successful, and they included 
>>>> a summary of what they found.  So this message has started an 
>>>> internal conversation on what traffic we should be filtering and 
>>>> how.
>>>> 
>>>> This security test was not about BCP 38 for ingress traffic from our 
>>>> customers, nor was it about BGP ingress filtering.  This tested our 
>>>> ingress filtering from the rest of the Internet.
>>>> 
>>>> It seems to me like we should be filtering traffic with spoofed IPs 
>>>> on our transit, IX, and peering links.  I have done this filtering 
>>>> in the enterprise world extensively, and it's very helpful to keep 
>>>> out bad traffic.  BCP 84 also discusses ingress filtering for SP's 
>>>> briefly and seems to advocate for it.
>>>> 
>>>> We have about 15 IP blocks allocated to us.  With a network as small 
>>>> as ours, I chose to go with a static ACL approach to start the 
>>>> conversation.  I crafted a static ACL, blocking all ingress traffic 
>>>> sourced from any of our assigned IP ranges.  I made sure to include:
>>>> 
>>>> * Permit entries for P-t-P WAN subnets on peering links
>>>> 
>>>> * Permit entries for IP assignments to customers running multi-homed 
>>>> BGP
>>>> 
>>>> * The "permit ipv4 any any" at the end :)
>>>> 
>>>> The questions I wanted to ask the SP community are:
>>>> 
>>>> * What traffic filtering do you do on your transits, on IX ports, 
>>>> and your direct peering links?
>>>> 
>>>> * How is it accomplished?  Through static ACL or some flavor of 
>>>> uRPF?
>>>> 
>>>> * If you use static ACLs, what is the administrative overhead like?  
>>>> What makes it easy or difficult to update?
>>>> 
>>>> * How did you test your filters when they were implemented?
>>>> 
>>>> Thanks a lot,
>>>> 
>>>> -Brian
>>>> 
>>>> 