Managing CE eBGP details & common/accepted CE-facing BGP practices
justin at justinshore.com
Sun Dec 21 18:19:47 CST 2008
Evening, Justin. Thanks for the reply.
Justin M. Streiner wrote:
> You could certainly store all of the relevant config details in a
> database of some sort, and it certainly can't hurt to do so. Same goes
> for backing up your device configurations - always a good idea. As far
> as storing things like ASNs, allowed prefixes, etc, you may want to look
> at storing that information in an RPSL format. Many providers require
> their customers to register route/AS/policy objects either in one of the
> Internet Routing Registries (RADB, AltDB, etc...), or in a similar
> system that's operated by the provider. They then use this information
> for pushing out configuraiton changes such as the list of prefixes that
> a customer is allowed to announce, etc.
RPSL could definitely be useful when we starting reaching that class of
customer. It's probably too grand for our users at this point
(especially having them register anything on their own). I'll
definitely do more research on this though.
> It's good that you're thinking about this now, particularly the "is this
> customer legitimately allowed to announce prefix ABC, or source their
> traffic from AS XYZ"? Most larger providers put at least some
> limitations on what they will allow a customer to announce, though the
> level of investigation done to attempt to establich legitimacy for those
> announcements varies. Some providers require customers to provide some
> sort of Letter of Authorization/Agency for the prefixes they want to
The process hasn't been established yet for validating a request to
permit a prefix announcement. I expect it will be a manual verification
process involving WHOIS lookups on the prefix, route-view queries to see
if it's currently being advertised and probably a historical check on
that prefix to see if it was previously advertised and from where. The
best way to avoid network abuse issues is to not let them happen to
begin with. I think we will also require some sort of written legal
agreement stating that the prefix in question belongs to the customer
and that we're authorized to permit it's advertisement across our
network. If anyone has any sample documents for use in this process I
would be interested in seeing them.
> Yes. As long as the organizations that are using the private AS aren't
> 1. trying to advertise the same space, 2. possibly connect directly to
> each other directly, or 3. expecting to be able to connect to multiple
> upstream providers, then you should be OK. VZB (former UUNET) did
> something similar, using AS7046 (not a private AS, but the principle is
> the same), and I believe other carriers have had similar arrangements
> for customers.
Ah, yes connecting to each other could be a problem. I would think that
it would only be an issue if I carried the private ASN across my iBGP
infrastructure, each customer received full routes from me and I let
them see the private ASNs as well. I could mitigate that problem with
remove-private-as, I believe. I'd need to think on that some more. If
the customer wants to be multi-homed and expect reachability then they
should get an ASN. Otherwise both SPs are advertising their prefixes
and the customer won't have much or possibly any control over which
inbound path was preferred.
> MD5 is good, but most providers I've seen make this an opt-in feature -
> they don't force their customers to use it. Setting a reasonable
> max-prefix limit and adjusting it as the number of prefixes a customer
> announces is always a good idea, and I'd consider it to be a best
> practice. Prefix lists and route maps can do different things, or
> accomplish the same task in different ways. It also depends on what
> functionality you want to offer your customers. Do you plan to publish
> and support a consistent set of customer-settable BGP communities for
> doing things like selective AS prepends? Do you plan to tag incoming
> advertisements with communities to identify them as customer routes, and
> pass those communities to your customers and peers? Some providers use
> AS path ACLs, and others avoid them at all costs.
I think I'll make MD5 part of the default config and let the customer
ask for it to be removed if they choose to not have it. Same for GTSM.
I'm a fan of max-prefixes. I think double the routes I expect to
receive, 75% warning and a restart interval of 5m would be a good place
to start. That would let me catch things happening before they got out
of hand (in normal circumstances) and give me a fail-safe in case they
decide to get crazy.
I do plan on implementing a BGP community solution but for now I'm going
to keep it simple. I have bigger fish to fry at the moment but I'll try
to get it done before we get asked for it by a customer. I will tag
transit and customer routes. The ISP Essentials book had some good
insight on that if memory serves me correctly. How fancy it gets will
depend on my time and customer demand. I've seen some extremely complex
setups that I could not replicate if my life depended on it.
The AS-Path filtering should only come into effect when we get a
customer with their own ASN in which cases we'll actually pass on their
advertisements (whereas with private ASNs we're only carrying them
internally and summarizing on the upstream edges). That way I can
ensure 1) that they claim to source the prefixes from their ASN and not
someone else's and 2) that they don't insert BS ASNs into the path.
> If the customers need to connect to multiple upstream providers, it's
> much less hassle to have them originate their prefixes from their own
> AS, and then you just need to propagate them. If they are single-homed,
> you can announce the prefix(es) for them and then just statically route
> them to the customer. Other things to consider are sound ingress/egress
> filtering policies, loose/strict RPF etc. Implementing a Netflow based
> monitoring solution, and applying flow caching to PE-CE interfaces is
> also agood idea.
I have a good set of residential and tiny SMB customer ACLs. I'm trying
to decide what I should use for these larger SMBs. I'm sure some will
be custom but I need to come up with a sane default set of ACLs. Some
things I refuse to not block. The customer can find a different
provider if they want certain things to not be blocked.
Until we get a multi-homed customer that wants inbound reachability
we'll go with strict uRPF; that's already in my interface templates.
I have NFSen set up but all it's really doing right now is filling my
hard drives. I haven't had much time to do anything else with it I'm
afraid. I'll point NF to MARS box when it gets here though. I forget,
does route-cache flow alter the packet switching capabilities of an
interface these days or is that just to turn on NF on an interface?
Seems like since CEF it only turns on NF.
> Again, it's great that you're being proactive about this. Setting up
> good policies now will make things much easier to maintain and provision
> in the future as your network grows. Don't forget to document those
> policies thoroughly, particularly if other people will be tasked with
> implementing it on a dat yo day basis, i.e. handling new customer
> turn-ups, making BGP prefix list modifications, etc.
That's my goal. I prefer to be organized from the get go whenever
possible. I don't like surprises in my work (unless it's a big,
> It seems like you have a pretty good handle on what you need to do. I
> don't know that there will be much of the holy war angle to the
> responses. Different people adopt different customer routing policies
> for different reasons - some technical, some business, some political.
> While there are different ways of accomplishing this task, most of the
> more scalable ways have already been a part of most large providers'
> policies for awhile.
I have an idea of what all needs to be implemented but I'm short on time
to hone my skills by trial and error.
> One-offs are bad, in my opinion, so the more you do to avoid them now,
> the fewer headaches you will have down the road.
Definitely. One-offs are like using duct tape to repair a Porsche.
It's just not right.
Thanks for the input. It's much appreciated.
More information about the NANOG