asymmetric routes/security concerns/Fortinet

Fri Jan 7 19:37:57 UTC 2011

On Fri, Jan 07, 2011 at 01:56:00PM -0500, Greg Whynott said:
  >Based on the fact that we access ORION via one of our ISPs (3rd party, we
  don't BGP/directly peer with ORION), I'm not sure if i can use this solution
  here.  I could do that for the routes learned from that ISP, but we receive
  the entire internet routing table from them?  I'd have to understand things
  more before I went down that road.  perhaps I shouldn't be accepting the
  full table from them.
  >
  >the localpref is something I'll look at, thanks for that.  I'm not a BGP
  expert by any stretch, and our requirements here are "simple".  we are not a
  transit.  I've only attempted to make the config safe, not efficient.
  >
  > i'd like to hear what you have to say about the original question, is
  there good reason in this day and age to drop traffic as described in the
  original post in your opinion?

It sounds like the target site has a possible misconfiguration if this is a
long term issue. If they're using the open internet to get back to you and not
ORION (when your packets arrived from ORION-based connection), then something
is misconfigured or down. The problem is a conflict in the way BGP works and
how people assume it works :) BGP is designed to get packets to where they
want to go, not drop them if they're going the wrong way.

The route back to you via ORION might not be available temporarily for a
legitimate reason (outtage, etc), or due to other unintentional side effects
of preference configurations.  They are likely not seeing a route to you being
preferred via ORION. Try some traceroutes from your end and from their end and
compare.  They're likely different paths. However, that shouldnt be suprising
- or a reason to filter traffic, really.

Symmetric routes across any chunk (big or small) of the internet are a
mythological thing of the past. Don't rely on that being the case at
any time.

As for your getting a full table from your upstream, you would likely
expect and want that. Mixed in would be ORION's prefixes, and if things
are working right, you are using an ORION path to get to your target. That's
up to the upstream's config preferences for packets destined to go through
ORION, so if you are the one using the open internet to get to the target
(check your traceroutes), then you need to talk to your upstream and get them
to fix their route preferences or whatever link or peering session is down.

As for dropping traffic, that's a strong fail-fast signal there. If you want
to ensure you are getting the intended bandwidth, say if you needed a 100mbps
path guaranteed that ORION can easily give you but the open internet/your
respective ISPs cant or wont (or you didnt pay for nor want to), then having
it fail immediately instead of using a slow backup path might be what you
want. There's a balance to be struck between failing fast thus generating
sufficient complaints that pressure to fix the best (and only) path that has
the required capacity is done ASAP, but then you are also left with no
alternate connectivity to the target in the meantime. Which scenario you
prefer is up to you and dependant on your organizations' needs.

An alternative is to generate sufficient alarms on the best connection's
failure, fail over to slower alternates, alert users to the degraded quality
(and modify their expectations and behaviour), and have your respective tech
teams respond promptly. This all requires awareness, training and a more
sophisticated failure-handling system. (How do you automatically alert all
users that the alternate degraded path is in use for eg? Email? Pager?) Having
alternate connectivity tends to dilute responce urgency from my experience. A
question of discipline/(dis)incentives. :)

As for using an outtage as a security measure, yes you will reduce the
opportunities for the open internet to be a source of spoofed packets from
other 'semi-trusted' entities that you expect to only go across ORION for.

However, it sounds like you have no opportunity to determine such packets'
arrivals/departures, as it all goes through your upstream(s). The other end
however seems to have a firewall system that does determine this disparity of
paths. This is a minor security benefit, IMHO, which may be worth it to you,
depending on how important some connectivity is vs the increased risk of
spoofed packets from the general internet during the primary link via ORION's
downtime. And, it seems this is the other org's decision to make, not yours,
since you dont directly control your own ORION peering, and you are getting a
full routing table from your upstream.

If you wanted to control your ORION traffic, you would likely get a second BGP
feed and link from your upstream if you cant connect to ORION directly somehow.

Are you not on TORIX? If so you could connect to ORION directly as we at Heavy
Computing do.

/kc

  >
  >-g
  >
  >
  >
  >On Jan 7, 2011, at 1:15 PM, John Kristoff wrote:
  >
  >> On Fri, 7 Jan 2011 12:40:32 -0500
  >> Greg Whynott <Greg.Whynott at oicr.on.ca> wrote:
  >>
  >>> we have multiple internet connections of which one is a research
  >>> network where many medical institutions and universities are also
  >>> connected to threw out the country.  This research network (ORION)
  >>> also has internet access but is not meant to be used as a primary
  >>> path to the internet by its customers.     Connected to the ORION
  >>> network are many sites we exchange email with daily who also have
  >>> multiple internet connections.   One of these sites is not reachable
  >>> by us.   After investigating,  it was discovered this site is
  >>> dropping our connections as the path back to use would use a
  >>> different interface on the firewall ( a Fortinet device) than that
  >>> which it arrived upon.
  >>
  >> Correct me if I'm wrong, I'm not very familiar with ORION, but if it's
  >> like some of the research networks in the U.S. have been built in the
  >> past, ORION is dedicated high speed, low latency network that
  >> interconnects research institutions together.  The way these are often
  >> used is that you localpref routes you learn from ORION participants so
  >> that traffic between each of you goes over the research network.  You'd
  >> typically want this since the performance is good and there is plenty of
  >> capacity available, but it is also paid for, probably through some
  >> research grant, helping to reduce the use and expense of your commercial
  >> transit.
  >>
  >> You should be sending your traffic to them via ORION and they
  >> likewise.  However, if that path is down, then it would make sense for
  >> it to go via another route.  Hence, asymmetry may happen.
  >>
  >> Are you not sending the traffic via ORION?  If so, then I'd suggest you
  >> both have something to fix.  :-)
  >>
  >> John
  >
  >
  >--
  >
  >This message and any attachments may contain confidential and/or privileged information for the sole use of the intended recipient. Any review or distribution by anyone other than the person for whom it was originally intended is strictly prohibited. If you have received this message in error, please contact the sender and delete all copies. Opinions, conclusions or other information contained in this message may not be that of the organization.

-- 
Ken Chase - ken at heavycomputing.ca - +1 416 897 6284 - Toronto CANADA
Heavy Computing - Clued bandwidth, colocation and managed linux VPS @151 Front St. W.