Subject: [GROW] Route Server ASN stripping hiding considered harmful?

From: Job Snijders <[email-masked]>
Date: Mon, 18 Dec 2017 10:50:34 +0000

Dear GROW,

After the gazillionth routing incident in which an IXP route server
amplified the problem, I think it is time we explore mechanisms to
improve accountability, auditability & make debugging easier.

It is common practise for route server operators to configure their
route server in such a way that the route server does not append its own
ASN to the AS_PATH attribute. Many IXPs view this practise as
justifiable because it gives them a competitive advantage over transit
providers.

RFC 7947 reasons "a route server does not participate in the process of
forwarding data between client routers", but meanwhile quite some IXPs
have more equipment and layer-2 hops as part of the forwarding path than
comparable IP networks for similar distances. [1] [2] [3] Does it even
matter that the route server device itself is not part of the
forwarding path? For all intents and purposes it is a BGP hop. The route
server is not under administrative control of the RS participants.

Whenever routing incidents occur, it takes considerable effort to
pinpoint which route server BGP speakers amplified the problem. Quite
some correlation & verification work is needed to reliably point out
which route server at what IXP contributed to the propagation of a
hijack or a route leak. If IXP route servers were visible in the
AS_PATH (just like normal IP networks), I think we'd see a lot more
responsible behavior and the implementations of route filters. 

Consider the following scenarios. When client A & client B at the same
IX have a bilateral session with each other, it doesn't matter whether
the route server injects its AS in the AS_PATH: the bilateral session
makes the route server control-plane path irrelevant. Since it is common
practise to assign a higher local preference to 'peering' routes
compared to 'transit' routes, in the case that client A & client B don't
have a bilateral session and only see each other via the route server,
padding the AS_PATH with the route servers ASN still won't have an
negative effect. It is only in corner cases that the AS_PATH becomes the
tie-breaker, and I find it hard to use those as justification to
sacrifice all of the route server's visibility.

Another benefit is that if route servers behave like normal BGP
speakers, client's dont need to disable safety checks such as "no bgp
enforce-first-as". Or that routing policy to match on prefixes coming
from the route server (on devices not connected to the IX) are simpler
if the ASN is vislble, and of course the same applies to analysis of MRT
dumps or use of BGP data in applications such as spam filtering. Or
think of hunting down announcements that became ghost routes, without
all ASNs visible in the AS_PATH as operator you'll be going through
severe pains to find the ghost route perpetrator.

Decades ago some argued that route servers should provide an experience
as close to bilateral peering as possible. But fast forward to 2017, we
see route servers with hundreds of thousands of paths, thousands of
sessions, in essence route servers became partial transit providers. I'd
argue that route server operators should assume their fiduciary duty and
stop hiding themselves. This will allow both operators & researchers to
more easily pinpoint issues, and help fix them.

In summary, I think RFC 7947 section 2.2.2.1 & 2.2.2.2 were misguided,
should be revisited, perhaps deleted. Thoughts?

Kind regards,

Job

[1]: https://ams-ix.net/technical/ams-ix-infrastructure
[2]: https://www.linx.net/tech-info-help/network-topology
[3]: https://www.nl-ix.net/network/projected-core-map-2017/

---

From: Fredrik Korsbäck <[email-masked]>
Date: Mon, 18 Dec 2017 12:41:07 +0100

> [3]: https://www.nl-ix.net/network/projected-core-map-2017/

Hi.

This makes alot of sense actually. Now that we live in a world where a route-server and an IXP is not what it used to be 
we need to consider IXPs as any ISP out there. Heck, some of these IXPs runs bigger infrastructures then most of their 
customers do.

Injecting the RS asn into the path makes troubleshooting much easier and helps us understand the routingt-able alot 
better, and having a RS ASN show up when backtracking an incident on f.e BGP Play will shorten the lead-time to figuring 
out what actually happened significantly.

I do see a problem however that we kinda need to change this all at the same time to not get very wonky routing for 
those relying heavily on routeservers  - this i have no idea how to coordinate.

but should it be done? Yes it should

-- 
hugge

---

From: Nick Hilliard <[email-masked]>
Date: Mon, 18 Dec 2017 13:04:03 +0000

> improve accountability, auditability & make debugging easier.

you missed the term "misconfigured" or "malconfigured" when describing
the route server in question.

> justifiable because it gives them a competitive advantage over transit

It's also common practice for transit providers to use a single ASN
spanning the globe e.g. 174, 2914, 3356, etc. What you're describing
here is an aspect of the fact that that as-pathlen has not been a useful
determinant for the bgp decision engine for many years.

> comparable IP networks for similar distances.

tbh, most don't - the vast majority are localised to well-constrained
metro areas and keep local interconnection local.  It would be unhelpful
to this majority if observations were made about one type of IXP, and
and extrapolations were imposed on topologically dissimilar platforms.

> should be revisited, perhaps deleted. Thoughts?

Updating rfc4271 would be more productive - and getting IXPs to filter
ingress bgp feeds by default.

Nick

---

From: Job Snijders <[email-masked]>
Date: Mon, 18 Dec 2017 14:13:06 +0100

Hi Nick,

> the route server in question.

Which route server in question? They are close to invisible. :)

> > route server in such a way that the route server does not append its
> > own ASN to the AS_PATH attribute. Many IXPs view this practise as
> > justifiable because it gives them a competitive advantage over
> > transit providers.
> 
> here is an aspect of the fact that that as-pathlen has not been a
> useful determinant for the bgp decision engine for many years.

A common deployment strategy is that folks connect 2 transit providers
and an IXP, and uppref whatever is announced via the IXP. Since LP wins
over AS_PATH lenght, I maintain that for the common use case the
addition of the route server's ASN in the AS_PATH has no negative
effects and quite some positive effects.

> > RFC 7947 reasons "a route server does not participate in the process
> > of forwarding data between client routers", but meanwhile quite some
> > IXPs have more equipment and layer-2 hops as part of the forwarding
> > path than comparable IP networks for similar distances.
> 
> metro areas and keep local interconnection local.  It would be
> unhelpful to this majority if observations were made about one type of
> IXP, and and extrapolations were imposed on topologically dissimilar
> platforms.

I don't think smaller IXPs are any less susceptible to problems than
large IXPs. In both instances it is helpful if there is visibility on
what AS (and who is in administrative control of that BGP speaker) was
involved.

Even IXPs that indicate they apply ingress filters could suffer from
software bugs (ghost routes) or filtering issues. In those type of cases
having better visiblity is of tremendous value.

And of course, if the AS_PATH length tie-breaker _does_ make for
concerns, people can (and will) set up direct bilateral sessions with
each other.

> > In summary, I think RFC 7947 section 2.2.2.1 & 2.2.2.2 were
> > misguided, should be revisited, perhaps deleted. Thoughts?
> 
> Updating rfc4271 would be more productive

What kind of update would you suggest?

> - and getting IXPs to filter ingress bgp feeds by default.

It would be great if we can easily identify which IXPs are not
filtering.

Kind regards,

Job

---

From: "i3D.net - Martijn Schmidt" <[email-masked]>
Date: Mon, 18 Dec 2017 13:23:29 +0100

Dear Job, Fredrik,

I disagree. The last thing we need is for IXPs to randomly begin
applying new "inject one's own ASN into AS_PATH" behaviour to their
routeservers - anything which deviates from the de-facto standard will
result in a lot of time spent with IXPs working on migrating to the new
thing. Just look at all the effort it takes to get a no-brainer like
proper ingress route filtering pushed through..

I'm happy that everything has finally gotten to the point where
route-servers consistently apply the AS_PATH stripping behaviour - and I
consider it a reason _not_ to participate in the route-servers if their
ASN is still visible in the AS_PATH. For example, this proposed change
would wreak a lot of havoc on those networks with a wide geographical
footprint relying on consistent route announcements to ensure traffic
doesn't experience scenic routing. During the migration period, longhaul
traffic would be all over the place as a result of the differing AS_PATH
lengths that are seen at the various ingress points of an autonomous
system participating in route-servers.

What I _would_ support is a semi-standardized or well-known BGP
community to indicate whether a prefix was, at some point in its
lifecycle, propagated by a routeserver on an IXP. Better yet, a BGP
Large Community so that it's not stripped by other "community-filtering"
networks before it reaches a point where the route can be stored into a
database for later forensics.

Best regards,
Martijn Schmidt

On 18-12-17 12:41, Fredrik Korsbäck wrote:
> This makes alot of sense actually. Now that we live in a world where a
> route-server and an IXP is not what it used to be we need to consider
> IXPs as any ISP out there. Heck, some of these IXPs runs bigger
> infrastructures then most of their customers do.
>
> Injecting the RS asn into the path makes troubleshooting much easier
> and helps us understand the routingt-able alot better, and having a RS
> ASN show up when backtracking an incident on f.e BGP Play will shorten
> the lead-time to figuring out what actually happened significantly.
>
> I do see a problem however that we kinda need to change this all at
> the same time to not get very wonky routing for those relying heavily
> on routeservers  - this i have no idea how to coordinate.
>

---

From: Nick Hilliard <[email-masked]>
Date: Mon, 18 Dec 2017 15:06:56 +0000

> justifiable because it gives them a competitive advantage over transit

This is both incorrect and mildly pejorative. The justification for
stripping the RS-ASN is that multilateral peering is (mostly)
functionally equivalent to bilateral peering.  If the RS-ASN is injected
into the AS path, then they become substantially non-equivalent, and
bilateral peering sessions become preferred.

This is not ideal: it's extremely unusual for bilateral peering over an
ixp ever to have any filtering, so if you push IXPs to inject the route
server ASN, this will create pressure to implement more bilateral
peering and consequently more problems in the longer term, which are
effectively impossible to deal with.

I can see why you think that injecting the rsasn into the as-path is
alluring, but be careful what you wish for, because fiddling with this
will have consequences which extend far outside your assumed scope.

At least in most cases, it's pretty straightforward these days to
implement route servers with full, dynamically rebuilt filtering out of
the box.  The technical work has been done.  What's left is to get the
remaining IXPs which don't filter their inbound prefixes to deploy this.

Nick

---

From: Job Snijders <[email-masked]>
Date: Mon, 18 Dec 2017 16:07:41 +0100

Dear Martijn,

On Mon, Dec 18, 2017 at 01:23:29PM +0100, i3D.net - Martijn Schmidt wrote:
> routeservers 

Interesting use of the word 'randomly'! :)

> - anything which deviates from the de-facto standard will result in a
> lot of time spent with IXPs working on migrating to the new thing.
> Just look at all the effort it takes to get a no-brainer like proper
> ingress route filtering pushed through..

Right now some IXPs are using the IETF RFCs as a justification to not
hide their ASN in the AS_PATH - and unfortunately some of them gloss
over the recommendations to apply ingress filters. I don't see a need to
continue endorsing a practise that doesn't benefit the Internet as
whole: people cherry-pick from the RFCs, there isn't much we can do
about that. But we can at least ensure that the the set of documents
describes an operating mode that allows others to do troubleshooting,
and ensures there is a degree of visiblity.

I mentioned before, I suspect that the route server's lack of visiblity
is a direct contributor to the reluctance to apply filters on route
servers. We can often observe that improved transparency leads to
improved operations.

Participants of the routing eco system that don't hold the best interest
of all other participants at heart should at least be visible so that
their choices can be researched.

> route-servers consistently apply the AS_PATH stripping behaviour - and
> I consider it a reason _not_ to participate in the route-servers if
> their ASN is still visible in the AS_PATH. For example, this proposed
> change would wreak a lot of havoc on those networks with a wide
> geographical footprint relying on consistent route announcements to
> ensure traffic doesn't experience scenic routing. During the migration
> period, longhaul traffic would be all over the place as a result of
> the differing AS_PATH lengths that are seen at the various ingress
> points of an autonomous system participating in route-servers.

I think you are somewhat overestimating the consequences. The internet
is constantly in flux and peers come & go at internet exchanges and
route servers. Already today you'll be monitoring for performance and
deploying routing policy accordingly. If one IXP has a route server
which inserts their ASN in the AS_PATH, and another doesn't (AND this
affects your business), you can just as easily level the playing field
by prepending the RS ASN yourself.

At this point in time most route servers can be considered the
equivalent of a medimum to large sized regional operator. We don't ask
normal IP transit networks to strip their ASN either. A route server
adding its ASN to the AS_PATH is not the end of the world - quite the
opposite - it enables others to make better choices.

> Large Community so that it's not stripped by other
> "community-filtering" networks before it reaches a point where the
> route can be stored into a database for later forensics.

Communities propagate less reliably compared to AS_PATH.

Some brought up in offlist conversations that the value of a route
server is that a new participant of the internet exchange will glean
immediate value from connecting to the IXP. These benefits and this
concept does not change one jot if the route server presents itself as
like any normal BGP speaker. It would be very strange to operate under
the mode that route servers immediately become worthless if 1 ASN is
added to the AS_PATH.

Networks that operate on a global scale will have to make their own
considerations anyhow as to how they their traffic distribution. Some
have a backbone, some don't. Whenever the exchange of traffic becomes a
more complex question, parties are expected to set up direct bilateral
sessions anyhow. Route servers being a more visible part of the
ecosystem doesn't change that.

Kind regards,

Job

---

From: Nick Hilliard <[email-masked]>
Date: Mon, 18 Dec 2017 15:32:13 +0000

Job Snijders wrote:
> over the recommendations to apply ingress filters. 

the two RS RFCs are split: one is a specification of what BGP must do,
and the other is a recommendation on operational issues.  Glossing over
the fact that IXP operators have not been inserting the rsasn since the
mid 1990s and that the RFCs are dated 2016, the document that IXPs need
to use is the operations doc: rfc7948, which says that RSs should be
operated in compliance with all the bits of rfc7454 that talk about
filtering. Stopping routing leaks is the aim of the game here.

> servers.

Job, seriously, if you have helpful or useful comments to make, then
please make them.

Nick

---

From: Job Snijders <[email-masked]>
Date: Mon, 18 Dec 2017 16:36:56 +0100

Hi Nick,

On Mon, Dec 18, 2017 at 03:06:56PM +0000, Nick Hilliard wrote:
> functionally equivalent to bilateral peering.

Which part is incorrect? The desire for functional equivalence with
bilateral peering, stems from the desire to be able to compete with
transit while at the same time providing a convenience service to the IX
participants (reduced workload by aggregating many sessions onto a
select few sessions). I don't view this is not a bad thing, internet
exchanges have brought significant improvements around the world.

> If the RS-ASN is injected into the AS path, then they become
> substantially non-equivalent, and bilateral peering sessions become
> preferred. This is not ideal: it's extremely unusual for bilateral
> peering over an ixp ever to have any filtering, 

If bilateral sessions exist in addition to the multilateral sessions,
the point is moot anyhow. If the RS blocks an invalid announcement,
it'll pass via the bilateral session.

> so if you push IXPs to inject the route server ASN, this will create
> pressure to implement more bilateral peering and consequently more
> problems in the longer term, which are effectively impossible to deal
> with.

It does not follows that insertion of an ASN in the AS_PATH will create
pressure to set up bilateral sessions. People have always had a choice
whether to use multiateral or bilateral or both, the set of options
doesn't change.

> will have consequences which extend far outside your assumed scope.

Such as?

> implement route servers with full, dynamically rebuilt filtering out
> of the box.  The technical work has been done.  What's left is to get
> the remaining IXPs which don't filter their inbound prefixes to deploy
> this.

It is quite laborious to track down which IXPs are not filtering, given
the lack of visiblity due to the AS hiding. Let's take an example from
RIPE RIS RRC03 updates.20171216.0300.gz:

	TIME: 12/16/17 03:00:06
	TYPE: BGP4MP/MESSAGE/Update
	FROM: 80.249.208.34 AS1103
	TO: 80.249.208.69 AS12654
	ORIGIN: INCOMPLETE
	ASPATH: 1103 7713 2914 286 34984
	NEXT_HOP: 80.249.208.34
	COMMUNITY: 0:65535 1103:101 1103:1000 1103:1011 2914:420 2914:1008 2914:2000 2914:3000 34307:50197 34307:60002 65504:286
	ANNOUNCE
	  212.252.176.0/20
	  213.153.247.0/24

AS 7713 inadvertently announced some of its peering & upstream routes to
other peering partners. Only through manual verification with AS 1103 we
could discover that there there is no bilateral session between 1103 and
7713, but a multilateral device is hiding between 1103 and 7713. This
could further be confirmed through PeeringDB where we see both ASNs
share a common IXP, and the IXP's RS ASN shows up in the BGP
communities: "34307:50197 34307:60002".

Life would be much simpler if we can simply read off our screen:

    AS_PATH: 1103 34307 7713 2914 286 34984

Kind regards,

Job

---

From: Job Snijders <[email-masked]>
Date: Mon, 18 Dec 2017 16:54:18 +0100

On Mon, Dec 18, 2017 at 03:32:13PM +0000, Nick Hilliard wrote:
> Job Snijders wrote:
> > I mentioned before, I suspect that the route server's lack of
> > visiblity is a direct contributor to the reluctance to apply filters
> > on route servers.
> 

You are free to disagree with my suspicions, but please ensure you
maintain a degree of decorum when doing so.

I've perceive advantages to transparency in many areas of business.
(well, in this context 'transparency' should be interpreted as 'explicit
visibility'). Oftentimes immediate improvement can be noticed when a
business operation is ranked against similar business, and reports are
published on performance indicators. This of course requires the
business to be visible in the first place.

The AS_PATH attribute serves multiple functions: its length is used as a
tie-breaker in best path selection, and the contents of the AS_PATH
itself serves as an (mutable) track record on what administrative
domains the announcement passed through.

You and Martijn appear to argue that the 'best path selection' component
should not be fiddled with, which leaves me wondering whether we have
any other methods to implement a track record ala 'this path
announcement passed through RS AS XYZ' other than communities. Or are
you of the opinion that the lack of visibility is not a problem to begin
with?

Respectfully,

Job

---

From: Nick Hilliard <[email-masked]>
Date: Mon, 18 Dec 2017 17:38:22 +0000

Job Snijders wrote:
> You and Martijn appear to argue that the 'best path selection'
> component should not be fiddled with, which leaves me wondering
> whether we have any other methods to implement a track record ala
> 'this path announcement passed through RS AS XYZ' other than
> communities. Or are you of the opinion that the lack of visibility is
> not a problem to begin with?

communities are low-hanging fruit and non-intrusive, and probably not a
bad thing to do.

If you plan to spend time and energy dealing with the underlying
problem, i.e. routing leaks, then I'd suggest continuing to work on
getting IXPs to implement prefix filtering on their route servers.  It's
laborious and thankless but will actually fix the problem in the longer
term - and it needs to be done anyway.

Nick

---

From: Alexander Azimov <[email-masked]>
Date: Tue, 19 Dec 2017 01:58:39 +0300

Hi all,

I'd like to share my support to Job's position, and the reason isn't only
'transparency' or filtering (while they are worth to fight for).

I see as the core of the problem, that one ISP MUST add its AS Number in
the AS_PATH and another ISP is permitted no to do it. Yes, one can tell me,
that IXP doesn't provide transit, it provides only local connectivity,
through L2 matrix, but does ISP becomes IXP if it provides local
connectivity? Imagine a situation when a customer has a choice: to purchase
partial transit (only customer cone) from ISP or to become a member of IX.
The price for both services could be quite comparable, but customer is
looking forward not only for lesser costs, but also for increase its
margin, and even if ISP will have all IX members as customers, it will be
less preferable because through IX the path will have -1 hop. Some ISPs are
already asking themselves a question, maybe they can also remove their AS
number, at least when announcing routes to customers?

And this pandora box has been already opened, I can name several ISPs that
provide transit and remove its ASN from AS_PATH attribute, some of them are
removing it even when announcing routes to upstreams. It's not right, it
violates BGP loop detection, but it has just same motivation as RS at IXP -
to make their routes more preferable, make 'virtual direct' connections.
Some of them even name themselves as IXPs.

In BGPSec it's getting even worse with pCount = 0. There is a statement in
section 7.2.:

> if a BGPsec speaker does not expect a peer AS to set its pCount=0 and
>    if an UPDATE message received from that peer violates this, then the
>    UPDATE message MUST be considered to be in error (see the list of
>    checks in Section 5.2).  See Section 8.4 for a discussion of security
>    considerations concerning pCount=0.

Well, I can assume that some transit providers will be checking this pCount
value, but I don't believe it will be checked from the customer side (it's
even impossible for all, except direct neighbor). It can end up with
everybody setting pCount=0 and AS_PATH attribute will lose much of its
current use cases.

2017-12-18 20:38 GMT+03:00 Nick Hilliard <[email-masked]>:

> _______________________________________________
> GROW mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/grow
>

-- 
| Alexander Azimov  | HLL l QRATOR
| tel.: +7 499 241 81 92
| mob.: +7 915 360 08 86
| skype: mitradir
| mailto: [email-masked]
| visit: www.qrator.net

---

From: Jared Mauch <[email-masked]>
Date: Mon, 18 Dec 2017 18:47:28 -0500

> On Dec 18, 2017, at 12:38 PM, Nick Hilliard <[email-masked]> wrote:
> 
> Job Snijders wrote:

I know that Job has been pushing for the above, perhaps not in your view
but in the view of others here.

I do think that having a RS emit a community saying “RS_ASN:xxx” would be
of value.  As mentioned in other emails, finding these edges can be quite
complex when doing operations.

If we continue to see the active threats, market forces will dictate the
resulting implementations.

I’d like to see improvements in routing security, but the path forward is murky.

Aside from AS_PATH & Communities, are there other ideas?

- Jared

---

From: heasley <[email-masked]>
Date: Tue, 19 Dec 2017 03:06:29 +0000

Mon, Dec 18, 2017 at 04:36:56PM +0100, Job Snijders:
> pressure to set up bilateral sessions.

Is that negative?  Why not encourage ASes to have direct relationships
and stop maintaining route servers?  They should develop those relationships
and it might even be suggested that security and trouble shooting could
rely upon it.

---

From: Nick Hilliard <[email-masked]>
Date: Tue, 19 Dec 2017 08:22:48 +0000

> but in the view of others here.

definitely.  It's been incredibly productive work, and his efforts have
produced major results in terms of pushing IXPs towards filtering.  This
needs to be continued.

Nick

---

From: Gert Doering <[email-masked]>
Date: Tue, 19 Dec 2017 09:25:34 +0100

Hi,

> and it might even be suggested that security and trouble shooting could

Economies of scale.  DECIX has over 600 active participants, and I have
little interest in talking to about 550 of them - but my routers would
appreciate having direct routes to the networks behind them.

So, route servers are a very (very!) welcome feature.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Nick Hilliard <[email-masked]>
Date: Tue, 19 Dec 2017 08:40:46 +0000

> and it might even be suggested that security and trouble shooting could

in general, yes, this is negative.  If you have the resourcing to deploy
proper automation, including dynamic filtering and session management,
then bilateral is better.  If you don't, then properly managed route
servers will provide a better experience in practice.

Nick

---

From: Gert Doering <[email-masked]>
Date: Tue, 19 Dec 2017 09:45:36 +0100

Hi,

On Tue, Dec 19, 2017 at 08:40:46AM +0000, Nick Hilliard wrote:
> If you have the resourcing to deploy
> then bilateral is better.  

To emphasize some aspect of this for the non-operators on this list: the
technical bits of this are manageable.  Talking to 500 other ISPs that
are not responding to e-mail, do not have anyone in the same time zone,
generally lack understanding of BGP, etc. - this is what really costs,
and where a well-maintained route server really shines.

(And then, there's "poor BGP implementations that just do not scale
properly with 600+ peers on the same router"...  like, "most of them")

If there were only 10 other ISPs at the IXPs we connect to, we could
do without route servers :-)

Gert Doering
        -- NetMaster

---

From: Job Snijders <[email-masked]>
Date: Tue, 19 Dec 2017 09:53:08 +0100

On Tue, Dec 19, 2017 at 09:45:36AM +0100, Gert Doering wrote:
> > If you have the resourcing to deploy proper automation, including
> > dynamic filtering and session management, then bilateral is better.  
> 
> To emphasize some aspect of this for the non-operators on this list:
> the technical bits of this are manageable.  Talking to 500 other ISPs
> that are not responding to e-mail, do not have anyone in the same time
> zone, generally lack understanding of BGP, etc. - this is what really
> costs, and where a well-maintained route server really shines.
> 

Yes, route servers can be very useful, no question about it. I think
their value as a service would increase if they become visible
participants of the routing ecosystem.

Gert, do you think it would affect your operations if the route server
would insert its ASN into the AS_PATH?

Would you be willing to run a test? In your routing policy you can
append "6695" on inbound and prepend your own AS on outbound to simulate
what it would be like if that RS would insert itself.

Kind regards,

Job

---

From: Christoph Loibl <[email-masked]>
Date: Tue, 19 Dec 2017 10:43:57 +0100

Hi,

Feedback on suggested path changes from another operator:

I imagine that in our case having *all* route-servers consistently add their ASN would not lead to too many path changes. The changes that we will see is that we will prefer direct peerings over route server prefixes because of AS-Path length (today it is another metric that is the tie breaker, since AS-path length is equal). 

For our transit customers however, this may be a slightly different story because they suddenly see the direct peerings with a shorter as-path than routes propagated via route servers and may select different paths.

Suggestion:

I would prefer a solution that has no impact on the as-path: 

> On 19.12.2017, at 00:47, Jared Mauch <[email-masked]> wrote:
> 

However, communities are very fragile when it comes to route propagation. Removing / modifying communities is happening everywhere (mostly at AS borders). I would not like to rely on BGP communities for this.
 
We have a solution to a slightly similar problem in iBGP route-reflection. For that purpose we added an optional non-transitive attribute CLUSTER_LIST. I suggest a similar approach for IXP route-servers (ROUTESERVER_LIST optional,transitive = list of route-server ASNs that have propagated the prefix). That seems less fragile and the implementation changes are limited to the route-servers themselves.

Christoph

---

From: Gert Doering <[email-masked]>
Date: Tue, 19 Dec 2017 10:50:29 +0100

Hi,

> participants of the routing ecosystem.

Not sure about that.  IXP participants know where the route is coming
from, and why should it be of anyone else's concern whether I have a
direct peering with you, or use a RS for it?

> would insert its ASN into the AS_PATH?

Yes.  BGP works by comparing AS path lengths, you know :-) - and that
would cause significant work as direct peerings would automatically
have shorter AS paths than via-route-server peerings.

Thus, traffic shift, which needs to compensated - like, by adding 
extra prepends to direct peerings - which would have consequences for
the AS path lenghts seen by our customers.

In other words: we asked for AS-Path transparent RSes 15 years ago, and 
this is still what we want today.

Gert Doering
        -- NetMaster

---

From: Wolfgang Tremmel <[email-masked]>
Date: Tue, 19 Dec 2017 10:04:36 +0000

Hi,

first a remark: At DE-CIX customers can choose if they want the RS AS in the path of the prefixes they receive from the RS or not (they need to send an email to support to change the config).

Another thought - please correct me if I am talking rubbish as I have only glanced at the RFCs...

If the RS would insert itself as an aggregator with the leftmost AS in the path: 
The path would not become any longer as AS_SETs count as one hop and the RS would be visible... 

cheers
Wolfgang

-- 
Wolfgang Tremmel, Head of DE-CIX Academy                     

Phone +49 69 1730902 0 | Fax +49 69 4056 2716 | [email-masked]
Geschaeftsfuehrer Harald A. Summa | Registergericht AG Köln HRB 51135
DE-CIX Management GmbH | Lindleystrasse 12 | 60314 Frankfurt am Main | Germany | www.de-cix.net

---

From: Job Snijders <[email-masked]>
Date: Tue, 19 Dec 2017 11:07:55 +0100

> direct peering with you, or use a RS for it?

When route leaks or hijacks occur, I approach all ASNs in the AS_PATH to
figure out who originated it and which ASN(s) failed to apply filters. 

As I'm stating in this thread (ad nauseam), if the ASN of an involved
BGP operator is not visible, I can't approach them and help them improve
their operations.

When we look at a blogpost like https://radar.qrator.net/blog/born-to-hijack
it is easy to see that it is of paramount importance to ensure filters
are applied on _every_ boundary in the AS_PATH, also the ones that are
currently invisible.

> > Gert, do you think it would affect your operations if the route
> > server would insert its ASN into the AS_PATH?
> 
> In other words: we asked for AS-Path transparent RSes 15 years ago,
> and this is still what we want today.

If I can summarize the above, you state "fiddling with the as_path
length would negatively affect my operations" - but do you agree with my
concern that a lack of visibility leads to a lack of accountability?

Kind regards,

Job

---

From: Nick Hilliard <[email-masked]>
Date: Tue, 19 Dec 2017 12:36:57 +0000

Alexander,

I share your concerns, but we're not really discussing ISP filtering
politics, and there is a huge difference between an ISP providing a
partial peering transit service and an IXP.

The difference is that an ISP necessarily makes all the decisions about
what prefixes you receive as a customer, because you get a single bgp
feed from the ISP.  This means that the customer is 100% beholden to the
ISP about the selection of prefixes that is transmitted.

An IXP provides a platform and allows participants to make up their own
minds about what works best for them.  If party A wants to interconnect
with party B, then the IXP shouldn't interfere.  Same if party C doesn't
want to interconnect with party D.  It's their business and they can
make their own decisions.

The reason that IXPs are disinclined to insert the rsasn in a route
server feed is that a route server attempts to replicate what you would
see if you had bilateral peering sessions to all other RS clients.  If
the IXP operator inserts the rsasn, it's materially changing the bgp
feed and breaking the rough equivalence between bilateral and
multilateral peering.  This is generally considered to be a bad thing,
both by IXPs and by IXP participants.

Regarding loop detection, this isn't an issue at an IXP RS because the
RS does not participate in traffic forwarding.

If you have concerns about ISP behaviour, this isn't really something
that the IETF can deal with, except to say that stripping out the
next-hop ASN is stupid, dangerous and likely to end up causing routing
loops.  No doubt the people who did this are already aware of this, though.

Regarding Job's concerns, I agree that it would be more helpful if we
had some easy way of figuring out if there were a route server involved
in prefix propagation, but his proposed fix for this comes at a cost
which I think is too high for IXPs and IXP participants.  That, and the
fact that it doesn't solve the root problem relating to filter deployment.

Nick

Alexander Azimov wrote:
> Hi all,
> 
> I'd like to share my support to Job's position, and the reason isn't
> only 'transparency' or filtering (while they are worth to fight for).
> 
> the AS_PATH and another ISP is permitted no to do it. Yes, one can tell
> me, that IXP doesn't provide transit, it provides only local
> connectivity, through L2 matrix, but does ISP becomes IXP if it provides
> local connectivity? Imagine a situation when a customer has a choice: to
> purchase partial transit (only customer cone) from ISP or to become a
> member of IX. The price for both services could be quite comparable, but
> customer is looking forward not only for lesser costs, but also for
> increase its margin, and even if ISP will have all IX members as
> customers, it will be less preferable because through IX the path will
> have -1 hop. Some ISPs are already asking themselves a question, maybe
> they can also remove their AS number, at least when announcing routes to
> customers? 
> 
> And this pandora box has been already opened, I can name several ISPs
> that provide transit and remove its ASN from AS_PATH attribute, some of
> them are removing it even when announcing routes to upstreams. It's not
> right, it violates BGP loop detection, but it has just same motivation
> as RS at IXP - to make their routes more preferable, make 'virtual
> direct' connections. Some of them even name themselves as IXPs. 
> 
> In BGPSec it's getting even worse with pCount = 0. There is a statement
> in section 7.2.:
> 
> Well, I can assume that some transit providers will be checking this
> pCount value, but I don't believe it will be checked from the customer
> side (it's even impossible for all, except direct neighbor). It can end
> up with everybody setting pCount=0 and AS_PATH attribute will lose much
> of its current use cases.
> 
> 2017-12-18 20:38 GMT+03:00 Nick Hilliard <[email-masked]
> <mailto:[email-masked]>>:
> 
>     Job Snijders wrote:
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/grow>
> 
> 
> 
> 
> -- 
> | mailto: [email-masked] <mailto:[email-masked]>
> | visit: www.qrator.net <http://www.qrator.net/>

---

From: <[email-masked]>
Date: Tue, 19 Dec 2017 12:54:10 +0000

My 2 cents

> From: Job Snijders
[...]
 > domains the announcement passed through.
 
Good point.
I'm wondering if decoupling both would not be cleaner, and would avoid the incentive to "tweak" the AS Path.
One option would be to keep the AS Path for loop detection, administrative record, and policies.
Inter domain metric used for best path selection could be left for another field/data/extension. Yet given the incentive to tweak such data, we could even realize that since we can't trust it, we don't really need it. We could only let the end user (originator) express some high level policy (nominal, backup1, backup2) using well known communities. (which indeed may also be removed/changed by transit AS, but the end user could see it and negotiate transparency though a contract).

Best regards,
--Bruno

 > you of the opinion that the lack of visibility is not a problem to begin

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: [email-masked] (Marco d'Itri)
Date: Tue, 19 Dec 2017 14:28:42 +0100

> both by IXPs and by IXP participants.
I agree. I understand that Job's goal is to be able to identify the RSes 
in the path, and this can be done with much less operational troubles
with a well known community.

-- 
ciao,
Marco

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 19 Dec 2017 15:02:17 -0500

> determinant for the bgp decision engine for many years.

While somewhat orthogonal to this discussion, path length (and thus
prepending) is about the only useful knob many BGP speakers have to try to
bias incoming traffic.  I suspect you mean something a bit different above.

> ingress bgp feeds by default.

I hope to be retired before that level of "incremental update" is expected
to work in the Internet at large. :-)

-- Jeff

P.S. many providers provide knobs to ignore path length as a consideration.
No spec work is required.

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 19 Dec 2017 15:06:59 -0500

> The path would not become any longer as AS_SETs count as one hop and the RS would be visible... 

This is a somewhat perverse hack... and work quite nicely. :-)

That said, sets offended enough people on this list and elsewhere (makes
bgpsec problematic) that it resulted in RFC 6472.

-- Jeff

---

From: Nick Hilliard <[email-masked]>
Date: Tue, 19 Dec 2017 20:14:13 +0000

> bias incoming traffic.  I suspect you mean something a bit different above.

point taken, but I meant for outgoing: it's not useful when you have
upstream or peer global asns for outbound traffic.

> to work in the Internet at large. :-)

Probably hell would freeze over before consensus was reached on a
replacement decision engine.

> No spec work is required.

yeah, but you know dealing with ixp participants is hard.  IXPs are
firmly aimed at the mid- to low-end provider market, and often you're
dealing with people who just don't understand routing well or who don't
have time to twiddle knobs.  It's better policy to provide ixp
participants with simple guidelines and a well-managed 90% solution
rather than spend all day pushing them up the hill.

Nick

---

From: Gert Doering <[email-masked]>
Date: Tue, 19 Dec 2017 21:17:09 +0100

Hi,

> rather than spend all day pushing them up the hill.

I like being pushed uphill :-) - but if I don't even have to *get* up
that hill, even better.  So, "what Nick says".

Gert Doering
        -- lazy ISP operator

---

From: "UTTARO, JAMES" <[email-masked]>
Date: Tue, 19 Dec 2017 21:50:36 +0000

Interesting discussion, coming from the VPN space the selection of an AS as a global or regional value has been a long discussion.. For better or worse AS value per region forces a different paradigm than AS per global.. In the VPN space we take great pains to not affect the AS or AS-Path at is meaningful to customers. IMO AIGP is for more useful for selection for certain applications i.e 3107..

Jim Uttaro

-----Original Message-----
From: GROW [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Tuesday, December 19, 2017 3:02 PM
To: Nick Hilliard <[email-masked]>
Cc: [email-masked]; Job Snijders <[email-masked]>
Subject: Re: [GROW] Route Server ASN stripping hiding considered harmful?

-- Jeff

_______________________________________________
GROW mailing list
[email-masked]
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_grow&d=DwICAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=3qhKphE8RnwJQ6u8MrAGeA&m=pj_0O26biD9_MZUbccTTQ2LlWH2PMPXLuRPgQU0olFs&s=8K5jqMaS-4xLe7x_zFLoLUZOCAkHEMuARWekULTt39A&e=

---

From: Randy Bush <[email-masked]>
Date: Wed, 20 Dec 2017 10:07:31 +0100

if the rs injects it as, then traffic will be biased against rs paths

randy