Subject: Re: [GROW] New Version Notification for draft-hilliard-grow-no-export-via-rs-00.txt

From: Nick Hilliard <[email-masked]>
Date: Mon, 09 Oct 2017 15:02:04 +0100

New ID submission, as below.  This is to solve a real world problem.

Comments welcome.

Nick

[email-masked] wrote:
> A new version of I-D, draft-hilliard-grow-no-export-via-rs-00.txt
> has been successfully submitted by Nick Hilliard and posted to the
> IETF repository.
> 
> Name:		draft-hilliard-grow-no-export-via-rs
> Revision:	00
> Title:		The BGP NO_EXPORT_VIA_RS Community for Route Servers
> Document date:	2017-10-08
> Group:		Individual Submission
> Pages:		5
> URL:            https://www.ietf.org/internet-drafts/draft-hilliard-grow-no-export-via-rs-00.txt
> Status:         https://datatracker.ietf.org/doc/draft-hilliard-grow-no-export-via-rs/
> Htmlized:       https://tools.ietf.org/html/draft-hilliard-grow-no-export-via-rs-00
> Htmlized:       https://datatracker.ietf.org/doc/html/draft-hilliard-grow-no-export-via-rs-00
> 
> 
> Abstract:
>    This document describes a BGP Well-known Community called
>    NO_EXPORT_VIA_RS.  This community allows BGP route server clients to
>    instruct an Internet Exchange BGP Route Server to tag prefixes
>    destined for other route server clients with the NO_EXPORT Well-Known
>    Community.  This mechanism allows route server clients to
>    transitively control distribution of their prefixes in other
>    Autonomous Systems connected to the Internet Exchange Route Server.
> 
> 
>                                                                                   
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat
>

---

From: Wolfgang Tremmel <[email-masked]>
Date: Mon, 9 Oct 2017 14:56:57 +0000

Hi,

just reading your draft, a few remarks, speaking as a private internet citizen:

"If a route server client announces a prefix tagged with both the
  NO_EXPORT and NO_EXPORT_VIA_RS communities to a route server, the
  route server MUST ignore the NO_EXPORT community,"

--> that means that NO_EXPORT_VIA_RS is "stronger" then NO_EXPORT and actually changes the meaning of NO_EXPORT.
I know that users make stupid mistakes and this should perhaps cover the mistake of setting both, but IMHO with NO_EXPORT being much older and really well understood and used, I think NO_EXPORT should take precedence.

(IMHO NO_EXPORT should be interpreted, and never passed through, independent of a router or a route server is used)

My first idea when reading this was why not make it general, why limit the community to route servers and not have a "SET_NO_EXPORT" community which directs the next hop AS to set NO_EXPORT?
( I already sent this email to Nick and his answer that its limited to RS because otherwise some kind of TTL needs to be implemented makes sense)

cheers,
Wolfgang

> On 9. Oct 2017, at 16:02, Nick Hilliard <[email-masked]> wrote:
> 
-- 
Wolfgang Tremmel                     

Phone +49 69 1730902 26 | Fax +49 69 4056 2716 | Mobile +49 171 8600 816 | [email-masked]
Geschaeftsfuehrer Harald A. Summa | Registergericht AG Köln HRB 51135
DE-CIX Management GmbH | Lindleystrasse 12 | 60314 Frankfurt am Main | Germany | www.de-cix.net

---

From: Nick Hilliard <[email-masked]>
Date: Mon, 09 Oct 2017 22:52:51 +0100

Wolfgang Tremmel wrote:
> --> that means that NO_EXPORT_VIA_RS is "stronger" then NO_EXPORT and
> actually changes the meaning of NO_EXPORT.
> 
> I know that users make stupid mistakes and this should perhaps cover
> the mistake of setting both, but IMHO with NO_EXPORT being much older
> and really well understood and used, I think NO_EXPORT should take
> precedence.
> 
> (IMHO NO_EXPORT should be interpreted, and never passed through,
> independent of a router or a route server is used)

According to rfc7947, it is a matter of local policy whether NO_EXPORT
is interpreted or passed through.  This has always been the case in the
IXP world: some IXP route servers interpret NO_EXPORT and some pass it
through. So it's not correct to say that the behaviour of NO_EXPORT is
well-understood when it comes to route servers - this topic has been the
subject of a good deal of debate in the IXP community over the years,
with no resolution.

The approach we're taking with NO_EXPORT_VIA_RS is to effectively say:
"we recommend not using NO_EXPORT because you don't know what it's going
to do.  However, if you choose to use it in combination with
NO_EXPORT_VIA_RS, it's clear what your intention is, and the lack of
ambiguity with NO_EXPORT_VIA_RS when announced to a RS means that it
should override NO_EXPORT, in this and only this situation".

> My first idea when reading this was why not make it general, why
> limit the community to route servers and not have a "SET_NO_EXPORT"
> community which directs the next hop AS to set NO_EXPORT?

> ( I already sent this email to Nick and his answer that its limited
> to RS because otherwise some kind of TTL needs to be implemented
> makes sense)

Limiting the community to route servers is necessary because otherwise
you'd need to implement a bgp-path TTL in order to limit the radius of
interpretation.  I don't think you could safely use the as-path for
determining how far the prefix should propagate.  Also there would be
other problems with prefix distribution which would be hard to solve,
e.g. what would happen in situations where an ASN's routing kit wasn't
updated to interpret this correctly?  The prefix would be propagated
across asn boundaries according to standard policy, but the NO_EXPORT
interpretation might hit a couple of ASNs away, in an unexpected place.
 I don't see how it would be possible to stop this from happening.

At least if there is a requirement that this is only interpreted on
route servers and then religiously deleted on export, the scope for
creating problems is a good deal smaller.

Nick

---

From: Christopher Morrow <[email-masked]>
Date: Mon, 9 Oct 2017 21:14:52 -0400

Do you all want to chat about this in singapore? or just keep discussing
on-list?
do you seek WG Adoption of the draft 'now' or would you like to chat about
it a bit more first?

On Mon, Oct 9, 2017 at 5:52 PM, Nick Hilliard <[email-masked]> wrote:

> _______________________________________________
> GROW mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/grow
>

---

From: Job Snijders <[email-masked]>
Date: Tue, 10 Oct 2017 03:46:49 +0000

Hi nick,

Have you considered just updating RFC 7947 to resolve the described
ambiguity by stating that a route server SHOULD pass the NO_EXPORT
community unaltered, rather than interpret it or block it?

The advance of this approach would be that a portion of deployed route
servers are already compliant, as where using a new NO_EXPORT_VIA_RS
community would require every route server to be updated.

>From the route server client perspective it may also be simpler if there is
just one “no export function” community instead of multiple.

Why is there no consensus amongst route server operators on what the
correct behavior is? Can you provide a citation?

Kind regards,

Job

On Mon, 9 Oct 2017 at 16:02, Nick Hilliard <[email-masked]> wrote:

> > URL:
> https://www.ietf.org/internet-drafts/draft-hilliard-grow-no-export-via-rs-00.txt
> > Status:
> https://datatracker.ietf.org/doc/draft-hilliard-grow-no-export-via-rs/
> > Htmlized:
> https://tools.ietf.org/html/draft-hilliard-grow-no-export-via-rs-00
> > Htmlized:
> https://datatracker.ietf.org/doc/html/draft-hilliard-grow-no-export-via-rs-00
> >
> >
> > Abstract:

---

From: Wolfgang Tremmel <[email-masked]>
Date: Wed, 11 Oct 2017 11:41:11 +0000

Hello,

> On 10. Oct 2017, at 05:46, Job Snijders <[email-masked]> wrote:
> 
> Have you considered just updating RFC 7947 to resolve the described ambiguity by stating that a route server SHOULD pass the NO_EXPORT community unaltered, rather than interpret it or block it?

From my reading of RFC1997 and RFC7947 the ambiguity is not really one:

RFC1997 states that any community-aware BGP speaker MUT NOT advertise prefixes received with NO_EXPORT
--> a route server is a BGP speaker
--> it is community aware

RFC7947 uses wording SHOULD NOT and MAY which IMHO are weaker. 

> 
> 
> Why is there no consensus amongst route server operators on what the correct behavior is? Can you provide a citation?
> 

all that aside, DE-CIX already has all that functionality using our own communities (we actually have two, one for adding NO_EXPORT and one for adding NO_ADVERTISE, and also allow selective adding using Large Communities):
https://www.de-cix.net/en/locations/united-states/dallas/routeserver-guide

best regards
Wolfgang

-- 
Wolfgang Tremmel

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 11 Oct 2017 13:04:49 +0100

It's formally ambiguous:

>    If a Communities attribute is intended for processing by the
>    route server itself, as determined by local policy, it MAY be
>    modified or removed.

i.e. the ixp operator explicitly has the option to interpret communities
or not.  This reflects the operational reality that some IXPs have
chosen to interpret and some to pass through.  Each has carefully
considered reasons for doing what they do.

>From a practical point of view, without a statement of intent from the
IXP operator, you cannot know whether a particular IXP will interpret
NO_EXPORT or forward it.

Because of this practical ambiguity, clarity would be useful.

Nick

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 11 Oct 2017 13:27:37 +0100

we've discussed this several times in the IXP community, with no general
agreement.  In terms of public positions, for example:

- DE-CIX honours RFC1997 to the letter:

> The well-known BGP Communities NO-EXPORT (65535:65281) and
> NO-ADVERTISE (65535:65282) are also honored meaning that a BGP
> announcement marked by one of these communities is not re-distributed
> to any peer.

(https://www.de-cix.net/en/locations/united-states/dallas/routeserver-guide)

- Netnod documented problems with this for their DNS root server, while
expressing the opposite opinion to DE-CIX

> Some route-servers honors the communities as if they where the receivers (like 
> no-export - duh!).  

see page 24 of
https://www.netnod.se/sites/default/files/aboutnetnod/Netnod_updates/Netnod_update_spring_meeting_2014.pdf

- MSK-IX realised the ambiguity and rolled their own
(https://kb.msk-ix.ru/en/ix/services/route-server/)

Other IXPs go one way or the other.

There really is no consistency here, hence the I-D.

Nick

Job Snijders wrote:
> From the route server client perspective it may also be simpler if there
> is just one “no export function” community instead of multiple. 
> 
> On Mon, 9 Oct 2017 at 16:02, Nick Hilliard <[email-masked]
> <mailto:[email-masked]>> wrote:
> 
>     [email-masked] <mailto:[email-masked]> wrote:
>     > until the htmlized version and diff are available at
>     tools.ietf.org <http://tools.ietf.org>.
>     >
>     > The IETF Secretariat
>     >
> 
>     [email-masked] <mailto:[email-masked]>

---

From: Job Snijders <[email-masked]>
Date: Wed, 11 Oct 2017 14:35:13 +0200

> RFC7947 uses wording SHOULD NOT and MAY which IMHO are weaker. 

OK, but you are not explaining to me why the current set of RFC's can't
be updated to encourage different NO_EXPORT behaviour on route servers
(compared to the rest of the BGP speakers), but instead a new community
is required.

> all that aside, DE-CIX already has all that functionality using our
> own communities (we actually have two, one for adding NO_EXPORT and
> one for adding NO_ADVERTISE, and also allow selective adding using
> Large Communities):

This doesn't tell me why there is no consensus :)

Kind regards,

Job

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 11 Oct 2017 13:46:01 +0100

> (compared to the rest of the BGP speakers), but instead a new community

it's because there is no consensus in the IXP world and there are a lot
of deployed instances which would need to be changed one way or another.

> This doesn't tell me why there is no consensus :)

because:

- the RFCs are formally ambiguous, dating way back to RFC1863 (which is
acknowledged to be experimental, but documented the practices going back
to the mid 1990s)

- each IXP which has gone one way or the other has carefully considered
reasons for doing what they did

- because of this, it will not be possible to reach consensus on
interpreting NO_EXPORT or passing it through (we have tried)

- the authors of this ID believe that the only way to fix this problem
is to acknowledge that it cannot be fixed, then move on.

Nick

---

From: Job Snijders <[email-masked]>
Date: Wed, 11 Oct 2017 14:59:29 +0200

Hi all,

> because:
>
> - the RFCs are formally ambiguous, dating way back to RFC1863 (which
> is acknowledged to be experimental, but documented the practices going
> back to the mid 1990s)
>
> - each IXP which has gone one way or the other has carefully
> considered reasons for doing what they did
>

Alright, it seems an Internet-Draft is warranted to clarify or improve
what route servers should do with certain well-known communities. I
support solving that problem.

The way I look at it is quite simple, let's assess how big the
populations are that need to do work:

Option A) An I-D that clarifies the behaviour of NO_EXPORT on a RS (and
          updates RFC1997 and RFC7947)

Option B) An I-D that defines behaviour for a new flavor of NO_EXPORT
          (NO_EXPORT_VIA_RS) on a RS (and updates RFC1997 and RFC7947)

Option A's impact:
    * A few Internet Exchange RS operators need to deploy a new routing
      policy.

Option B's impact:
    * Every non-RFC7947 EBGP speaker needs to start scrubbing the new
      NO_EXPORT_VIA_RS community. 
    * Every IXP RS operator needs to deploy a new configuration.
    * Every IXP RS participant needs to memorize that there is a new
      specific NO_EXPORT-ish community.

In other words, I don't think it is fair to follow option B
(draft-hilliard-grow-no-export-via-rs-00) because it causes the most
work.

For the route server operators that want to follow RFC 1997 to the
letter: honoring NO_EXPORT on a RS is a fringe activity. If there really
is a usecase to do so the IX should offer a locally significant
community. Also realize that if RFC 1997 is updated according to option
A, not honoring NO_EXPORT means you'll be compliant with the standards.

Option A aligns with the principle of least astonishment. I look forward
to draft-hilliard-grow-no-export-via-rs-01 which drops the definition of
a new NO_EXPORT-ish community and simply defines the pragmatic,
participant-friendly behaviour that RFC7947 BGP speakers should apply.

Kind regards,

Job

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 11 Oct 2017 14:44:57 +0100

>     * A few Internet Exchange RS operators need to deploy a new routing

Then you should speak directly to the IXPs who disagree with your
opinions and talk them into changing their opinions.  As I said, this
has already been tried and it didn't work, but no-one is going to get in
your way if you want to try again.

Nick

---

From: Job Snijders <[email-masked]>
Date: Wed, 11 Oct 2017 14:56:42 +0000

On Wed, 11 Oct 2017 at 15:45, Nick Hilliard <[email-masked]> wrote:

> Then you should speak directly to the IXPs who disagree with your opinions
> and talk them into changing their opinions.  As I said, this
> has already been tried and it didn't work, but no-one is going to get
> in your way if you want to try again.

Wouldn’t GROW be an excellent forum for that conversation, since GROW
brings together both IXP operators and network operators?

Of course I’d love to hear from IXP RS operators who think that “Option A”
is not feasible. What use case is there to strictly honor NO_EXPORT as
described in RFC 1997 on a route server? Can they describe how that is in
the best interest of the IXP RS participants?

Kind regards,

Job

Ps. You say “this has been tried before”, but I’m not aware of such
efforts. Was this perhaps because the topic was discussed in a non-public
special-interest group?

---

From: Job Snijders <[email-masked]>
Date: Tue, 17 Oct 2017 10:27:09 +0200

Dear all,

For route server operators who wonder how they can transparently
passthrough NO_EXPORT / NO_ADVERTISE, the arouteserver configuration
generator now supports passthrough mode for both BIRD and OpenBGPD.

It is implemented as following:

https://github.com/pierky/arouteserver/commit/4b861b3d22b68efc758af8e49ce507c723bd0ee1

Kind regards,

Job

On Wed, Oct 11, 2017 at 4:56 PM, Job Snijders <[email-masked]> wrote:
> Wouldn’t GROW be an excellent forum for that conversation, since GROW brings
> together both IXP operators and network operators?
>
> Ps. You say “this has been tried before”, but I’m not aware of such efforts.
> Was this perhaps because the topic was discussed in a non-public
> special-interest group?