Subject: Re: [Idr] Request to adopt draft-heitz-idr-large-community - Working Group Adoption call (9/6 to 9/20)

From: "Susan Hares" <[email-masked]>
Date: Tue, 6 Sep 2016 12:54:59 -0400

Job:

We'd be glad to start a WG adoption poll starting today (9/6 to 9/20).  

The Authors should indicate if they have IPR or know of any IPR.   If the
authors indicate IPR within 1-2 days, we'll let the poll continue.  If not,
we'll halt this adoption poll to await the authors response. 

Thank you, 

Sue 

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Job Snijders
Sent: Tuesday, September 6, 2016 7:39 AM
To: [email-masked]
Subject: [Idr] Request to adopt draft-heitz-idr-large-community

Dear IDR, fellow network operators,

I would like to request that the IDR Working Group adopts
draft-heitz-idr-large-community [1] as a working group document.

Background
----------
RFC1997 BGP communities are the most common method to signal
meta-information between autonomous systems. RFC1997 communities are a
32 bit entity. The convention is that the first 16 bits are the ASN in which
the last 16 bits have a meaning. RFC1997 is so popular because of its
elegant simplicity and ubiquitous support.

The operator community (no pun intended!) is suffering from a fatal flaw.
One in five ASNs in the Default-free zone are a 4-byte ASN (RFC 4893). One
cannot fit a 32-bit value into a 16-bit field.

4-byte ASN Operators work around this issue by either resorting to kludges
such as using private 16-bit ASNs as in the "Global Administrator" field, or
by returning the ASN to their respective RIR and requesting a 16-bit ASN.
However, both the RIRs and the IANA have depleted their supply of 16-bit
ASNs.

Work to address the issue of BGP communities has been ongoing for years.
Notable examples are 'flexible communities' (12 years ago) and 'wide
communities' (6 years ago). The WG so far has been unable to produce an
internet standard which enjoys a status similar to RFC1997. Now that the
RIRs are running out, the issue has become a matter of extreme urgency.

The Large BGP Community specification gives every network operator
(regardless of whether they have a 2-byte ASN or a 4-byte ASN) 8 bytes to
signal meta-information in an opaque fashion. This will align with current,
well-established practices deployed by network operators.

The Large BGP Community has purposefully been specified to be narrow and as
simple as possible to meet the operator community immediate needs, without
dissuading from existing community extensions that are in the standards
process pipeline.

The Large Community, by design, is not extendable, because extensibility
comes at a cost. Knowing that the amount of noise generated by an idea is
inversely proportional to the complexity of the idea, I urge the WG to
consider the Large Community's simplicity not a disadvantage, but a virtue.

We ask for your support in this narrow focus to re-imagine the RFC1997
communities in this way as it should have been done when RFC4893 was
published.

Kind regards,

Job Snijders
(co-author draft-heitz-idr-large-community)

[1]: https://tools.ietf.org/html/draft-heitz-idr-large-community

_______________________________________________
Idr mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/idr

---

From: Keyur Patel <[email-masked]>
Date: Tue, 6 Sep 2016 16:58:01 +0000

Not aware of any IPR.

Sent from my iPhone

> On Sep 6, 2016, at 9:56 AM, Susan Hares <[email-masked]> wrote:
> 
> Job:
>

---

From: Job Snijders <[email-masked]>
Date: Tue, 6 Sep 2016 19:02:08 +0200

Dear Susan,

> We'd be glad to start a WG adoption poll starting today (9/6 to 9/20).

Thanks!

> we'll halt this adoption poll to await the authors response.

I am not aware of any IPR.

Kind regards,

Job

---

From: Ignas Bagdonas <[email-masked]>
Date: Tue, 6 Sep 2016 19:11:18 +0100

Hi Sue,

I do not have any IPR and not aware of any IPR.

Ignas

---

From: "Simpson, Adam 1. (Nokia - CA)" <[email-masked]>
Date: Tue, 6 Sep 2016 18:28:34 +0000

We have no IPR related to this draft.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Tue, 6 Sep 2016 21:47:14 +0000

I have no IPR, know of no IPR and have no intention of claiming any IPR.

Thanks,
Jakob.

> On Sep 6, 2016, at 12:56 PM, Susan Hares <[email-masked]> wrote:
> 
> Job:
>

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 8 Sep 2016 18:04:28 -0400

Sue and John,

> we'll halt this adoption poll to await the authors response. 

It's worth pointing out the functionality covered by
draft-heitz-idr-large-community is already covered in 
draft-ietf-idr-wide-bgp-communities.  

wide -02 previously handled the encoding in TLV format.  Per discussion at
the recent IETF, -03 moved to a flattened format.  

The wide communities draft is already adopted IDR work.

Please also see my note to idr in subject "community of the day - common
header".

Please take the above in consideration for the adoption call.  

-- Jeff

---

From: Jared Mauch <[email-masked]>
Date: Thu, 8 Sep 2016 18:20:41 -0400

> The wide communities draft is already adopted IDR work.

<operator hat=on>

Having waited since the original work started in 2010, I'm concerned
the oven is broken.

> Please also see my note to idr in subject "community of the day - common

We need something.  I'll respond to that thread, as there is something
important there I agree with.

> Please take the above in consideration for the adoption call.  

I would also say take 2010->2016 on the existing wide into account as
well.

	- Jared

-- 
Jared Mauch  | pgp key available via finger from [email-masked]
clue++;      | http://puck.nether.net/~jared/  My statements are only mine.

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Thu, 8 Sep 2016 22:26:39 +0000

Hi Jeff, 

I thought the result of this discussion at IETF 96 was to separate out the
wide community header into a separate draft? This would provide
compatibility for coexistence. Correct me if I am wrong.

Also, I believe if you go through the complete E-mail thread, you’ll see
that the positioning of the two drafts has already been discussed.

Now that all the authors have responded to the IPR poll, I’d advise the
chairs to the consider the “bring a horse to water” rule when considering
whether or not to allow the WG adoption poll to proceed.

Thanks,
Acee 

On 9/8/16, 6:04 PM, "Idr on behalf of Jeffrey Haas" <[email-masked]

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 8 Sep 2016 22:54:36 +0000

Jeff, you proposed to split a common header out of the wide communities draft as a separate draft and I agreed. Then Robert didn't want to do that and published -03 of the wide communities. So be it.

Thanks,
Jakob.

> On Sep 8, 2016, at 6:27 PM, Acee Lindem (acee) <[email-masked]> wrote:
> 
> Hi Jeff, 
>

---

From: Jared Mauch <[email-masked]>
Date: Thu, 8 Sep 2016 19:22:47 -0400

> Jeff, you proposed to split a common header out of the wide communities draft as a separate draft and I agreed. Then Robert didn't want to do that and published -03 of the wide communities. So be i…

	The nice thing about the IETF is we can do both.  This problem
you highlight isn't something new, it was commented on here as well:

http://www.gossamer-threads.com/lists/nanog/users/168381#168381

	With many of the same people on the thread.

	- Jared

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 9 Sep 2016 11:03:14 +0200

Jakob,

>  Then Robert didn't want to do that and published -03 of the wide
communities.

It is not that as you know. The discussions went to halt state where you
did not allow any flexibility in the common header to accommodate other
then large community needs. That is not what "common header" intention was.

But what is really important it is no longer Robert's or Jeff's or any
other editor's choice to split or not to split.

Wide Communities being a WG document requires WG rough consensus to split
headers out it. And if WG decides this is what IDR WG wants non of the wide
authors nor editors will object (nor have even right to object).

Last ... to the point of broken oven ... well for significant time authors
wanted to make wide comms even more universal. So few years were gone with
that and during that we did not push any implementation to start working on
it. Then as also mentioned none of us really went went to all NOGs to
explain what it is and what it does. Also IDR WG has also its own set of
grouped priorities - its hard to work on everything all at once.

Good news is that I was told that three maybe not that big implementations
have wide on the roadmap. The plan was to use that at least as a demo to
the operators.

Thx,
R.

On Fri, Sep 9, 2016 at 12:54 AM, Jakob Heitz (jheitz) <[email-masked]>
wrote:

> Jeff, you proposed to split a common header out of the wide communities
> draft as a separate draft and I agreed. Then Robert didn't want to do that
> and published -03 of the wide communities. So be it.
>
> Thanks,
> Jakob.
>
>

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 9 Sep 2016 11:25:27 -0400

> Jeff, you proposed to split a common header out of the wide communities draft as a separate draft and I agreed. Then Robert didn't want to do that and published -03 of the wide communities. So be i…

The discussion circled around the *logistics* of the split.  Robert would
prefer not to, I'm supportive of it.  John requested we take this discussion
to the mail list, hence I'm doing so.

Tersely, with a common header, I'm supportive of a separate 4:4 draft as
long as it's built on a common header as per the other discussion.  That
header can be in a separate draft; Robert has updated wide -03 in a way that
shows how the split may be done.

In the absence of a common header, I would not be supportive of the current
proposal.

-- Jeff

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 9 Sep 2016 15:30:05 +0000

I had endless discussions with Robert on the format of the common header and I don't have the appetite anymore. Besides, it's not needed. Attribute exhaustion is a strawman.

Thanks,
Jakob.

> On Sep 9, 2016, at 11:24 AM, Jeffrey Haas <[email-masked]> wrote:
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 9 Sep 2016 15:35:35 +0000

Here is my common header

2.1.  BGP Community Container Header

   Containers always start with the following header:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Type     |  Flags    | T |          Length               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                       Context ASN                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   The fields are as shown below:

        Type -  The type of community following the header.  The types
             are to be defined in companion documents.

        Flags -  Sent as 0.  Ignored upon receipt.  MUST be propagated
             to other BGP speakers unchanged.  These flags may be
             assigned meanings by future specifications.  Thus it is
             important that speakers that do not recognize the flags
             propagate them unchanged.

        T -  Transitivity field

             Value 0:  The communities in the container are transitive
                  across all ASes.

             Value 1:  The communities in the container are transitive
                  across AS boundaries, but not across an administration
                  boundary.  An administration in this sense is an
                  arbitrary set of connected ASes, possibly owned by a
                  single administration.  How such an administration
                  boundary is determined is out of scope of this
                  document.  However, by default, every autonomous
                  system is its own administrative region.

             Value 2:  The communities in the container are transitive,
                  but not across an AS boundary.

             Value 3:  The communities in the container are not
                  transitive.

        Length -  The number of octets occupied by the communities after
             the header.

        Context ASN -  Context Autonomous System Number.

             The meaning of this is defined in the specifications of
             each container type

   The Transitivity field governs how a received community is propagated
   in a BGP UPDATE message to the next speaker.  It does not apply to
   the originator of a community.  That means that a BGP speaker may
   send a community over any boundary if it is configured to do so.  The
   transitivity applies when a speaker is simply passing the community
   on.

   A BGP speaker that does not recognize a container type SHOULD ignore
   it, but propagate it to other speakers as defined by the Transitivity
   field.  This behavior MAY be overridden by local policy.

Thanks,
Jakob.

> On Sep 9, 2016, at 11:30 AM, Jakob Heitz (jheitz) <[email-masked]> wrote:
>

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 9 Sep 2016 11:50:47 -0400

> Here is my common header

See? That wasn't so hard.

Summary of diffs from wide -03 header proposal:
- No confed bit for transitivity. 
- IANA registration bit.

The confed behavior description in wide has been there for completeness.  I
don't believe the wide comm authors are strongly opinionated about it
remaining.  That said, behavior of these new community types in a
confederation needs to be specified.  (Likely, they're transitive.)

The IANA registration bit is something that requires a bit more discussion
as we began having in private thread.  The motivation of that flag in the
wide communities spec was specificall to permit *customers* to create their
own internal types without having to get a code point registered with IANA
and a format specified at IETF level.

My feedback on that point is there are two options that I believe cover that
desired behavior:
- We decide that we want to only have registered types in common headers,
  but potentially reserve a type (e.g. 255) for opaque attributes.  In that
  case, recommending that the non-transitive bits are required (or presumed)
  covers the case.
- We leave such internal format to specific formats, e.g. wide comms.

I'm fine with the second option since semantic flexibility is more something
covered by the wide case anyway and future formats can similarly make it
available as part of their specification.

Presuming the other wide comm authors agree that we don't care about the
community case and agree with the above assessment on iana registration,
we've converged on a common header.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 9 Sep 2016 18:13:07 +0200

Jeff,

As far as common header details there were three points which privately
Jakob and I were not able to solve. Hence as recommended by chairs we moved
the discussion to IDR list.

1. IANA registered or private indication.

As you pointed out that can be a flag or it can be embedded within the
type. I was fine with any of those - Jakob said no to both. My personal
preference is a flag. Simple and non overlapping with anything - very easy
to filter as well without going one level deep.

And even if implementations always go to policy processing to filter
communities tools like BMP parsers or other monitoring tools may not need
to. It costs nothing to have such explicit flag.

2. Size of type field

Original wide had two octets, Jakob locked his position on single octet. I
offered compromize with 12 bits. The compromize was rejected. Maybe 255 is
enough - maybe not ... but today we already see need for 4 or 5 (ext. comm
successor). It really does not cost that much to leave extra room IMHO.
Especially that tomorrow someone may come with brilliant idea to steal most
significant bit or two of type field for some other use.

3. Size of flags

That one was not that much of contradiction - I guess one octet or 12 bits
for extra space would be fine too.

Rest of the elements in common header are context AS - here we agreed
easily that what it means is a local matter and will be described in a
given type. Length of two octets (in octets) also seems fine to everyone.

As you all see we were and we still are so close to reach consensus if only
one implementer would indicate willingness and flexibility to accept the
compromise.

Maybe voices from the list will convince him ;-).

Cheers,
R.

On Fri, Sep 9, 2016 at 5:50 PM, Jeffrey Haas <[email-masked]> wrote:

> > > I had endless discussions with Robert on the format of the common
> header and I don't have the appetite anymore. Besides, it's not needed.
> Attribute exhaustion is a strawman.
> > >
> > > Thanks,
> > > Jakob.
> > >
> > >
> > >>> Jeff, you proposed to split a common header out of the wide
> communities draft as a separate draft and I agreed. Then Robert didn't want
> to do that and published -03 of the wide communities. So be it.
> > >>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 9 Sep 2016 16:15:02 +0000

More on transitivity.

Transitivity is just a request by the sender to transit unknown types. A receiver is fully within its rights to refuse transit. That's why we have attribute filtering. We will need to extend attribute filtering into community types.

An administrative region is defined by the operator. By default, every AS is its own administrative region. To create an administrative region of multiple ASes will require extra configuration on the routers involved. In this way, someone who has no interest in administrative regions need do nothing.

Transitivity across confederation member ASes was considered, but it would require another bit and no real need for it was seen.

Thanks,
Jakob.

> On Sep 9, 2016, at 11:49 AM, Jeffrey Haas <[email-masked]> wrote:
>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 9 Sep 2016 18:19:59 +0200

>  We will need to extend attribute filtering into community types.
>

​The entire idea behind the T flag in the common header is not to touch
attribute filtering based on community types. ​If a bug comes it will
affect just the communities .. not all other 40 attributes.

> An administrative region is defined by the operator. By default, every AS
> is its own administrative region. To create an administrative region of
> multiple ASes will require extra configuration on the routers involved. In
> this way, someone who has no interest in administrative regions need do
> nothing.
>

​I am happy to add a separate flag for that in the common header as
apparently there is need for it across all types.

Transitivity across confederation member ASes was considered, but it would
> require another bit and no real need for it was seen.
>

On that we all agreed.

Thx,
R.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 9 Sep 2016 16:21:12 +0000

Registered type is a wide community thing only. Keep it there.
Large community is a single type and its registered.
Don't encumber large community implementations with this bit.

8 bits is enough size. It is unlikely to grow to more than 10 types. In the unlikely event that it grows to 256, we can define an extension type. It is vanishingly unlikely.

The common header needs no more flags. If you are desperate for flags, put them into your own header.

Thanks,
Jakob.

On Sep 9, 2016, at 12:13 PM, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

Jeff,

As far as common header details there were three points which privately Jakob and I were not able to solve. Hence as recommended by chairs we moved the discussion to IDR list.

As you pointed out that can be a flag or it can be embedded within the type. I was fine with any of those - Jakob said no to both. My personal preference is a flag. Simple and non overlapping with anything - very easy to filter as well without going one level deep.

And even if implementations always go to policy processing to filter communities tools like BMP parsers or other monitoring tools may not need to. It costs nothing to have such explicit flag.

2. Size of type field

Original wide had two octets, Jakob locked his position on single octet. I offered compromize with 12 bits. The compromize was rejected. Maybe 255 is enough - maybe not ... but today we already see need for 4 or 5 (ext. comm successor). It really does not cost that much to leave extra room IMHO. Especially that tomorrow someone may come with brilliant idea to steal most significant bit or two of type field for some other use.

3. Size of flags

That one was not that much of contradiction - I guess one octet or 12 bits for extra space would be fine too.

Rest of the elements in common header are context AS - here we agreed easily that what it means is a local matter and will be described in a given type. Length of two octets (in octets) also seems fine to everyone.

As you all see we were and we still are so close to reach consensus if only one implementer would indicate willingness and flexibility to accept the compromise.

Cheers,
R.

On Fri, Sep 9, 2016 at 5:50 PM, Jeffrey Haas <[email-masked]<mailto:[email-masked]>> wrote:

-- Jeff

> > On Sep 9, 2016, at 11:30 AM, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:
> >
> >>> On Sep 9, 2016, at 11:24 AM, Jeffrey Haas <[email-masked]<mailto:[email-masked]>> wrote:
> >>>

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 9 Sep 2016 16:22:09 +0000

Not a T flag. A T field of two bits.

Thanks,
Jakob.

On Sep 9, 2016, at 12:20 PM, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

?The entire idea behind the T flag in the common header is not to touch attribute filtering based on community types. ?If a bug comes it will affect just the communities .. not all other 40 attributes.

?I am happy to add a separate flag for that in the common header as apparently there is need for it across all types.

On that we all agreed.

Thx,
R.

---

From: Keyur Patel <[email-masked]>
Date: Fri, 9 Sep 2016 16:23:43 +0000

Robert,

*if* there is a bug then you probably want to drop an entire attribute instead of set of communities.

Regards,
Keyur

Sent from my iPhone

On Sep 9, 2016, at 9:20 AM, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

On that we all agreed.

Thx,
R.

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 9 Sep 2016 12:40:15 -0400

> *if* there is a bug then you probably want to drop an entire attribute instead of set of communities.

Similar to other 7606 handling behaviors, if we can establish framing of a
given type, there's no reason to drop all.  We could just drop the impacted
item.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 9 Sep 2016 18:40:36 +0200

Oh I think where the "T" disconnect is now.

​It is not .. that is entire point.

T flag means that implementation knows what to do with the community
without extra policy needed. As example - geo-stuff will be of registered
type. ​Portion of wide type one will also be registered ... the other
local.

> Large community is a single type and its registered.

​Large is a free form agreement between parties so is it not regsistered in
the sence of that flag, It is of local meaning to the agreeing parties only
and requires policy for processing it.

> Don't encumber large community implementations with this bit.

​That is the thing of contention. Common header means that the header is to
be used not only by large community. ​

8 bits is enough size. It is unlikely to grow to more than 10 types. In the
> unlikely event that it grows to 256, we can define an extension type. It is
> vanishingly unlikely.
>

​Now we all know your perspective on this.

Thx,
Robert.​

>
> Thanks,
> Jakob.
>
>
> On Sep 9, 2016, at 12:13 PM, Robert Raszuk <[email-masked]> wrote:
>
> Jeff,
>
> As you all see we were and we still are so close to reach consensus if
> only one implementer would indicate willingness and flexibility to accept
> the compromise.
>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 9 Sep 2016 18:45:17 +0200

Keyur,

I was actually not so much worried about dropping entire attribute ... I
was more worried about attribute + type new filtering code with associated
bug(s) which could result in dropping many innocent attributes ...

Best,
R.

On Fri, Sep 9, 2016 at 6:23 PM, Keyur Patel <[email-masked]> wrote:

> On Sep 9, 2016, at 9:20 AM, Robert Raszuk <[email-masked]> wrote:
>
>
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 9 Sep 2016 16:48:59 +0000

Robert,

It looks like you don't agree to the common header, so I'll go without. My current code does not have it anyway.

Thanks,
Jakob.

On Sep 9, 2016, at 12:40 PM, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

T flag means that implementation knows what to do with the community without extra policy needed. As example - geo-stuff will be of registered type. ​Portion of wide type one will also be registered ... the other local.

​Large is a free form agreement between parties so is it not regsistered in the sence of that flag, It is of local meaning to the agreeing parties only and requires policy for processing it.

​That is the thing of contention. Common header means that the header is to be used not only by large community. ​

Thx,
Robert.​

Thanks,
Jakob.

Jeff,

2. Size of type field

3. Size of flags

Cheers,
R.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 9 Sep 2016 12:51:40 -0400

Robert,

On Fri, Sep 09, 2016 at 06:13:07PM +0200, Robert Raszuk wrote:
> type. I was fine with any of those - Jakob said no to both.

In the case of 4:4, that's the choice of the discussion for that format.  I
don't think there's a lot of call for it in that mechanism.

For wide, we have a use case.  I don't think Jakob gets any more say there
than the rest of WG for consensus.

> significant bit or two of type field for some other use.

I'm a fan of 2 octets for any new type field in any proposal myself.  I'm
not going to stress over it too much.  It appears the motivation for sizing
is more for alignment of the following context AS field on a word boundary
for implementations that care about such things.

> for extra space would be fine too.

Bits tend to be the thing that get burned fastest, but since this needs to
be common container-wide, a constrained space may have to make sense.

Probably the best analogy for this headache is covered by
draft-ietf-idr-bgp-attribute-announcement.  

> given type. Length of two octets (in octets) also seems fine to everyone.

I'd like to see context AS get some level of standardization as "who these
communities are addressed to, and their interpretation of how they are
used".  I believe that's the semantic we have today.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 9 Sep 2016 18:52:45 +0200

Jakob,

I think this discussion is not relevant to what your code does have or does
not have or to what I agree or do not agree with.

It is about IDR WG consensus. You are of course free to go without common
header in XR and ship it.

Btw what are the plans for XE and NX trains ?

Thx,
R.

On Fri, Sep 9, 2016 at 6:48 PM, Jakob Heitz (jheitz) <[email-masked]>
wrote:

> On Sep 9, 2016, at 12:40 PM, Robert Raszuk <[email-masked]> wrote:
>
> ​Large is a free form agreement between parties so is it not regsistered
> in the sence of that flag, It is of local meaning to the agreeing parties
> only and requires policy for processing it.
>
>
>
> ​That is the thing of contention. Common header means that the header is
> to be used not only by large community. ​
>
>
> 8 bits is enough size. It is unlikely to grow to more than 10 types. In
>> the unlikely event that it grows to 256, we can define an extension type.
>> It is vanishingly unlikely.
>>
>> Original wide had two octets, Jakob locked his position on single octet.
>> I offered compromize with 12 bits. The compromize was rejected. Maybe 255
>> is enough - maybe not ... but today we already see need for 4 or 5 (ext.
>> comm successor). It really does not cost that much to leave extra room
>> IMHO. Especially that tomorrow someone may come with brilliant idea to
>> steal most significant bit or two of type field for some other use.
>>
>> 3. Size of flags
>>

---

From: Andrew Lange <[email-masked]>
Date: Fri, 9 Sep 2016 14:16:31 -0700

Hi Robert,

It’s funny the way old arguments come up again.   In an early pre-draft of flexcomms, there were all sorts of “clever” attribute special-types, inferred values and the like.  I thought I was being very smart.  I was wrong.

When I talked to BGP implementors about this, the comments from the senior Cisco developers stuck out the most:  Bits are cheap, always favor explicit over implicit as it’s easier to parse and debug, make the fields bigger than you think you’ll need, since you’ll probably need them at some point, and change later is hard.  

This caused a complete re-write of the draft to hew to these principals.  That was in 2002.  These are even MORE true today, links, CPU and memory are all faster.  

- Who cares if we have an “extra” octet?  The cost is zero.  The future cost of not having the octet, however, is non-zero.  Easy call.

- How much is another hour of downtime worth while an operator or support engineer debugs a field that is non-explict, since they have to go and re-read the spec to find the special case?

Andrew

> On Sep 9, 2016, at 9:13 AM, Robert Raszuk <[email-masked]> wrote:
> 
> Jeff,
> 

> On Fri, Sep 9, 2016 at 5:50 PM, Jeffrey Haas <[email-masked] <mailto:[email-masked]>> wrote:
> > > On Sep 9, 2016, at 11:30 AM, Jakob Heitz (jheitz) <[email-masked] <mailto:[email-masked]>> wrote:
> > >
> > >>> On Sep 9, 2016, at 11:24 AM, Jeffrey Haas <[email-masked] <mailto:[email-masked]>> wrote:
> > >>>
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/idr <https://www.ietf.org/mailman/listinfo/idr>
>

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 9 Sep 2016 23:48:31 +0200

Hey Andrew,

Funny indeed as I recall exactly those discussions in 2002 as at that time
I was part of IOS BGP team in one of Cisco SJC buildings. Folks there were
not cheap on bits and were forward looking to make sure what is implemented
in the spec has some level of hope to last.

Now when I hear completely new crew of cisco IOS BGP team telling on the
list that 8 bits and no more I have to say it very lightly and politely I
get sad. RP/RE memory footprint has increased 100 times since 2002 and
overhead on the gig wires is really not significant.

You said it nicely .. the cost is zero !

So I support both yourself and Jeff to have two octet type in common header
and have few more bits for flags. I was also thinking about adding
versioning to the common header but I have not even got there with current
private dialogue with Jakob.

Anyhow let's all think over this topic over the weekend and resume
discussion next week.

After all we are all here both vendors and customers to solve the network
problems. All who contribute and propose solutions are extremely welcome I
think both by everyone who is involved in writing standards, writing
shipping implementations as well as those hoping to use it in their daily
operational practices.

Cheers,
R.

On Fri, Sep 9, 2016 at 11:16 PM, Andrew Lange <[email-masked]>
wrote:

> Hi Robert,
>
> It’s funny the way old arguments come up again.   In an early pre-draft of
> flexcomms, there were all sorts of “clever” attribute special-types,
> inferred values and the like.  I thought I was being very smart.  I was
> wrong.
>
> When I talked to BGP implementors about this, the comments from the senior
> Cisco developers stuck out the most:  Bits are cheap, always favor explicit
> over implicit as it’s easier to parse and debug, make the fields bigger
> than you think you’ll need, since you’ll probably need them at some point,
> and change later is hard.
>
> This caused a complete re-write of the draft to hew to these principals.
> That was in 2002.  These are even MORE true today, links, CPU and memory
> are all faster.
>
> - Who cares if we have an “extra” octet?  The cost is zero.  The future
> cost of not having the octet, however, is non-zero.  Easy call.
>
> - How much is another hour of downtime worth while an operator or support
> engineer debugs a field that is non-explict, since they have to go and
> re-read the spec to find the special case?
>
> Andrew
>

---

From: Jared Mauch <[email-masked]>
Date: Fri, 9 Sep 2016 18:08:15 -0400

> On Sep 9, 2016, at 5:48 PM, Robert Raszuk <[email-masked]> wrote:
> 
> Hey Andrew,
> 
> Funny indeed as I recall exactly those discussions in 2002 as at that time I was part of IOS BGP team in one of Cisco SJC buildings. Folks there were not cheap on bits and were forward looking to make sure what is implemented in the spec has some level of hope to last.
> 
> Now when I hear completely new crew of cisco IOS BGP team telling on the list that 8 bits and no more I have to say it very lightly and politely I get sad. RP/RE memory footprint has increased 100 times since 2002 and overhead on the gig wires is really not significant. 
> 
> So I support both yourself and Jeff to have two octet type in common header and have few more bits for flags. I was also thinking about adding versioning to the common header but I have not even got there with current private dialogue with Jakob. 

Could you perhaps have some of that on-list?  I believe Jakob e-mailed his proposal for that header, I think constructive feedback would be welcome.

> Anyhow let's all think over this topic over the weekend and resume discussion next week. 

I suspect there will be no reason to hit the resume button monday, my mailbox will be full with more messages regardless :)

> After all we are all here both vendors and customers to solve the network problems. All who contribute and propose solutions are extremely welcome I think both by everyone who is involved in writing standards, writing shipping implementations as well as those hoping to use it in their daily operational practices.

There are those of us who care less about the wire format and more about the usability/existence in running code.  I’ve got some goals here that I think are reasonable and attainable, expect some nudges (or pushes) if you’ve chimed in.

- Jared

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 10 Sep 2016 00:15:47 +0200

Jared,

> > So I support both yourself and Jeff to have two octet type in common
> header and have few more bits for flags. I was also thinking about adding
> Could you perhaps have some of that on-list?  I believe Jakob e-mailed his
> proposal for that header, I think constructive feedback would be welcome.
>

​Oh sorry. Jeff and I pointed already twice to it in the public WG
document. But I am more then happy to copy and paste it to the list as you
wish :) The C flag is agreed  to be removed so do not consider that one as
a problem.

Type and flags fields below are shorter then they should be as were really
trying to get consensus with large folks.

3.1 <https://tools.ietf.org/html/draft-ietf-idr-wide-bgp-communities-03#section-3.1>.
Wide BGP Community Attribute Common Container Header

   Containers always start with the following common header:

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |      Type     |  Flags  |R|C|T|            Length             |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                       Context AS Number                       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   This document defines Types 1,2,3 and 4.  See the Section 14
<https://tools.ietf.org/html/draft-ietf-idr-wide-bgp-communities-03#section-14>
for
   information on additional type registration policies.

   +------+-------+----------------------------------------------------+
   | Bit  | Value | Meaning                                            |
   +------+-------+----------------------------------------------------+
   |  0   |   0   | Transitive across AS boundary                      |
   |      |   1   | Not Transitive across AS boundary                  |
   |  1   |   0   | Transitive across confederation boundaries         |
   |      |   1   | Not transitive across confederation boundaries     |
   |  2   |   0   | Local community value.                             |
   |      |   1   | IANA Registered community type or value.           |
   | 3..7 |   -   | RESERVED - MUST be zero when sent and ignored upon |
   |      |       | receipt.                                           |
   +------+-------+----------------------------------------------------+

   Flags are defined globally, to apply to all wide community container
                                  types.

                              Table 1: Flags

   Bit 0 (aka T bit) Transitivity bit: Value 0: The communities in the
   container are transitive across all ASes.  Value 1: The communities
   in the container are transitive across AS boundaries, but not across
   an administration boundary.  An administration in this sense is an
   arbitrary set of connected ASes, possibly owned by a single
   administration.  How such an administration boundary is determined is
   out of scope of this document.
Raszuk, et al.            Expires March 6, 2017                 [Page 6]

------------------------------

  <https://tools.ietf.org/html/draft-ietf-idr-wide-bgp-communities-03#page-7>Internet-Draft
           wide-bgp-communities            September 2016

   Bit 1 (aka C bit) Confederation bit: Is used to manage the
   propagation scope of a given Wide BGP Community across confederation
   boundaries.  When not set (value of 0) indicates that communities in
   a given container are transitive across confederation boundary.  When
   set (value 1) communities are not transitive across confederation
   sub-AS boundary.

   Bit 2 (aka R bit) Registered bit: When set (value 1) indicates that
   the given container carries a Wide BGP Community which is registered
   with IANA.  When not set (value 0) it indicates that community value
   which follows is locally assigned with a local only meaning and local
   behaviour.

   The Length field represents the total length of a given container in
   octets.

   Context Autonomous System Number: 4 octets

   This identifies the AS that is to interpret the community container.
   For example, an ISP may publish a set of community specifications.  A
   customer of the ISP will use those specifications to formulate the
   communities.  The ISP will read the communities and perform the
   instructions encoded as per the specifications.  The Context ASN is
   the ASN of the ISP.  The ISP is usually directly connected to the
   customer, but if not, then the intervening network has the
   opportunity to pass the routes to the Context ASN if it chooses to do
   so.

There are those of us who care less about the wire format and more about
> the usability/existence in running code.  I’ve got some goals here that I
> think are reasonable and attainable, expect some nudges (or pushes) if
> you’ve chimed in.
>

I am all to help here. You know IETF for me is more of the hobby rather
then day job :)

Cheers,
R.
​

---

From: Sander Steffann <[email-masked]>
Date: Sat, 10 Sep 2016 00:40:01 +0200

Hi,

I may be missing something here, but how is this going to work with routers that don't know about wide communities? Aren't communities marked as not transitive across boundaries going to leak across anyway and cause unexpected problems there?

>    Flags are defined globally, to apply to all wide community container

This common header feels overengineered to me. How about just type and length, no magic?

Cheers,
Sander

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 10 Sep 2016 00:48:35 +0200

Hi Sander,

There is nothing there specific to wide communities.

Transitive across AS is applicable to all TLV encoded communities.

We may add a new flag to mark transitiveness across same operator ASes.

We agreed to dropped C flag as per my comment so forget it.

Local vs registered I already explained .. this is not only about wide
comms. This is also about geo encoding. This is about meaning in the
community (even large) which is understood without explicit policy code to
handle it.

The comment below needs to edited if it goes to standalone doc - you are
right. I just copied for now from main spec.

Type, length size is recommended to be of two octet each.

Context-AS is agreed already among everyone.

Cheers,
R.

On Sat, Sep 10, 2016 at 12:40 AM, Sander Steffann <[email-masked]>
wrote:

> I may be missing something here, but how is this going to work with
> routers that don't know about wide communities? Aren't communities marked
> as not transitive across boundaries going to leak across anyway and cause
> unexpected problems there?
>

---

From: Sander Steffann <[email-masked]>
Date: Sat, 10 Sep 2016 01:48:35 +0200

Hi Robert,

> Transitive across AS is applicable to all TLV encoded communities. 

Yes, the path attribute is transitive. Adding flags to restrict that in some way will not stop older implementations from forwarding it anyway, which is why it seems not very useful to me to include them in the first place...

My personal opinion is that the header really should contain as little as possible. Extra fields (like the ASN) can be added by those subtypes that need it. Later there will be subtypes that don't need it, they'll have to specify a magic value for the ASN (probably 0) and implementations will have to make exceptions, increasing the likelihood of bugs etc etc etc. I could see a little use for filtering communities from a given ASN, so I won't object to it, but please seriously consider if it's worth it.

Look at it from the other side: a community is a simple label. A label is just a sequence of octets. There may be multiple ways to construct those octets (although every operator I know would be happy with just the 4:4:4 large version, but ok, maybe a bit more flexibility) so a subtype identifier and a length field are necessary, but think hard before you make simple labels more complicated.

Also be careful when judging what people want for this draft. Many people are about to agree to anything just to get this over with. Most operators prefer a simple solution that works over a fancy one any day :)

An engineer is done not when there is nothing more to add, but when there's nothing more to take away :)  I think this I'd forgotten too often these days.

Cheers!
Sander

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Sat, 10 Sep 2016 00:25:12 +0000

Let's do one quick and do one right.
To do it right certainly needs discussion.

Thanks,
Jakob.

---

From: <[email-masked]>
Date: Mon, 12 Sep 2016 13:38:11 +0000

TL;DR: I support the use of a common header, to make it easier for the future.

1) +1 for common header
Looking at the past (regular community, extended community with multiple types) and the present (large, wide, geo-tags) I think we should expect new community types after the large-comm (e.g. XL, wide) hence we should prepare for this.

2) header format
IMHO, the most needed capabilities are extensibility and filtering of communities (and in particular the filtering of future community types which may be unknown for some nodes/implementations)

2.1) Regarding filtering, I would propose the following fields in the header, in descending order of (my) priorities: Transitive flags/field, Context ASN, Community value (32bits)

"Community value" is new on this thread and may be debatable, but I would expect that any community type would need a value field, e.g. to define the meaning of the community. Putting this in the header, would help the filtering granularity of unknown/unsupported community types. If "community value" name is too specific and not agreed upon, any name would work for me (e.g "Local Data Part 1" as per wide-com terminology). Again, this point is least priority and I can live without.

On a side note, I'm fine with no Confed flags/filtering.

2.2) Regarding extensibility
We need a TLV to be able to define new types of variable length. Extended community was right to introduce the "Type" part, and it has proved useful. But it missed the variable length.
Regarding the size of "T", I agree with Jakob that 8 bits seems a priori enough and if not, it will be possible to extend it e.g. using a specific type. However, we have a track record of defining too small fields which latter requires protocol and implementation extension to handle their increase. (e.g. BGP community...); such extension would add complexity for the future (implementation); I don't see adding 1 octet as a significant cost.
So to be on the safe side, and to not create technical debt for the future, I support a 16 bits Type field. (but could live with 8 bits if there is a significant reason)

2.3) IANA registration bit.
seems to non-consensual, but I think we could imagine a way to accommodate this. 

My 2cs:
- IMHO, large community could make use of it, to define "well-known" behavior. e.g. draft-ietf-grow-blackholing is a recent work showing that it's sometime easier to define a well-known value, rather than a specific value for each ISP. But this is really up to large-com authors.
- wide community could split (hard code) the range of types between IANA allocated and private use. With a 50/50 split, that would mean 1 dedicated flag.
- the common header could define this bit and the large-community could choose to not use it (i.e. ignore)
- the common header could define this bit and make it optional. If optional behavior are now not acceptable for the IETF/WG, then the common header can be defined without, and an additional 5 lines draft/RFC could define this extension. (progressing both documents jointly may reduce the overhead)
- wide-com could define it's own flags field.

On my side, I'm pretty open to any option. I can't believe this would be the reason to reject the common header.

Thanks,
Regards,
-- Bruno

> From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas > Sent: Friday, September 09, 2016 5:51 PM
> 
> > > I had endless discussions with Robert on the format of the common header and I don't
> have the appetite anymore. Besides, it's not needed. Attribute exhaustion is a strawman.
> > >
> > > Thanks,
> > > Jakob.
> > >
> > >
> > >>> Jeff, you proposed to split a common header out of the wide communities draft as a
> separate draft and I agreed. Then Robert didn't want to do that and published -03 of the
> wide communities. So be it.
> > >>

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Jared Mauch <[email-masked]>
Date: Wed, 14 Sep 2016 12:15:26 -0400

> I may be missing something here, but how is this going to work with routers that don't know about wide communities? Aren't communities marked as not transitive across boundaries going to leak acros…

	None of these transitive things are necessary.  Operators already set the
transitive nature of the policies by enabling sending communities on a per-session
or group basis already.  If they are going beyond boundaries, it's because
the operator explicitly is permitting them.

	There can be no leakage as this is not a default action on any
BGP implementation that I'm aware of.  (comparing XR, XE, JunOS, Zebra, etc).

> This common header feels overengineered to me. How about just type and length, no magic?

	It is.

	The transitive stuff should be removed as it provides no benefit for an
operator who already controls these policies.

	- Jared

---

From: Jared Mauch <[email-masked]>
Date: Wed, 14 Sep 2016 12:17:54 -0400

> community (even large) which is understood without explicit policy code to

	While some people may see the need to make well defined
geo communities (similar to no-export), the reality is operators are
going to make their own opaque communities and not use IETF well-knowns.

	We for example do not honor certain well knowns such as
no-export or local-as.

> Context-AS is agreed already among everyone.

	- jared

---

From: Jared Mauch <[email-masked]>
Date: Wed, 14 Sep 2016 12:25:37 -0400

> Let's do one quick and do one right.

	I think we can get the proper units together to
help with making the drafts be implementable.  The big feedback
I heard from my vendors is how -wide- is not implementable in
the historical form, hence the tens of hours of discussion
in berlin about how to make something which is implementable
and viable.

> To do it right certainly needs discussion.

	There's a difference between discussion for the sake
of discussion and reaching a point of viability.  I think
we are close to having something viable that will meet
the needs.

	I want to avoid having lengthy discussions about
the many sub-types people may dream up or see as valuable
and keep focused on the problem at hand.  if we try to
over-engineer a solution, we end up with a aircraft with
7 engines when 2 would meet the needs.

	having a common header and allowing tlv sub-types
for extensibility provides an excellent balance on permitting
people to treat the communities as they are in rfc1997
based world, which is an opaque set of bits that may or may
not have meaning to the receiver.

	for me personally, this isn't about enumerating the
new types that we dream someone may want, if there is an
operator with that need, let them bring it to the IETF
directly or via their vendor(s) with their support.

	- Jared

---

From: heasley <[email-masked]>
Date: Wed, 14 Sep 2016 16:27:02 +0000

> the operator explicitly is permitting them.

While it would appear that someone is trying to be overly clever with the
flags, I disagree.  It would be useful to mark a currently defined community
as non-transitive and, since the whole premise of the draft is to allow for
other applications and formats of a "community", transitivity is portentially
a universally useful attribute.  eg: I may want to send route origin
communities to customers, but not want them to forward that.

---

From: Jared Mauch <[email-masked]>
Date: Wed, 14 Sep 2016 12:29:19 -0400

> communities to customers, but not want them to forward that.

	Attempting to impose policy on a 3rd party in this way
is an interesting suspension of reality.  Once you send it, you have
no control over what they use it for, and pretending otherwise is
wishing for something that will not happen.

	If you don't want someone to use information, do not forward to them.

	The same rule goes for selfies, e-mail and paperwork around
your home.

	- Jared

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 14 Sep 2016 12:29:31 -0400

> On Sep 14, 2016, at 12:27 PM, heasley <[email-masked]> wrote:
> 

You nailed it.

An awful lot of policy clutter exists to simply strip things that should be stripped for you.  

That said, as Jared posted, some people (configurations) may choose to ignore such automatic mechanisms.

-- Jeff

---

From: heasley <[email-masked]>
Date: Wed, 14 Sep 2016 16:32:47 +0000

> wishing for something that will not happen.

Yes, you can use RPL to override such things, but you're indicating intention
AND you can also have free RPL as a result - ie: the device will automatically
remove the community without you specifying RPL to do so.

---

From: Jared Mauch <[email-masked]>
Date: Wed, 14 Sep 2016 12:38:14 -0400

> remove the community without you specifying RPL to do so.

	There is nothing precluding someone from asking their vendors for
such a knob, and this is infact the default way some things work, eg:
IOS clears all communities, including well-known when you set
a community unless you use the additive keyword.

	The difference I see here is: should this be a part of the protocol?

	my thought process leads me to No, it opens up even more
unecessary complexity in the protocol design.

	- Jared

---

From: heasley <[email-masked]>
Date: Wed, 14 Sep 2016 16:48:14 +0000

> unecessary complexity in the protocol design.

I do not see it that way.  Its checking a bit flag; TLVs are more complicated
that checking a bit.  All the VPN crap thats stuffed into BGP seems like a
far larger burden of complexity.  Vendors also have RPL to apply a regex to
remove communities.

But, yes, Vendors please allow me to configure RPL to ignore these transitivity
flags - I like options.

---

From: Sander Steffann <[email-masked]>
Date: Wed, 14 Sep 2016 19:07:19 +0200

Hi,

> AND you can also have free RPL as a result

The big problem here is that these "intention indicating" flags are global for all the communities in the container, and as BGP only allows one path attribute of each type all the wide communities have to share the same fate. The sender cannot indicate which wide communities should be transitive and which should be not. This "all or nothing" approach isn't useful and will only create confusion. Better to just drop the flags field.

Cheers,
Sander

---

From: Jared Mauch <[email-masked]>
Date: Wed, 14 Sep 2016 13:20:58 -0400

> The big problem here is that these "intention indicating" flags are global for all the communities in the container, and as BGP only allows one path attribute of each type all the wide communities…

	Thanks for putting it that way, I think that's helpful.

	- jared

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 14 Sep 2016 19:53:50 +0200

Folks,

But this is not what the T flags (Transitive or Filter) are to be all
about. You can have one attribute in BGP UPDATE message ... but since we
are converging on common header there is nothing that prevents you to have
more then one common header of the same type.

One can differ with T flags, the other can differ with Context-AS.

Otherwise of course it would be pointless, but the above is the intention.

And while you can't trust what your peer does with your flag - you can
filter yourself by that flag and never send him something which one ingress
added to the message and is not intended to be send outside.

You enable send-all, you may enable send-type. But I hope no one here is
eager to match by community content on 100s of ebgp egress points to filter
it.

That is why T or D (same domain) or R flags are intended to be used for.

Cheers,
R.

On Wed, Sep 14, 2016 at 7:20 PM, Jared Mauch <[email-masked]> wrote:

> > The big problem here is that these "intention indicating" flags are
> global for all the communities in the container, and as BGP only allows one
> path attribute of each type all the wide communities have to share the same
> fate. The sender cannot indicate which wide communities should be
> transitive and which should be not. This "all or nothing" approach isn't
> useful and will only create confusion. Better to just drop the flags field.
>

---

From: Jared Mauch <[email-masked]>
Date: Wed, 14 Sep 2016 14:27:16 -0400

(btw your text is still showing up tiny in my client, can you please switch to sending in plain-text to the IETF lists, that would be helpful).

> On Sep 14, 2016, at 1:53 PM, Robert Raszuk <[email-masked]> wrote:
> 
> Folks,
> 
> But this is not what the T flags (Transitive or Filter) are to be all about. You can have one attribute in BGP UPDATE message ... but since we are converging on common header there is nothing that prevents you to have more then one common header of the same type. 
> 
> And while you can't trust what your peer does with your flag - you can filter yourself by that flag and never send him something which one ingress added to the message and is not intended to be send outside. 
> 
> You enable send-all, you may enable send-type. But I hope no one here is eager to match by community content on 100s of ebgp egress points to filter it.
> 

I understand the intention, but the use cases don’t seem to justify this complexity in the header.  I would expect an implementation to provide knobs to filter out unknown types if necessary, but this certainly isn’t a requirement.

- jared

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 14 Sep 2016 15:10:26 -0400

> I understand the intention, but the use cases don’t seem to justify this complexity in the header.  I would expect an implementation to provide knobs to filter out unknown types if necessary, but t…

I must admit to being boggled by this conversation.  Let's pick on the large (4:4) case:

T = transitivity bits
C = context AS (4 octets)

Currently we have:
Header: type = 2, length, T, C
<list of 4:4 8 octets>

If you decide you're not doing C as part of header:
Header: type = 2, length, T
<list of C:4:4 12 octets>

If C is common across many items in the list, which is very likely given
existing community practices, you've expanded the amount of space needed to
send them.

Similarly, if T moves out of header and becomes part of the community
exactly like we have for extended communities:

Header: type = 2, length, T
<list of T:C:4:4 13 octets (min)>

Obviously extended communities made the choice to squat on top of its type
field.  

There's no "complexity" here.  Just space savings.  And part of the initial
hand-wringing earlier in this conversation was saving bits.

-----

The secondary consideration is once you decide C and T are good ideas, do
you really want to force every container type to individually have to
specify them?  That's almost as bad as having said that BGP Path Attributes
shouldn't have a common flags field and that the specification for its use
should have been repeated in every code point's documentation.

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 14 Sep 2016 21:35:59 +0200

/* Trying to send in plain txt */

I also do not see any complexity neither - just the opposite simplicity in
filtering - with the proposed encoding as well as in the policy language.

Flags basically are to group special known flooding scopes or use scopes
for any communities below them.

Are we now back to even discuss flags and context-as in common header ?

Thx,
R.

On Wed, Sep 14, 2016 at 9:10 PM, Jeffrey Haas <[email-masked]> wrote:

> > I understand the intention, but the use cases don’t seem to justify this
> complexity in the header.  I would expect an implementation to provide
> knobs to filter out unknown types if necessary, but this certainly isn’t a
> requirement.
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Wed, 14 Sep 2016 20:08:31 +0000

I agree with Jared.

An AS is not going to carry anything that they didn't previously agree upon.
You can filter by context-ASN.
If the context-ASN is yourself, then you act upon the community.
If the context-ASN is different and you agreed to carry it, then you will have a route-policy to pass it on.
Else you drop it.
On the egress, you can test the context-ASN for peer-ASN before sending.
Not hard.

Thanks,
Jakob.

> From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
> Sent: Wednesday, September 14, 2016 12:10 PM
> To: Jared Mauch <[email-masked]>
> Cc: John Heasley <[email-masked]>; [email-masked]; Susan Hares <[email-masked]>; Robert Raszuk <[email-masked]>
> Subject: Re: [Idr] Request to adopt draft-heitz-idr-large-community - Working Group Adoption call (9/6 to 9/20)
> 
> > > But this is not what the T flags (Transitive or Filter) are to be all about. You can have one attribute in BGP
> UPDATE message ... but since we are converging on common header there is nothing that prevents you to have more then
> one common header of the same type.
> > >
> > I understand the intention, but the use cases don’t seem to justify this complexity in the header.  I would expect
> an implementation to provide knobs to filter out unknown types if necessary, but this certainly isn’t a requirement.
>

---

From: Sander Steffann <[email-masked]>
Date: Wed, 14 Sep 2016 22:23:21 +0200

Hi Robert,

> But this is not what the T flags (Transitive or Filter) are to be all about. You can have one attribute in BGP UPDATE message ... but since we are converging on common header there is nothing that…

Yes there is: RFC 4271 (BGP-4). From section 5 of that RFC: "The same attribute (attribute with the same type) cannot appear more than once within the Path Attributes field of a particular UPDATE message.". Do you intend to change the basic BGP standard in this regard?

> Otherwise of course it would be pointless, but the above is the intention.

In that case let's abandon this idea with flags and Context-AS as BGP forbids multiple path attributes of the same type. Including anything besides sub-type TLVs in the wide attribute is therefore indeed pointless and even harmful.

I think our only remaining options are wide with no extra data in the common header, or just go for large which already has running code and the support of a large part of the intended user-base.

Cheers,
Sander

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Wed, 14 Sep 2016 20:30:39 +0000

I don't think it's an "or".
Large is designed to be quick to develop and deploy for a limited use case, albeit a widely and strongly demanded one.
Wide is the extensible powerful that can do the rest. We can take the time to do that one right.

Thanks,
Jakob.

> I think our only remaining options are wide with no extra data in the common header, or just go for large which
> already has running code and the support of a large part of the intended user-base.
> 
> Cheers,
> Sander

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 14 Sep 2016 22:33:51 +0200

Sander,

I think there is still some confusion what the plan is. Let me try to
explain as this is key for this discussion.

BGP UPDATE can have given attribute present in the message only once and no
one is attempting to change that.

What we have defined is a new attribute type for communities and within
that type a common header. Treat this header as a TLV wrapper for
communities.

As such within given attribute you can have many headers of the same type
as current RFC will define how to parse given attribute.

And that allow to keep one attribute for any type of communities of any
size. Be it large, wide, geo etc ... The common header can be present
multiple types in the community attribute we are defining.

That is the crux of the discussion. So all backwords compatible to existing
BGP with sufficient flexibility to be extensible to any community one
choose to invent.

Cheers,
R.

On Wed, Sep 14, 2016 at 10:23 PM, Sander Steffann <[email-masked]>
wrote:

> Yes there is: RFC 4271 (BGP-4). From section 5 of that RFC: "The same
> attribute (attribute with the same type) cannot appear more than once
> within the Path Attributes field of a particular UPDATE message.". Do you
> intend to change the basic BGP standard in this regard?
>
> In that case let's abandon this idea with flags and Context-AS as BGP
> forbids multiple path attributes of the same type. Including anything
> besides sub-type TLVs in the wide attribute is therefore indeed pointless
> and even harmful.
>
> I think our only remaining options are wide with no extra data in the
> common header, or just go for large which already has running code and the
> support of a large part of the intended user-base.
>
> Cheers,
> Sander
>
>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 14 Sep 2016 22:35:52 +0200

Jakob,

We are not discussing details of wide community type here.

We are discussing common format of the attribute for any current and future
community needs.

Best,
R.

On Wed, Sep 14, 2016 at 10:30 PM, Jakob Heitz (jheitz) <[email-masked]>
wrote:

> Large is designed to be quick to develop and deploy for a limited use
> case, albeit a widely and strongly demanded one.
> common header, or just go for large which

---

From: Martin Millnert <[email-masked]>
Date: Wed, 14 Sep 2016 22:38:05 +0200

> case, albeit a widely and strongly demanded one.

I don't see any clear objection to this part, in the thread, which did
spin off a bit from its Subject: line IMO.

> time to do that one right.

Sounds good.

Jakob: at XR, how much wall time would you bestguesstimate a first
customer release of Large would take, if you were able to independently
ship only this feature? (Like a bgpd plugin or something)

I'm asking because I expect it to be pretty harmless and easy on
vendors, leaving room/resources for more advanced implementations once
the have finished baking.

/M

> Thanks,
> Jakob.
> 
> > 
> > I think our only remaining options are wide with no extra data in
> > the common header, or just go for large which

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Wed, 14 Sep 2016 20:57:01 +0000

I have code now in my development branch that does all the useful things, tested.
That is the wire encoding, aggregation, route-polict set, match in various ways, including ios-regex and show commands.
By Seoul IETF, I expect to have other loose ends tied up, such that all code that applies to communities also applies to large-communities and test automation done.
After that, I expect there to be requests for additional functionality, maybe specialized matching and setting statements.
That is for https://tools.ietf.org/html/draft-heitz-idr-large-community-04.
As far as getting it into a release goes, it competes for space with lots of other features. I have no control over that. The level of demand controls it.

Thanks,
Jakob.

> From: Martin Millnert [mailto:[email-masked]]
> Sent: Wednesday, September 14, 2016 1:38 PM
> To: Jakob Heitz (jheitz) <[email-masked]>
> Cc: [email-masked]

---

From: Sander Steffann <[email-masked]>
Date: Wed, 14 Sep 2016 23:13:43 +0200

Hi,

> We are discussing common format of the attribute for any current and future community needs. 

Right, seems my brain was off by one level of indirection :)

In that case I think it's definitely overengineering and overcomplicating the whole thing. Including flags and an ASN field in every subtype makes configuring all future communities more complex than necessary for some/most of them. All these fields have to be configured in some way by the user, complicating the configuration interface. It feels like mandatory unwanted complications...

Cheers,
Sander

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 14 Sep 2016 22:18:48 +0100

> We are discussing common format of the attribute for any current and

No, really we're not.

It's a great idea to work towards consensus on a TLV based community
framework, and I support this fully for the community's future needs.

But let's not let this get in the way of getting -large- out the door
and into deployment for the community's current needs, because a TLV
based framework is a long and protracted discussion - one proposal for
which has already been on the table for the last 6 years without
reaching consensus, vendor adoption or attaining the necessary level of
traction to get it through IDR.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 14 Sep 2016 23:28:37 +0200

Nick,

Ok. Assume we move with 4:4:4 format as standalone attribute.

Can we get a rough consensus of the IDR WG community that someone tomorrow
or the day after will not come and request new attribute for 4:4:4:4 ? Or
16:4:4 ? Or define Large Ext Communities with yet one more attribute ?

If we reach that consensus here I am personally fine to give up on trying
to put large under common header and will not longer push it.

Cheers,
R.

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 14 Sep 2016 22:49:06 +0100

Robert Raszuk wrote:
> Can we get a rough consensus of the IDR WG community that someone
> tomorrow or the day after will not come and request new attribute for
> 4:4:4:4 ? Or 16:4:4 ? Or define Large Ext Communities with yet one more
> attribute ?
> 
> If we reach that consensus here I am personally fine to give up on
> trying to put large under common header and will not longer push it. 

There are no proposals on the table for this and I'm not aware of anyone
with plans to push any.

The people proposing large have already stated that their draft is
sufficient for their needs; it certainly scratches my communities itch
to satisfaction and the same for any other operators I've talked to.

Does anyone else on the WG mailing list have any plans in this area that
couldn't be accommodated in a TLV framework?

Nick

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 14 Sep 2016 18:35:21 -0400

> There are no proposals on the table for this and I'm not aware of anyone

Funny, I sat at a table for most of IETF-96 talking PDU formatting issues for
reconciling wide and large to a common header where exactly those formats
were discussed.  Some of the same crowd that stirred up the operators with
pitchforks and torches. :-)

> couldn't be accommodated in a TLV framework?

Any reason to not throw a common header on this one as the first and stop
kicking this football down the field another round?

-- Jeff

---

From: John Heasley <[email-masked]>
Date: Wed, 14 Sep 2016 22:42:47 +0000

> If the context-ASN is different and you agreed to carry it, then you will have a route-policy to pass it on.

why would i need policy to pass it on?  if i do not explicitly delete it and
it is transitive (or i use RPL to make it transitive) - pass it on.

the context-ASN may not be mine, but i may choose to take action based on it.

> On the egress, you can test the context-ASN for peer-ASN before sending.

This check should not be done, because you can not guess which ASes are 
beyond peer-ASN and they may intentionally be the target.

---

From: Jared Mauch <[email-masked]>
Date: Wed, 14 Sep 2016 18:54:53 -0400

I would highlight this thought. Just because the ASN is mapped into these 4 bytes, be careful with your assumptions. It may or may not have relevance based on someone else's opaque numbering system. This is less defined than it seems, even though some of us have a structure we plan to map into it. 

Jared Mauch

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 00:55:50 +0200

Thank you John H. !

Emails like yours are very important to the community.

Cheers,
R.

---

From: heasley <[email-masked]>
Date: Thu, 15 Sep 2016 00:01:34 +0000

> multiple types in the community attribute we are defining.

ie:

0                   1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  Attr. Flags  |Attr. Type Code|
|               | "MONGO COMMS" |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| total sum of  |
| MONGO COMMS   |
+-+-+-+-+-+-+-+-+

0                   1                   2                   3
|                       VALUE MONGO COMM #1                     |

0                   1                   2                   3
|                       VALUE MONGO COMM #2                     |

	usw....

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 09:28:32 +0200

... and just to make it very clear if there are multiple communities
sharing the same TLV type, same flags and same context-as implementation is
free to group it together under the same common header:

0                   1                   2                   3
|                       VALUE MONGO COMM #3                     |
|                       VALUE MONGO COMM #4                     |
|                                 ...                           |
|                       VALUE MONGO COMM #N                     |

That in fact means that for large communities if large wants to always use
same flags there is 4-6 bytes of common header overhead and no difference
to the user/operator in BGP policy in the CLI or API at all.

Thx,
R.

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 15 Sep 2016 12:13:01 +0100

> kicking this football down the field another round?

yes - an extensible communities framework capable of handling >= 32b:32b
has not materialised since support for 4-octet AS numbers was formalised
in 2007.  This issue is now burning hot in the operational community: we
are scraping the bottom of the barrel with asn16 pools at the RIRs and
we are no longer in the position that we can wait until consensus is
reached on a common header.

If we go for a common header, this does two things: first, it will take
inherently longer to get something out the door due to the usual
consensus bashing and secondly, whatever appears will be more complex.
This isn't about PDU encoding: it's a matter of overall systems
complexity: defining flexible grammars to handle the complexity
introduced by a TLV based common header, coding, documentation,
training, inter vendor compatibility, testing, etc, right down to
operational complexity.  This raises the bar significantly in terms of
both vendor and to a slightly lesser extent, operator adoption.  Raising
the bar for vendor implementation means that timeframes will be pushed
down the road.

We need something now and it needs to be simple.  The decision about
using a common header is something that ought to have taken place years
ago, and we no longer have the luxury of waiting.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 13:23:49 +0200

Hi Nick,

Just a bit side question ...

If this is so urgent have you considered to use extended community which
within 6 octets allow you to put both 4 octet AS and 2 octet for meaningful
marker ? From that thread I got that context-as is not so critical for
large communities.

All implementations support that already in all deployed gear so you can
get 32:16 essentially now - even without any IDR and IETF process.

Thx,
R.

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 15 Sep 2016 12:36:56 +0100

> get 32:16 essentially now - even without any IDR and IETF process. 

people have repeatedly said going back many years that 6 octets is not
enough.  We need more than 32b:32b:

- 32 bits for subject ASN
- 32 bits for object ASN
- enough bits for semantics

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 13:50:23 +0200

Nick,

Many times folks on this list quoted RFC1997 which allows 16:16.  First two
octets were usually ASN second two octets usually agreed semantics.

Example: https://onestep.net/communities/as3356/

No doubt that 32:32:32 would be nice to have, but there is difference
between nice to have and absolute must have - especially considering the
requested urgency.

If the only reason for large communities are 4 byte AS numbers I am just
asking why 32:16 would not cut it ?

Please do not interpret this as any attempt against Large Comms ... it is
just a question how Large Comms are to serve the same as RFC1997 except
accommodating 4 octet ASNs ?

Thx,
R.

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 15 Sep 2016 13:01:41 +0100

> asking why 32:16 would not cut it ? 

Because it's operationally deficient in terms of the signaling semantics
we need an as4octet world.  There's been a good deal of discussion about
this already, in many forums, going back years.

Nick

---

From: "Dickinson, Ian" <[email-masked]>
Date: Thu, 15 Sep 2016 12:03:44 +0000

> If the only reason for large communities are 4 byte AS numbers I am just asking why 32:16 would not cut it ?

I believe both the transit provider and the IXP route server policy use case has been pretty clear on this.
When the context-ASN is 32b and the "subject-ASN" is also 32b, we have a problem, even before signalling the desired action.

Ian
Information in this email including any attachments may be privileged, confidential and is intended exclusively for the addressee. The views expressed may not be official policy, but the personal views of the originator. If you have received it in error, please notify the sender by return e-mail and delete it from your system. You should not reproduce, distribute, store, retransmit, use or disclose its contents to anyone. Please note we reserve the right to monitor all e-mail communication through our internal and external networks. SKY and the SKY marks are trademarks of Sky plc and Sky International AG and are used under licence.

Sky UK Limited (Registration No. 2906991), Sky-In-Home Service Limited (Registration No. 2067075) and Sky Subscribers Services Limited (Registration No. 2340150) are direct or indirect subsidiaries of Sky plc (Registration No. 2247735). All of the companies mentioned in this paragraph are incorporated in England and Wales and share the same registered office at Grant Way, Isleworth, Middlesex TW7 5QD.

---

From: Gert Doering <[email-masked]>
Date: Thu, 15 Sep 2016 14:19:38 +0200

Hi,

> 16:4:4 ? Or define Large Ext Communities with yet one more attribute ?

4:4:4 already has more bells and whistles than I would want to implement
in my network, so it nicely works for me.

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (0)89/32356-444           USt-IdNr.: DE813185279

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 15 Sep 2016 15:13:07 +0200 (CEST)

> marker ? From that thread I got that context-as is not so critical for

Of course people have considered that. If this was sufficient, people 
wouldn't come here screaming about needing 4+4 NOW. Well, it's not needed 
now, it was needed 5-10 years ago. People have returned 32bit ASNs to RIRs 
for years and years because they just weren't feature complete with 16 bit 
ASNs. Now we're out of 16bit ASNs, and we need 32bit ASNs to be feature 
complete, and we need it ASAP.

We need a quick fix for this specific issue, then you can work on the 
"proper long term, extensible solution" in peace and quiet, and to get it 
right. It doesn't matter if that takes 12-18 months, and it takes another 
12-18 months to get this into widely available running code. The issue is 
no longer urgent as the 32bit ASN issue then has a solution.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 15:18:26 +0200

Mikael,

The reality is that if something is not urgent it never get's any attention
and never get's done neither by IETF nor by the vendors.

There always comes something urgent and more important on top.

So if we do not do things properly now we may as well forget about doing it
ever.

Cheers,
R.

---

From: "i3D.net - Martijn Schmidt" <[email-masked]>
Date: Thu, 15 Sep 2016 16:28:27 +0200

Hi Robert,

I think you have answered your own question quite nicely already. 16:16
allows one to put a 2-byte AS number into the first two octets, and
another 2-byte AS number into the last two octets, if so desired. If we
went with 32:16 we can't replicate RFC1997 functionality with 4-byte AS
numbers because we could only enter a 2-byte number into the second half
of the community tag. This in turn creates a feature disparity (a
downgrade even) between the good ol' RFC1997 system and the new one.

Furthermore - even if we lived in a world where nobody used 4-byte AS
numbers we could do with more space in the second half of the RFC1997
communities for reasons I will outline below. That means there are other
motivations for a swift implementation of a new community standard and
while the "Wide" communities standard seems to have interesting
properties, vendors have previously indicated it is relatively difficult
to implement which is evidenced by 6 years spent as a draft with zero
examples of running code. "Large" fits my requirements as well and even
though it's only two months old there are already several working
implementations today, thanks in no small part to its very simple design.

As promised, I'd like to share some background information on how we
operators use the RFC1997 communities in the real world so you can
understand why we're asking for a third set of octets in the "Large"
communities standard resulting in the 32:32:32 format. These were part
of the design considerations for our own AS49544 BGP action community
system [1]:

1) We should never confuse the end user as to which network is being
targeted by the community tag. Though RFC1997 is entirely opaque, we
tend to avoid using anything but our own RIR-allocated ASN or RFC1930
private AS numbers in the first two octets for externally visible BGP
communities.

2) The target of my action could be a specific AS number - like my
upstreams AS2914, AS3320, AS3356, etc - or an entire Internet exchange
for which I've typically allocated a number from the RFC1930 range.
Because we'd prefer not to use another operator's AS number in the first
two octets as per item #1, and we don't want to create confusion [2] as
to which ASN we're targeting, we have decided to put the target
information into the last two octets.

3) We typically implement at least four actions per target - prepend
once, prepend twice, prepend thrice, and do not advertise. This means a
format like "49544:target" can't cover all the actions, and again as per
item #1 we don't want to confuse the end users, so we have to resort to
using numbers from the RFC1930 range to define my routing policy
actions. We have chosen a format of 65501:3356, 65502:3356, 65503:3356
to signal one, two, and three prepends and 65500:3356 to signal do not
advertise to our direct interconnections with AS3356.

4)  Unfortunately there are nowhere near enough bits to avoid collisions
in the first two octets because we're all competing for the small chunk
of RFC1930 space where we can use the last digit to signal our desired
action of 0, 1, 2 or 3. AS49544's own design happens to collide with the
one in use by our direct upstream AS2914 [3] and the real-world impact
is that we have to strip such action communities before exporting to our
BGP neighbors to avoid unintentional routing table segmentation. An
example which comes to mind is using 65500:3356 to stop AS49544 from
exporting a prefix to AS3356, but since AS2914 will take the same action
that means AS3356 and its single-homed customers would no longer hear
that specific prefix via any path if we didn't strip that community
before exporting to AS2914. In turn, this means information as to why
AS49544 is doing something to the routing table is lost in transit.

What I've described above is a very common design philosophy amongst
major backbone operators - one that should be preserved as much as
possible while still tackling the aforementioned problems. We need to
avoid having to re-educate the majority of the operator community, so a
ton of new "knobs and twiddles" are quite simply put undesirable when
the same could be achieved with a much simpler "more of the same" design
that everyone who's familiar with RFC1997 should intuitively grasp.

Coming back to the question as to why 32:32:32 would be desirable - if
we could signal something like 49544:3356:0 instead we can tackle item
#4 to clarify that only AS49544 should take action, we can tackle item
#2 because we have the ability to put any operator's 2-byte or 4-byte AS
number into the set of octets which our network has defined as target,
and we can tackle item #3 to signal an action which could now achieve a
lot more than just a simple prepend or don't export. Use cases which
come to mind are AS286's beautifully engineered "extended regional Black
Hole" system that even lets the end user target specific PoP's within a
single metro area for a blackhole.

Best regards,
Martijn Schmidt

[1] http://noc.i3d.net/bgp-communities/ - we need to update this
document to account for some changes in routing policy / upstream
providers, but the general idea is still the same.
[2] https://onestep.net/communities/as3549/ - as you can see in this
operator's real-world example, the former GlobalCrossing network used a
format like 3549:8122 to prepend twice on peerings with AS2914. However,
2914 does not show up in the action community tag 3549:8122 and that
could be quite confusing for an end user such as myself.
[3] https://www.us.ntt.net/support/policy/routing.cfm
[4] http://as286.net/AS286-erBH.html

On 09/15/2016 01:50 PM, Robert Raszuk wrote:
> Nick,
>
> Many times folks on this list quoted RFC1997 which allows 16:16. 
> First two octets were usually ASN second two octets usually agreed
> semantics. 
>
> between nice to have and absolute must have - especially considering
> the requested urgency. 
>
> Please do not interpret this as any attempt against Large Comms ... it
> is just a question how Large Comms are to serve the same as RFC1997
> except accommodating 4 octet ASNs ?
>
> Thx,
> R.
>
> On Thu, Sep 15, 2016 at 1:36 PM, Nick Hilliard <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>     Robert Raszuk wrote:
>     <https://www.ietf.org/mailman/listinfo/idr>
>
>
>
>

---

From: Gert Doering <[email-masked]>
Date: Thu, 15 Sep 2016 16:32:20 +0200

Hi,

> So if we do not do things properly now we may as well forget about doing it

If nobody wants -wide that urgently, this might be an option to consider.

Gert Doering
        -- NetMaster

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 15 Sep 2016 16:12:22 +0100

Robert Raszuk wrote:
> The reality is that if something is not urgent it never get's any
> attention and never get's done neither by IETF nor by the vendors. 
> 
> So if we do not do things properly now we may as well forget about doing
> it ever. 

Robert, this is the sort of argument that should have been presented in
the 2001-2007 time frame, while the IDR was working on the as4octet ID.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 17:26:58 +0200

Nick,

I wish I could move back time and present current ideas 10 years back.

But just for the record the idea for Wide Communities was first born in
2010 after Hannes and I + bunch of contributors listed in section 8 and 9
had a bunch of discussions with folks tired about current manual way of
managing common policies in BGP. Handling v6 and 4 octet AS was there in
the draft day one.

Ref: https://tools.ietf.org/html/draft-raszuk-wide-bgp-communities-00

Thx,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 17:45:27 +0200

And as far as formal adoption of Wide BGP Communities along with separated
per IDR WG recommendation Registered List of Most Common BGP policies
embedded into RFC1997 communities just happened one year ago - June 2015.

So calling it names like old, stalled, rejected to be implemented isn't a
bit premature?

Copy of adoption pool attached below with number of operators supporting it.

Thx,
R.

---

From: Randy Bush <[email-masked]>
Date: Thu, 15 Sep 2016 08:59:57 -0700

> using a common header is something that ought to have taken place
> years ago, and we no longer have the luxury of waiting.

and this discussion should be bookmarked to illustrate why ops are
really unhappy with the ietf.

randy

---

From: heasley <[email-masked]>
Date: Thu, 15 Sep 2016 16:01:25 +0000

> that everyone who's familiar with RFC1997 should intuitively grasp.

Robert, Also understand that transitioning from these jumbled communities
to make due with 16:16 to another jumbled community using 32:16 and later
to 32:32:32 is not free nor cheap.  Each of those steps takes time and
entails costly opex for the operator and customers - and all change invites
errors.  Transitioning only once is important in many ways.

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 18:04:10 +0200

Fully agree ! Adding more work to anyone was not an intention.

It was just a quick idea to address urgency to encode something like this
today in RFC1997 style. By no means I was prescribing this as proper
solution ... just a quick hack.

---

From: Randy Bush <[email-masked]>
Date: Thu, 15 Sep 2016 09:04:54 -0700

there is no proof of termination arguing with robert.  and it does not
get the job done anyway.

we need to get this out.  lack of 32+32+semantics has become intolerably
painful to operators.  can we focus on a single draft now and get it the
hell done?

randy

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 15 Sep 2016 17:11:06 +0100

Robert Raszuk wrote:
> So calling it names like old, stalled, rejected to be implemented isn't
> a bit premature?

I have not ever used these words to describe -wide, nor have I ever said
anything disparaging about the draft.  I have, however, said that
previous versions were too complex, and stand by that.

It also does not have running code, vendor traction or consensus on a
final protocol structure.  These are not criticisms; just statements of
the current status of the proposal.  All of them can be fixed, but it
will take time to do this, and we are out of time because IANA is out of
ASN16s.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Thu, 15 Sep 2016 18:22:32 +0200

Just to perhaps completely surprise Randy let me post final email in this
thread ...

This discussion is not about -wide communities. It is specifically about
either:

*A* progress AS:32:32 ​naked in the dedicated new attribute

or

*B* progress AS:32:32 with added TLV header such that said new attribute
can also accommodate other already known similar needs

I am in support of *B* provided that we converge quickly on the format of
just few octets defining such common header.

Thank you,
Robert.

---

From: Jared Mauch <[email-masked]>
Date: Thu, 15 Sep 2016 12:22:45 -0400

I'm not in favor of calling anything an "AS Number", even though
that could be what is stuck into that envelope.

	Today we provide customers communities that happen to have
not-our-ASN as well as include our ASN in that first integer space.

	if the desire is to have the wire have A then B:C of a
A:B:C community (an example) on the wire so we can do

A
B:C
D:E 

to encode A:B:C and A:D:E this seems like it could make sense.

	- Jared

---

From: Jared Mauch <[email-masked]>
Date: Thu, 15 Sep 2016 12:24:10 -0400

> get 32:16 essentially now - even without any IDR and IETF process.

	This doesn't let me target 32-bit ASNs so misses the mark.

	- jared

---

From: Jared Mauch <[email-masked]>
Date: Thu, 15 Sep 2016 12:25:37 -0400

> If nobody wants -wide that urgently, this might be an option to consider.

	I there is viable work to save from -wide, we should do so
and the rest can become another draft and call for adoption on
that work.  Cut + pate should help here to save the work.

	- jared

---

From: Jared Mauch <[email-masked]>
Date: Thu, 15 Sep 2016 12:32:06 -0400

> asking why 32:16 would not cut it ?

	Because I may have a 32-bit ASN and want to encode a policy that
targets a 32-bit ASN.

	eg: 232232:655000

	Suppress my route to 32-bit ASN 655000.

> accommodating 4 octet ASNs ?

	There is some operational elegance in having something like

232232:655000:1

	To say :for my value of 1, this has an action of supress, prepend, drop
or whatever that common action is.

	This way when I innovate with my new policy that says the :8675309
tagged route will supress my route to Unroutableastan, I can have a simple
iBGP policy that just matches on ingress in Unroutableastan that drops
routes with $a:$b:8675309

	This operational nuance doesn't seem captured in any of
the existing work, so may seem new to those in IDR and why something
like draft-ietf-idr-as4octet-extcomm-generic-subtype-03.txt does not
seem to capture the actual use case of passing what inherently are opaque
policy variables and code through as communities.

	- Jared

---

From: Jared Mauch <[email-masked]>
Date: Thu, 15 Sep 2016 12:33:34 -0400

> this already, in many forums, going back years.

	I'm concerned that this artifact seems to appear as a /new/ use case
for some members of the WG.

	- jared

---

From: Jared Mauch <[email-masked]>
Date: Thu, 15 Sep 2016 12:37:42 -0400

> So calling it names like old, stalled, rejected to be implemented isn't a

	I don't think so, because if you look at some of the "workarounds"
pointed to by IDR members include a document that appears by name
to say it will do 4-octets, but really can't handle a 4:4 use-case.

https://tools.ietf.org/html/draft-ietf-idr-as4octet-extcomm-generic-subtype-03

(the other changes appear to be just version bumps from 03 -> 09)

	- Jared

---

From: Randy Bush <[email-masked]>
Date: Thu, 15 Sep 2016 09:43:49 -0700

> Copy of adoption pool attached below with number of operators supporting it.

please update draft-idr-history-and-blame-42.txt with this

now can we please move forward.  this is KILLING US!!!

randy

---

From: Jared Mauch <[email-masked]>
Date: Thu, 15 Sep 2016 12:48:03 -0400

> painful to operators.  can we focus on a single draft now and get it the

	I expect the Chairs (& AD) are thinking about next-steps
once the adoption timer has run to completion.

	- Jared

---

From: Randy Bush <[email-masked]>
Date: Thu, 15 Sep 2016 09:56:27 -0700

> I expect the Chairs (& AD) are thinking about next-steps once the
> adoption timer has run to completion.

I am sure that thinking will be referred to the sub-committee on future
objectives prioritization to determine if thinking should be identified
as a stretch goal and the standing committee on committees will open a
hearing regarding how to expedite the creation of future committees that
would be empowered to actually take action after an incident was
identified by the board.  -- Gary Buhrmaster

can we please focus on the content and let the process go to the cloud,
where process is processed?

is
    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                      Autonomous System number                 |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Local Data Part 1                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          Local Data Part 2                    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

sufficient?  that is a binary question.  i am interested in answers from
operators such as nick, gert, ...

iff your answer is "no," what would be sufficient?

randy

---

From: Randy Bush <[email-masked]>
Date: Thu, 15 Sep 2016 10:19:38 -0700

> *A* progress AS:32:32 ​naked in the dedicated new attribute

32:32:32(or 42)

> can also accommodate other already known similar needs

i have lost faith that we can converge before we need 64-bit ASNs.
imiho, this path leaves way too much room for cleverness and we have
way too many players here who think clever is a good thing.

otoh, if you can get a 32:32:tlv draft out by monday, and we have no
technical objections from ops by the end of next week, cool.  i'll buy
you a nice dinner in seoul.  i have nothing against the idea, just a
deep skepticism of the idr process.  (yes, jeff, i am thinking of you)

32:32:96 has the advantage that it is brutally simple.  if we can push
the rich discussion of encodings and semantics of the righmost field to
other drafts, ask vendors to just do it (and we do have the cash to
motivate them), ops can get on with trying to keep the internet cobbled
together while this erudite group can do ever more clever things.

randy

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Thu, 15 Sep 2016 17:24:53 +0000

On 9/15/16, 12:48 PM, "Idr on behalf of Jared Mauch" <[email-masked]

>once the adoption timer has run to completion.

I agree. Consensus needs to be gauged on:

  1. Adoption of the draft (which seems to be overwhelming apart from
objections from authors of the wide draft).
  2. Assuming “Yes” for #1, whether or not it should be progressed
immediately in its current form or with modifications to provide a base
for future requirements and preservation of the IANA Attribute identifier
space. 

Thanks,
Acee

---

From: Jeffrey Haas <[email-masked]>
Date: Thu, 15 Sep 2016 13:35:43 -0400

> On Sep 15, 2016, at 1:19 PM, Randy Bush <[email-masked]> wrote:

I haven't seen a single post from the influx of operational folk that is a technical objection on a common header.  I've seen a significant number of posts that seem to think that by adding a common header, "it's wide comms, it'll never get done, it's too hard"... whatever.  If a fixed header on the front of a list was considered too difficult, we should kill pretty much all IETF work.

Is -large sufficient for the specific use case? Yes.
Do I want a common header so I can stop telling people they have to filter objectionable optional transitive stuff causing their network issues using path attribute filtering (harmful to new features, period)? Absolutely.

A trimmed draft derived from the -wide-03 containing only common header and 4:4 dense packing can be produced in an hour.  That document would be functionally identical to -large with a common header.  This is not the issue.

And nice to know you're thinking so kindly of me, Randy.  We'll discuss our other collaborations at a later date.

-- Jeff

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 15 Sep 2016 18:38:39 +0100

> is

yep, operationally that works.

Nick

---

From: Christopher Morrow <[email-masked]>
Date: Thu, 15 Sep 2016 13:41:42 -0400

> for some members of the WG.

This is a by-product of network operations saying: "Today I have
2byte:2byte and and I expect that tomorrow clearly when I have a 4byte AS
I'll be able to 4byte:4byte"  'assumed to work" (or assumed to have been
planned to work).

this doesn't seem to be a bad assumption on the part of network operations,
at least to me. Overall, if the original definition of communities includes
this:

"The rest of the community attribute values shall be encoded using an
   autonomous system number in the first two octets."

I'd have expected someone to figure out (before emergency measures are
required) that 'huh, ASN are now 4bytes, maybe we should rev communities
and adjust for this?" I'd have expected that at or about the time:
  <https://tools.ietf.org/html/rfc4893>

In fact in 'protocol extensions' for 4893:
"BGP also carries the Autonomous System number in the BGP
   Communities attribute."

Oh, but that's dealt with handily here:
"5.  Handling BGP Communities

   As specified in [RFC1997], when the high-order two-octets of the
   community attribute is neither 0x0000 nor 0xffff, these two octets
   encode the Autonomous System number.  Quite clearly this would not
   work for BGP speakers that use 4-octets Autonomous System numbers.
   Such BGP speakers should use the Four-octet AS Specific Extended
   Communities [AS-EXT-COM] instead."

perhaps as-ext-com is:
  <https://tools.ietf.org/rfc/rfc5668.txt>

except that this 'steals' bytes for some global-scope stuff :( So, if at a
minimum you wanted: 4:4 then 5668 is short 2 :(

---

From: Randy Bush <[email-masked]>
Date: Thu, 15 Sep 2016 10:43:08 -0700

> I haven't seen a single post from the influx of operational folk that
> is a technical objection on a common header.  I've seen a significant
> number of posts that seem to think that by adding a common header,
> "it's wide comms, it'll never get done, it's too hard"... whatever.
> If a fixed header on the front of a list was considered too difficult,
> we should kill pretty much all IETF work.
> 
> Do I want a common header so I can stop telling people they have to
> filter objectionable optional transitive stuff causing their network
> issues using path attribute filtering (harmful to new features,
> period)? Absolutely.
> 
> A trimmed draft derived from the -wide-03 containing only common
> header and 4:4 dense packing can be produced in an hour.  That
> document would be functionally identical to -large with a common
> header.  This is not the issue.

as i said, if i see a complete simple single draft by monday and
agreement with ops by end of next week, i'll take it seriously.
otherwise, some of us have a 32-bit AS internet to run.

randy

---

From: Jared Mauch <[email-masked]>
Date: Thu, 15 Sep 2016 13:46:12 -0400

> identified by the board.  -- Gary Buhrmaster

	How cute.

	Yes.

	(with repeated caveat of not calling it ASN).

	- jared

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 15 Sep 2016 19:48:37 +0200 (CEST)

> otherwise, some of us have a 32-bit AS internet to run.

I agree, but I also want to hear from the people with running code, if 
they agree that the new proposal is easily implementable.

If all of this can be done by end of next week, I'll also support the 
proposal.

---

From: Randy Bush <[email-masked]>
Date: Thu, 15 Sep 2016 10:49:50 -0700

> they agree that the new proposal is easily implementable.

already have word from one notorious^Hfamous bgp hacker that 32:32:42
is a day's work.

---

From: Christopher Morrow <[email-masked]>
Date: Thu, 15 Sep 2016 14:03:06 -0400

(shortcut; I'd like to see -large get finished, some reasoning below)

> is

I'll bite... So, it seems to me that a minimum answer for compatibility
going forward is 4:4, it also seems that if we're adding more bits to this,
4:8 is attractive for some of the use-cases Jared brings up above.

The -large option seems like a good step to bring parity to tooling/etc
that we use today in the network.

The -wide option is... complex. I don't favor that, today.

It also seems to me that we can live in a world with -wide and -large. For
folk that need the extra parts that -wide has they'll be able to do what
they want, for folk that need to keep and grow their current deployments
-large seems like a good fit.

Does this have to be an either/or? Why can't it be 'and'?

I think jeff's single header option is an attractive abstraction layer,
will that be finalized and ready for deployment in before -large could be?
  o I expect it's at least 24 months away from 'on my network' (likely 36
to 48 given another year of discussion here before ietf-lc)

Can we move all of the use-cases for -large or -wide under -common-header
when that's arrived?
  o maybe? probably? again in 36-48 months + migration time...

Operationally, having no answer for; "my new asn needs to express policy as
I did in my old ASN" is .. bad.

-chris

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 15 Sep 2016 18:07:32 +0000

If large communities come under a common header, then any bits in it
will be ignored by large communities. Large communities will be transitive
regardless of the transitivity bits set in the header.
Otherwise these bits will need to be dragged into the route-policy code
to be matched and set. That could be enough to push the feature out
of its slot in the next release into some other future release.

It's not that it can't be done. It's that it competes with other features
that also need to get done. If it's too big, then it gets pushed out.

Another complexity is the attribute filter.
Because communities under the header are really just BGP attributes
in disguise, the attribute filter code needs to be enhanced to filter
community "types" also.

A further issue is that because the header is common with wide
communities, we will be forever changing it to suit wide communities,
which will hold up any draft depending upon it.

Thanks,
Jakob.

> From: Idr [mailto:[email-masked]] On Behalf Of Mikael Abrahamsson
> Sent: Thursday, September 15, 2016 10:49 AM
> To: Randy Bush <[email-masked]>
> Cc: Interminable Discussion Room <[email-masked]>

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 15 Sep 2016 14:10:35 -0400

Jeff: 

Thank you for pointing out the overlap between the two drafts.  John and I
will take this into account in determining the results of the WG Adoption
call.  We are monitoring the discussion.  A key part of the discussion is
the operator requirements that are driving the expansion of communities,
implementation issues, and deployment issues. 

Sue 

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Thursday, September 8, 2016 6:04 PM
To: Susan Hares
Cc: [email-masked]
Subject: Re: [Idr] Request to adopt draft-heitz-idr-large-community -
Working Group Adoption call (9/6 to 9/20)

Sue and John,

On Tue, Sep 06, 2016 at 12:54:59PM -0400, Susan Hares wrote:
> authors indicate IPR within 1-2 days, we'll let the poll continue.  If 
> not, we'll halt this adoption poll to await the authors response.

-- Jeff

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 15 Sep 2016 18:23:31 +0000

32:32:tlv is going down the road of wide.
Each "t" in tlv will need route-policy code to deal with it.
This is significantly harder than 32:32:32.
It's not just about getting a draft by Monday.

Thanks,
Jakob.

> From: Idr [mailto:[email-masked]] On Behalf Of Randy Bush
> Sent: Thursday, September 15, 2016 10:20 AM
> To: Robert Raszuk <[email-masked]>

---

From: t.petch <[email-masked]>
Date: Fri, 16 Sep 2016 11:11:57 +0100

----- Original Message -----
From: "Robert Raszuk" <[email-masked]>
Sent: Wednesday, September 14, 2016 8:35 PM

<tp>
And failing:-(

Looking at the CSS in the text/html I see

<div class="gmail_default" style="font-size:small"><font
face="monospace, monospace">/* Trying to send in plain txt
*/</font></div>

which explains the small font for those using HTML to display the
e-mail.

For myself, I configure my MUA to display plain text so given that this
e-mail, like many on the list is multipart/alternative, I see the text
version and have no problem with whatever the HTML alternative
specifies.

I note that this HTML formatting is carried across into the archives,
which is unfortunate.

Once upon a time, a WG chair had the power to influence what a mail list
accepted and caused the mailing list software to strip the HTML
alternative before resending to the list.  What a wonderful idea, but I
fear that technology has move on since then.  The size of the HTML
alternatives, which I have seen reach one Mbyte - yes Mega - are a
problem for me so I would love to see them purged.

Tom Petch

I also do not see any complexity neither - just the opposite simplicity
in
filtering - with the proposed encoding as well as in the policy
language.

Thx,
R.

> should have been repeated in every code point's documentation.

------------------------------------------------------------------------
--------

---

From: Jay Borkenhagen <[email-masked]>
Date: Fri, 16 Sep 2016 15:56:14 -0400

> 	(with repeated caveat of not calling it ASN).

Sufficient for me as well.  Strongly in favor of large at 32:32:32
over anything else I've heard discussed here.

						Jay B.

---

From: Gert Doering <[email-masked]>
Date: Fri, 16 Sep 2016 22:06:27 +0200

Hi,

> is

Operator-wise, anything that stores 4:4:4 is more than I have creative
ideas what to do with it.  So, "yes".  All use cases I see for our
network and IXPs are "youras:someotheras:something", and 32bits for 
all 3 sub-fields are good.

Implementation-wise, I leave the packing of these bits to those that
have written code and understand what stuff things work.

Gert Doering
        -- NetMaster

---

From: Randy Bush <[email-masked]>
Date: Fri, 16 Sep 2016 17:08:20 -0700

> all 3 sub-fields are good.

have faith in the ietf embellishment; they can 'enhance' it.

randy

---

From: Gert Doering <[email-masked]>
Date: Sat, 17 Sep 2016 22:41:53 +0200

Hi,

> have faith in the ietf embellishment; they can 'enhance' it.

I have total faith :-) - but I just wanted your question answered, from
an operator point of view (albeit a small network) - "yes, 4:4:4 is good
enough for us".

Gert Doering
        -- NetMaster

---

From: Arnold Nipper <[email-masked]>
Date: Sun, 18 Sep 2016 09:09:59 +0200

Hello,

> an operator point of view (albeit a small network) - "yes, 4:4:4 is good

4:4:4 should really do for the next couple of years and what we have in
mind. I fully support this draft being put forward.

Arnold
-- 
Arnold Nipper
email: [email-masked]      phone: +49 6224 5593407 2
mobile: +49 172 2650958      fax:   +49 6224 5593407 9

---

From: Sander Steffann <[email-masked]>
Date: Sun, 18 Sep 2016 18:11:02 +0200

Hi Randy,

> where process is processed?

Yep, that seems to do just fine. I like the simplicity.

Cheers,
Sander

---

From: Randy Bush <[email-masked]>
Date: Sun, 18 Sep 2016 09:25:42 -0700

> Yep, that seems to do just fine. I like the simplicity.

while operators keep saying this, it is utterly uncreative and shows
zero cleverness.  how do professional ietf standards makers turn this
into resume enhancing material?

randy

---

From: Sander Steffann <[email-masked]>
Date: Sun, 18 Sep 2016 18:29:57 +0200

Hi Randy,

> into resume enhancing material?

I like zero cleverness, it provides fewer creative ways to get things wrong :)  I guess people will have to be satisfied with "my standard was so simple to implement that nobody managed to f*ck it up" on their resumes...

Cheers,
Sander

---

From: John Scudder <[email-masked]>
Date: Sun, 18 Sep 2016 16:35:35 +0000

All,

I for one would appreciate it if the conversation could stay on a technical level and not stoop to questioning people's motives.

I'm sorry to have to say this, I know everybody knows it. But I guess we all need a reminder occasionally.

Thank you.

--John

---

From: Randy Bush <[email-masked]>
Date: Sun, 18 Sep 2016 09:43:39 -0700

my apologies.  i really do not intent to cast aspersions on any person
or their intent.  and sarcasm is not helpful.  but ops' many year
frustration with this is painful; and we have failed to communicate.

my apologies.  i'll shut up now.

randy

---

From: Joe Provo <[email-masked]>
Date: Mon, 19 Sep 2016 09:27:27 -0400

> that could be what is stuck into that envelope.

Yes please. While not stated as such in 1997, the exaples placed in 
1998 communicated that one implementation. By the time 4384 came around 
a decade later, using ASN was referred to as 'best current practice' 
with the implication being the BCP was for exclusively using ASN in 
the high octets.

However these proposal move forward, I strongly agree with Jared that
we should cease instructing others to paint themselves into a numberspace
corner.

Cheers!

Joe

-- 
        RSUC / GweepNet / Spunk / FnB / CotSG / Usenix / NANOG

---

From: "John G. Scudder" <[email-masked]>
Date: Mon, 19 Sep 2016 13:31:24 -0400

> problem for me so I would love to see them purged.

Now that you mention it, I do see mailman options including

- Should Mailman collapse multipart/alternative to its first part content?
- Should Mailman convert text/html parts to plain text? This conversion happens after MIME attachments have been stripped.

Right now these are turned off, as is the rest of the content filtering. If nobody objects, I'm inclined to try turning them on.

Comments unicast to chairs or to the list.

Thanks,

--John

---

From: Sander Steffann <[email-masked]>
Date: Mon, 19 Sep 2016 19:48:30 +0200

Hi,

> we should cease instructing others to paint themselves into a numberspace

+1

Making it a best practice is fine, mandating it might not be a good idea.

Cheers,
Sander

---

From: Job Snijders <[email-masked]>
Date: Mon, 19 Sep 2016 20:52:28 +0300

I am game writing a RFC1998-style document to help inspire how to use Large
Communities.

I don't believe it should contain a lot of MUST or SHOULD, but we can
capture some ideas and patterns. Martijn Schmidt's lengthy email can be
used as a starting point.

Job

---

From: Sander Steffann <[email-masked]>
Date: Mon, 19 Sep 2016 19:54:18 +0200

Hi Job,

> I am game writing a RFC1998-style document to help inspire how to use Large Communities.
> 
> I don't believe it should contain a lot of MUST or SHOULD, but we can capture some ideas and patterns. Martijn Schmidt's lengthy email can be used as a starting point.

Sounds like a plan :)
Sander

---

From: Joe Provo <[email-masked]>
Date: Mon, 19 Sep 2016 15:38:50 -0400

TL;DR - forward motion on -large would prevent making networks 
deploying 32b today ASNs as second-class citizens^Wcarriers of 
the global Internet.

> that we use today in the network.
[snip]

What my colleague says regarding parity is critical. We may not *like* 
the existing tools, approaches, or configuration management (and many 
parties are seeking to correct this), but the operations end of the 
Internet requires forward progress with the least friction. A more 
thorough rethinking of communicating routing policies is a laudable 
goal, but that's not something we'll see in the near-term.

As Nick observed, the well of 16b ASNs is dry at the source. When 
the RIRs are unable to fulfill requests, we'll see operators of 32b 
ASNs forced to abandon best current practices in policy signaling 
(the danger of overloading private-ASN signaling and making stripping 
errors or re-writes have already been raised) or find some means of 
obtaining 16b ASNs.  Flatly, any such with BGP downstreams will be 
forced to have substandard offerings to their customers and the 
collective we are affecting their business offering.

Many of the relevant points have been touched on by others, most 
notably the common framework indicated by Martijn. In a comment,
Jared pointed to the elegance of "<actor>:<target>:<action>" signals.
I would take that a step further to say empowering that simplicity 
would increase stability of the Internet. In Martijn's excellent
policy framework summary, one of the footnote-referred websites 
might not be familiar to non-operators: https://onestep.net/communities/
This site is a compilation of various provider community lists,
not always published to non-customers, and a fairly important 
reference for those crafting announcements or diagnosing what 
happened to a prefix or traffic. If you pick a few at random, 
you'll note that any which are providing action (and not just 
documenting origin) need to define their own format and action
look-up table. Human touching/configuring devices are known to 
be the root of most inter-provider turbulence; humans have been 
known to set the policy matching provider A's LUT on provider 
B's session, sometimes with bad results. Setting the stage to 
reduce complexity in this realm by providing the bitspace for 
a simple, uniform "<actor>:<target>:<action>" signal would 
increase stability as it was adopted.

The need is real and imminent. There are multiple implementations
of the proposal. A chorus of people facing these issues on a 
daily basis support the proposal.  Let us please move forward.

Cheers!

Joe

---

From: "Russ White" <[email-masked]>
Date: Mon, 19 Sep 2016 15:50:07 -0400

> couldn't be accommodated in a TLV framework?

No -- and I'm fine with draft-heitz-idr-large-community --

/r

---

From: Ben Maddison <[email-masked]>
Date: Mon, 19 Sep 2016 21:21:25 +0000

Hi all,

Thanks for all the contributions so far.

I'd like to echo the support for this proposal already voiced.
We operate a transit network, and define multiple informational and action communities for signaling policy to/from our customers and peers.
As it stands, the primary shortcoming in this scheme is our inability to indicate scope of a 4B ASN. This proposal fixes that simply and elegantly, and does not break any elements of our existing deployment.

It is trivial to map our current community namespace into a new enlarged namespace, thus providing a migration path that can remain in a transitional state for as long as necessary.
I would point out that this operational simplicity derives from the fact that the handling of the proposed attribute is the same as the current std. community attribute in every respect other than length - in particular transitivity. Without this, one cannot use both short and long communities to express a single policy during transition.

As others have pointed out, the same issue is felt acutely by holders of 4B ASN and by route server operators.
In the Afrinic numbers region, the community has even resorted to what can only be described as a "resource policy hack" [1] that reserves the remaining pool of 2B ASNs for allocation to IXP RS operators, so that they can define per-peer policy communities. Even this measure of course does not help where IXP participants have 4B ASNs.
Fixing operational problems with Internet standards is clearly not the proper role of resource policy, and highlights the brokenness of the status quo, whereby we are unable to agree on a reasonable protocol based solution to an immediate protocol related problem.
I have copied the authors of that policy, all of whom operate IXPs in our region, in case they would like to add their thoughts. 

[1] http://afrinic.net/en/library/policies/1423-afpub-2014-gen-004-draft-03 

Ben Maddison

Director
Workonline Communications (Pty) Ltd

Office:     +27 (0) 21 200 9000
Mobile:   +27 (0) 82 415 5545
Email:      [email-masked]
SIP:          [email-masked]

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of Joe Provo
Sent: Monday, September 19, 2016 10:39 PM
To: [email-masked]
Subject: Re: [Idr] Request to adopt draft-heitz-idr-large-community - Working Group Adoption call (9/6 to 9/20)

TL;DR - forward motion on -large would prevent making networks deploying 32b today ASNs as second-class citizens^Wcarriers of the global Internet.

On Thu, Sep 15, 2016 at 02:03:06PM -0400, Christopher Morrow wrote:
> > sufficient?  that is a binary question.  i am interested in answers 
> > from operators such as nick, gert, ...
> >
> I'll bite... So, it seems to me that a minimum answer for 
> compatibility going forward is 4:4, it also seems that if we're adding 
> more bits to this,
> The -large option seems like a good step to bring parity to 
> tooling/etc that we use today in the network.
[snip]

What my colleague says regarding parity is critical. We may not *like* the existing tools, approaches, or configuration management (and many parties are seeking to correct this), but the operations end of the Internet requires forward progress with the least friction. A more thorough rethinking of communicating routing policies is a laudable goal, but that's not something we'll see in the near-term.

As Nick observed, the well of 16b ASNs is dry at the source. When the RIRs are unable to fulfill requests, we'll see operators of 32b ASNs forced to abandon best current practices in policy signaling (the danger of overloading private-ASN signaling and making stripping errors or re-writes have already been raised) or find some means of obtaining 16b ASNs.  Flatly, any such with BGP downstreams will be forced to have substandard offerings to their customers and the collective we are affecting their business offering.

Many of the relevant points have been touched on by others, most notably the common framework indicated by Martijn. In a comment, Jared pointed to the elegance of "<actor>:<target>:<action>" signals.
I would take that a step further to say empowering that simplicity would increase stability of the Internet. In Martijn's excellent policy framework summary, one of the footnote-referred websites might not be familiar to non-operators: https://onestep.net/communities/ This site is a compilation of various provider community lists, not always published to non-customers, and a fairly important reference for those crafting announcements or diagnosing what happened to a prefix or traffic. If you pick a few at random, you'll note that any which are providing action (and not just documenting origin) need to define their own format and action look-up table. Human touching/configuring devices are known to be the root of most inter-provider turbulence; humans have been known to set the policy matching provider A's LUT on provider B's session, sometimes with bad results. Setting the stage to reduce complexity in this realm by providing the bitspace for a simple, uniform "<actor>:<target>:<action>" signal would increase stability as it was adopted.

The need is real and imminent. There are multiple implementations of the proposal. A chorus of people facing these issues on a daily basis support the proposal.  Let us please move forward.

Cheers!

Joe

---

From: Andrew Lange <[email-masked]>
Date: Tue, 20 Sep 2016 16:10:43 -0700

Jared,

You are, of course, correct from an information-theory point of view.  However, not everyone running BGP is as sophisticated, especially when it comes to BGP policy.  Expressing the intent, will in most cases, be respected as the desired propagation policy will have to be intentionally overridden.  

In addition, when it comes to direct-neighbor policies, on can also have contract terms and a business relationship at stake.  Two parties could agree, for example, to exchange more information than they usually would on the condition that the other party respects their desired propagation behavior.

Andrew

> On Sep 14, 2016, at 9:29 AM, Jared Mauch <[email-masked]> wrote:
> 
> Jared Mauch  | pgp key available via finger from [email-masked] <mailto:[email-masked]>
> clue++;      | http://puck.nether.net/~jared/ <http://puck.nether.net/~jared/>  My statements are only mine.
>

---

From: Andrew Lange <[email-masked]>
Date: Tue, 20 Sep 2016 16:11:09 -0700

Nick,

Your assumption is fundamentally flawed.

We ALREADY HAVE a common header, it won’t take any longer to get the right solution out the door.  

We define an initial opaque type and it will behave exactly like large. 

Problem solved, and solved with the future-resistant solution. 

Andrew

> On Sep 15, 2016, at 4:13 AM, Nick Hilliard <[email-masked]> wrote:
> 
> Jeffrey Haas wrote:

---

From: Andrew Lange <[email-masked]>
Date: Tue, 20 Sep 2016 16:11:16 -0700

Hi Martin,

> On Sep 15, 2016, at 7:28 AM, i3D.net - Martijn Schmidt <[email-masked]> wrote:
> 
> Hi Robert,
> 
> I think you have answered your own question quite nicely already. 16:16 allows one to put a 2-byte AS number into the first two octets, and another 2-byte AS number into the last two octets, if so desired. If we went with 32:16 we can't replicate RFC1997 functionality with 4-byte AS numbers because we could only enter a 2-byte number into the second half of the community tag. This in turn creates a feature disparity (a downgrade even) between the good ol' RFC1997 system and the new one.
> 
> Furthermore - even if we lived in a world where nobody used 4-byte AS numbers we could do with more space in the second half of the RFC1997 communities for reasons I will outline below. That means there are other motivations for a swift implementation of a new community standard and while the "Wide" communities standard seems to have interesting properties, vendors have previously indicated it is relatively difficult to implement which is evidenced by 6 years spent as a draft with zero examples of running code.

Incorrect - it has been there due to lack of operators asking their vendors for a solution to the community problem. This is why this is a good conversation to have.

> "Large" fits my requirements as well and even though it's only two months old there are already several working implementations today, thanks in no small part to its very simple design.
> 
> As promised, I'd like to share some background information on how we operators use the RFC1997 communities in the real world so you can understand why we're asking for a third set of octets in the "Large" communities standard resulting in the 32:32:32 format. These were part of the design considerations for our own AS49544 BGP action community system [1]:
> 
> 1) We should never confuse the end user as to which network is being targeted by the community tag. Though RFC1997 is entirely opaque, we tend to avoid using anything but our own RIR-allocated ASN or RFC1930 private AS numbers in the first two octets for externally visible BGP communities.
> 
> 2) The target of my action could be a specific AS number - like my upstreams AS2914, AS3320, AS3356, etc - or an entire Internet exchange for which I've typically allocated a number from the RFC1930 range. Because we'd prefer not to use another operator's AS number in the first two octets as per item #1, and we don't want to create confusion [2] as to which ASN we're targeting, we have decided to put the target information into the last two octets.
> 
> 3) We typically implement at least four actions per target - prepend once, prepend twice, prepend thrice, and do not advertise. This means a format like "49544:target" can't cover all the actions, and again as per item #1 we don't want to confuse the end users, so we have to resort to using numbers from the RFC1930 range to define my routing policy actions. We have chosen a format of 65501:3356, 65502:3356, 65503:3356 to signal one, two, and three prepends and 65500:3356 to signal do not advertise to our direct interconnections with AS3356.
> 
> 4)  Unfortunately there are nowhere near enough bits to avoid collisions in the first two octets because we're all competing for the small chunk of RFC1930 space where we can use the last digit to signal our desired action of 0, 1, 2 or 3. AS49544's own design happens to collide with the one in use by our direct upstream AS2914 [3] and the real-world impact is that we have to strip such action communities before exporting to our BGP neighbors to avoid unintentional routing table segmentation. An example which comes to mind is using 65500:3356 to stop AS49544 from exporting a prefix to AS3356, but since AS2914 will take the same action that means AS3356 and its single-homed customers would no longer hear that specific prefix via any path if we didn't strip that community before exporting to AS2914. In turn, this means information as to why AS49544 is doing something to the routing table is lost in transit.

I’ve engaged in the same set of manipulations.  And this sort of magic-number torture that makes BGP policy a lot harder than it has to be, and leads to a lot of mistakes.

> What I've described above is a very common design philosophy amongst major backbone operators - one that should be preserved as much as possible while still tackling the aforementioned problems. We need to avoid having to re-educate the majority of the operator community, so a ton of new "knobs and twiddles" are quite simply put undesirable when the same could be achieved with a much simpler "more of the same" design that everyone who's familiar with RFC1997 should intuitively grasp.

It’s not a design philosophy, it’s a work-around for a poor policy expression mechanism.

That said, if there are people who want to preserve this or, better yet, migrate their systems slowly to a better system, a common header, and the opaque format in the base type allows you to do exactly that.

Andrew

> 
> Coming back to the question as to why 32:32:32 would be desirable - if we could signal something like 49544:3356:0 instead we can tackle item #4 to clarify that only AS49544 should take action, we can tackle item #2 because we have the ability to put any operator's 2-byte or 4-byte AS number into the set of octets which our network has defined as target, and we can tackle item #3 to signal an action which could now achieve a lot more than just a simple prepend or don't export. Use cases which come to mind are AS286's beautifully engineered "extended regional Black Hole" system that even lets the end user target specific PoP's within a single metro area for a blackhole.
> 
> Best regards,
> Martijn Schmidt
> 
> [1] http://noc.i3d.net/bgp-communities/ <http://noc.i3d.net/bgp-communities/> - we need to update this document to account for some changes in routing policy / upstream providers, but the general idea is still the same.
> [2] https://onestep.net/communities/as3549/ <https://onestep.net/communities/as3549/> - as you can see in this operator's real-world example, the former GlobalCrossing network used a format like 3549:8122 to prepend twice on peerings with AS2914. However, 2914 does not show up in the action community tag 3549:8122 and that could be quite confusing for an end user such as myself.
> [3] https://www.us.ntt.net/support/policy/routing.cfm <https://www.us.ntt.net/support/policy/routing.cfm>
> [4] http://as286.net/AS286-erBH.html <http://as286.net/AS286-erBH.html>
> 
>> Many times folks on this list quoted RFC1997 which allows 16:16.  First two octets were usually ASN second two octets usually agreed semantics. 
>> 
>> Example:  <https://onestep.net/communities/as3356/>https://onestep.net/communities/as3356/ <https://onestep.net/communities/as3356/>
>> 
>> No doubt that 32:32:32 would be nice to have, but there is difference between nice to have and absolute must have - especially considering the requested urgency. 
>> 
>> Please do not interpret this as any attempt against Large Comms ... it is just a question how Large Comms are to serve the same as RFC1997 except accommodating 4 octet ASNs ?
>> 
>> Thx,
>> R.
>> 
>> On Thu, Sep 15, 2016 at 1:36 PM, Nick Hilliard <[email-masked] <mailto:[email-masked]>> wrote:
>> Robert Raszuk wrote:

---

From: Andrew Lange <[email-masked]>
Date: Tue, 20 Sep 2016 16:11:22 -0700

Nick,

It was - see 2002 flexcomms draft.  After I stopped being a customer, other operators were apathetic, as they saw it as a problem for the future.  The future is now.

Andrew

> On Sep 15, 2016, at 8:12 AM, Nick Hilliard <[email-masked]> wrote:
> 
> Robert Raszuk wrote:

---

From: Andrew Lange <[email-masked]>
Date: Tue, 20 Sep 2016 16:11:51 -0700

Sander,

Have you looked at the sort of magic-number mangling we do today?  It’s a 20-year old hack, that no one except a handful of backbone operators understand!  That is a LOT of “cleverness” that we expect people to grok at first glance.  It’s FAR from simple.  

Andrew

> On Sep 18, 2016, at 9:29 AM, Sander Steffann <[email-masked]> wrote:
> 
> Hi Randy,
>

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 20 Sep 2016 19:17:46 -0400

> Have you looked at the sort of magic-number mangling we do today?  It’s a 20-year old hack, that no one except a handful of backbone operators understand! That is a LOT of “cleverness” that we expe…

FWIW, I (speaking only as an individual WG member) agree. I hope this was evident from my previous message.

Thanks,

--John

---

From: Jared Mauch <[email-masked]>
Date: Tue, 20 Sep 2016 19:54:36 -0400

> On Sep 20, 2016, at 7:10 PM, Andrew Lange <[email-masked]> wrote:
> 
> Jared,
> 

This is certainly true and something that Joe Provo commented on as well, either here or on another mailing list (nanog) in the past week.  Communities tend to be honored where there is an existing relationship, but as with the default BGP policies of many devices, it’s something that requires some sophistication to configure.  Of the many ASNs in the DFZ, I suspect less than 1% have sophisticated policy knobs available.

> In addition, when it comes to direct-neighbor policies, on can also have contract terms and a business relationship at stake.  Two parties could agree, for example, to exchange more information tha…

This is what we see occur.  There are documented and undocumented communities that may exist.  This is why the central site that Joe Provo pointed to is critical to operators, many technical teams have difficulties getting the marketing folks to post it on the website.  Often like many things, it’s low on the todo list and just doesn’t get done in a timely fashion.

I see the role of IDR is to provide the suitable information-theory response via minimalistic protocol knobs.  Lack of complexity is often seen as a sophisticated response.

- jared

---

From: Brian Dickson <[email-masked]>
Date: Tue, 20 Sep 2016 17:28:57 -0700

TL;DR:

My $0.02 is as follows:
- The original 16:16 was very successful in the 16-bit ASN world,
SPECIFICALLY because it was very easy to implement for all vendors. Even
the dreaded TNT got that part right.
- There were (and are) many use cases supported (or supportable) by using
multiple communities, something that the IETF either is unaware of or is
being ignored
- I personally used (as engineer for AS 6453)  the base-ten expression of
the lower 16 bit values, to map functions exposed to customers in a
flexible, multi-dimensional way
- I also personally used (in AS6453) private_ASN:value in a manner not
exposed to customers, to control routing policy (e.g.
customers->peers/transit, vice versa, but no leaks) regardless of peer ASN.
Those private_ASN communities were stripped on both receipt and on
announcements.
- I also used similar communities to upstream ASNs (when we purchased
transit), and with peers (to control behavior of routes of our mutual
customers)
- There were use cases for which private ASN's as the first 16 enabled. The
understood behavior was, "This is local use only, strip this if you receive
it over a non-customer peering link".

This leads to my personal recommendation: 32:32:32 is the best suitable
analog for 16:16, and provides more than enough bits for arbitrary
semantics within the scope of <my_asn>:32:32.

To those who really really want TLVs, please do so using the following
method:
- reserve specific 32-bit ASNs from the reserved block(s)
- use those reserved ASN(s) as the T in your TLV, and do whatever you wish
on the 64 bits (TLV:32:32) left for you in the 32:32:32 space.
- There are like 200M reserved ASNs (I believe the range is 4,000,000
through 2^32-1). First come, first serve, go nuts.

I hope we can all agree that 32:32:32 will work just fine, and please get
on with adopting, approving, implementing, and enjoying this.

Brian Dickson

On Tue, Sep 20, 2016 at 4:54 PM, Jared Mauch <[email-masked]> wrote:

> > You are, of course, correct from an information-theory point of view.
> However, not everyone running BGP is as sophisticated, especially when it
> comes to BGP policy.  Expressing the intent, will in most cases, be
> respected as the desired propagation policy will have to be intentionally
> overridden.
>
> This is certainly true and something that Joe Provo commented on as well,
> either here or on another mailing list (nanog) in the past week.
> Communities tend to be honored where there is an existing relationship, but
> as with the default BGP policies of many devices, it’s something that
> requires some sophistication to configure.  Of the many ASNs in the DFZ, I
> suspect less than 1% have sophisticated policy knobs available.
>
> > In addition, when it comes to direct-neighbor policies, on can also have
> contract terms and a business relationship at stake.  Two parties could
> agree, for example, to exchange more information than they usually would on
> the condition that the other party respects their desired propagation
> behavior.
>
> This is what we see occur.  There are documented and undocumented
> communities that may exist.  This is why the central site that Joe Provo
> pointed to is critical to operators, many technical teams have difficulties
> getting the marketing folks to post it on the website.  Often like many
> things, it’s low on the todo list and just doesn’t get done in a timely
> fashion.
>
> I see the role of IDR is to provide the suitable information-theory
> response via minimalistic protocol knobs.  Lack of complexity is often seen
> as a sophisticated response.
>
> - jared
>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 21 Sep 2016 05:39:44 +0200 (CEST)

On Tue, 20 Sep 2016, Andrew Lange wrote:

> Incorrect - it has been there due to lack of operators asking their 
> vendors for a solution to the community problem. This is why this is a 
> good conversation to have.

Just because a few vendor representatives (or people who used to work at 
vendors) keep saying this, doesn't mean it's true.

---

From: Gert Doering <[email-masked]>
Date: Wed, 21 Sep 2016 09:47:01 +0200

Hi,

> It was - see 2002 flexcomms draft.  After I stopped being a customer, other operators were apathetic, as they saw it as a problem for the future.  The future is now.

This is why "large".  It has implementations.  *Running code*.

Gert Doering
        -- NetMaster

---

From: Ruediger Volk <[email-masked]>
Date: Wed, 21 Sep 2016 11:13:49 +0200

Dear colleagues,

  > we'll halt this adoption poll to await the authors response. 

I support adoption.

This addresses a serious gap in the requirements for upgrading the Internet
from 16 to 32 bit AS numbers
(hopefully being closed before another 10 years expire).

The basic requirements for the solution from my point of view are:

- avoid all unneccessary complexity

- required characteristics:
	optional transitive attribute
	stay out of defining semantics (not only due to rathole alarm)
	just provide sufficient bits to
	- identify owner of code space
		(using the IANA/RIR assigned AS is adequate in unusual
		cases; maybe there are unusual cases with solution tbd)
	- let owner define a repertoire of functions
	- and allow for parameters including at least one full ASN32

- avoid fate sharing where possible
	- in IETF process
	- in attribute codes
	(remember that attribute filters had to be made available
	to allow operators deal with "attribute of disturbance"
	that has hit us every couple of years)

I'm fine with fixed length 32+64 bits.

... and btw I already started during Berlin IETF to tell our
major vendors we want this - and discussed some ideas about requirements
for supporting policy language (so I can define and implement the
functions and semantics of our AS using large communities and 
interact with the existing definition and implementation using the
old standard communities)

Regards,
  Ruediger

Ruediger Volk

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 12:00:14 +0200

Jared,

> pointed to is critical to operators,

​If something as you admit is critical to operators to run the network
shouldn't it be maintained by IANA registry rather then sit on private web
sites ?

Thx,
R.​

> many technical teams have difficulties getting the marketing folks to post
> it on the website.  Often like many things, it’s low on the todo list and
> just doesn’t get done in a timely fashion.

---

From: Joe Provo <[email-masked]>
Date: Wed, 21 Sep 2016 06:34:48 -0400

Hey Robert,

This is way off beam, but...

On Wed, Sep 21, 2016 at 12:00:14PM +0200, Robert Raszuk wrote:
> ???If something as you admit is critical to operators to run the network
 
That's simply because the generic knob of policy signally (from IETF & 
vendors) support loosely-coupled autonomous actions.  Over time, operators 
determined a rich way to use this knob with locally-meaningful semantics 
which are autonomously defined without global co-ordination.  If a
restrictive set of codepoints were part of 199(7,8) then we would not 
have seen this.

[aside - that 'private web site' is merely documenting the plethora of
policy options created by independent parties and discovered in the wild. 
some are public, some only to customers, et al per Jared's note. one 
should not mistake the documentation of ethnographic research as any
form of proscriptive definitions :-)]

Now that a fairly common set of use cases are known, it'd be great to 
codify them in a *non-limiting way*, but that is orthogonal to correcting
the imbalance of technical options available to 16b and 32b ASN holders
today.

IMNSHO, we should be both be correcting that imbalance (where existing 
16b ASN holders have a market advantage) and folding the operational 
experience into the next generation of registered values for a TLV-based
world. We also leave the room for the next wave of innovators to create
new semantics and prevent ossification. Then everyone is happy.

Cheers!

Joe

---

From: "Dickinson, Ian" <[email-masked]>
Date: Wed, 21 Sep 2016 10:36:23 +0000

> ​If something as you admit is critical to operators to run the network shouldn't it be maintained by IANA registry rather then sit on private web sites ?

Not all policy is public, nor is all policy self-similar. This is as it should be.
(Which is part of why I like the relatively simple structure of -large)
Peers are consenting parties and can handle this as they have done for decades.
Publishing via website is simply a convenience.

FWIW,

Ian

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 21 Sep 2016 08:59:00 -0400

> On Sep 21, 2016, at 3:47 AM, Gert Doering <[email-masked]> wrote:
> 

Shipping code?

If not, the argument is a tedious distraction.

If so, you want random vendors and software-only BGP implementations to put not-even-adopted drafts into your routing system?

Fait accompli much?

-- Jeff

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 08:59:25 -0400

> Incorrect - it has been there due to lack of operators asking their vendors for a solution to the community problem. This is why this is a good conversation to have.

I’ve been talking to my vendors about this for a few years now.  The problem has been that -wide- was a mental stumbling block.  It took a few hours of discussion for vendors to realize “Oh, you’re trying to solve the rfc1997/1998 + rfc4893 use-case”.

I had thought I was being clear, apparently I wasn’t.  Much of this is because of things like extended communities that were too small, eg: 16:32.

I’ve learned from this experience that the mismatch was two sided.  A few years of dialogue finally converged to make me realize we were talking to each other but not communicating.  (I also received the message clearly from at least two vendors that -wide- was complex, likely only to be partially implemented, if at all).

It’s an outright falsehood to paint it as we’ve failed to ask our vendors.  I am willing to accept blame for the lack of communication regarding this.  The problems communicating did spill into the IDR list in recent weeks, and I think that’s caught a lot of people who were talking but not communicating with each other here.  That is certainly my lesson learned, and I hope to incorporate that lesson into further dialogue.  My phone numbers are in whois/search engines so people can contact me easily if they feel there is some disconnect/miscommunication.

- Jared

---

From: Gert Doering <[email-masked]>
Date: Wed, 21 Sep 2016 15:02:18 +0200

Hi,

> If not, the argument is a tedious distraction.

Uh, what?  IETF mantra "rough consensus and running code"?

> If so, you want random vendors and software-only BGP implementations to put not-even-adopted drafts into your routing system?

I do not understand what you are trying to tell me - "drafts with no 
implementations are better than drafts with running code, even if not
shipping yet"?

But then, I'm just an operator, and have never properly introduced to
the secret handshakes of IETF progress.

Gert Doering
        -- NetMaster

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 09:02:30 -0400

> On Sep 21, 2016, at 6:00 AM, Robert Raszuk <[email-masked]> wrote:
> 
> Jared,
> 
> This is what we see occur.  There are documented and undocumented communities that may exist.  This is why the central site that Joe Provo pointed to is critical to operators,
> 

No.  If you can’t find your provider, you have a larger problem and bringing IANA into it won’t help.

- Jared

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 09:09:02 -0400

> On Sep 21, 2016, at 8:59 AM, Jeffrey Haas <[email-masked]> wrote:
> 
> 

How do you define ‘Shipping code’?

> If not, the argument is a tedious distraction.

Or a troll :-)

> If so, you want random vendors and software-only BGP implementations to put not-even-adopted drafts into your routing system?

Actually, history says a lot of centralized/software based platforms do exist, likely in your home.  If you take something like an Arista box, you can drop on your own routing daemon, it updates the kernel which updates the hardware fib path to do the magic.

Be careful in the comparisons and equivalencies you draw.

Much of the rest of this is a discussion for over drinks about the future of hardware + software interaction in the ecosystem and the scaling properties thereof.  Not everything requires a custom ASIC these days.

- Jared

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 21 Sep 2016 09:20:47 -0400

> On Sep 21, 2016, at 9:02 AM, Gert Doering <[email-masked]> wrote:
> 
> Hi,
> 

Prototypes are great in proving a proposal is implementable.  That's the good part about the running code.

The existence of a prototype for a given proposal does not mean that proposal is inviolate.  That has been the tenor of much of this discussion and largely why this thread has been 8x the size of most on-list discussions about PDU formatting.

Thankfully, the list discussion has been moving back to technical discussion.  

I find it likely at this point that the existing proposal will move forward largely in its existing form.  But please stop treating the existence of one or more prototypes as making it a certainty that there will be no changes.  It doesn't advance the technical discussion.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 21 Sep 2016 09:34:44 -0400

> On Sep 21, 2016, at 9:09 AM, Jared Mauch <[email-masked]> wrote:
>> Shipping code?
> 

Stuff likely to leak out into the public Internet would do for this discussion.

> Actually, history says a lot of centralized/software based platforms do exist, likely in your home.  If you take something like an Arista box, you can drop on your own routing daemon, it updates th…

Absolutely.  "Consenting adults" as another list member would say.  The problem is what happens when stuff gets out of your own ecosystem and into the outside world.  

Dealing with the consequences of mis-matched specs in such shipping code isn't exactly fun.  Consult your own path attribute filtering to remember why.

-- Jeff (I work at that vendor. Not my fault, but still know the lesson.)

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 09:39:57 -0400

> On Sep 21, 2016, at 9:34 AM, Jeffrey Haas <[email-masked]> wrote:
> 

Thankfully we only have 11 devices left with this in place, and it’s decreasing on a regular basis.

We manage and remove bug workarounds based on code revision/fixes so unlike others that just do Cut+Paste SDN and have this config cruft live forever, I’d like to think we have positive knowledge of why we have each option enabled.

While harmful interactions are a problem, just blindly dropping or whitelisting attributes indefinitely is a problem for ongoing protocol development.  You and I at least agree on that.

- Jared

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 15:44:46 +0200

> bringing IANA into it won’t help.

​It is not about finding a provider -- you know that.

It is about making a common community type for say PREPEND which everybody
uses and which vendors recognize and implement already in their policy. All
you need to do is enable that with a single policy statement.

I see no need to use different community across my 4 providers to address
the same thing.

And of course there must still be room for free form local agreements as it
seems fit any two parties.

- - -

But we are again diverging ... this discussion already proven again today
that to realize basic prepend or med imposition I need to overload last 32
bits of 32:32:32 via correctly stated by Mikael "ugly hack".

"Communities have always been an ugly hack. Nobody disagrees. I'd love to
have better tools to do things."

Yet also no one provided me a solution how 32:32:32 will allow me to do any
BGP policy (prepend, med insertion, suppression etc ...) based on the v6
prefix to my EBGP CEs peer.  Sure I can color the links then overload last
32 bits with color+action+parameter ... but is this really what we want to
do here ?

Thx,
R.

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 21 Sep 2016 09:54:03 -0400

> On Sep 21, 2016, at 9:44 AM, Robert Raszuk <[email-masked]> wrote:
> 
> "Communities have always been an ugly hack. Nobody disagrees. I'd love to have better tools to do things."
> 
> Yet also no one provided me a solution how 32:32:32 will allow me to do any BGP policy (prepend, med insertion, suppression etc ...) based on the v6 prefix to my EBGP CEs peer.  Sure I can color the links then overload last 32 bits with color+action+parameter ... but is this really what we want to do here ? 

I'm amused to report that the only two times I had been asked about support for RFC 5701 was because the parties in question wanted to use the ascii expansion of the numbers positionally for the same hacks people are doing with regular communities.  They just wanted more characters to play with.

Obviously there are useful v6 things to do with that stuff, especially in VPN contexts, but that's not what I hear about.

-- Jeff

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 09:54:43 -0400

> Yet also no one provided me a solution how 32:32:32 will allow me to do any BGP policy (prepend, med insertion, suppression etc ...) based on the v6 prefix to my EBGP CEs peer.  Sure I can color th…

Actually, I think we have, it’s just you may not have been listening.

By defining something like

Actor1:Actor2:Action

The final action, can encode this Prepend, Med or other operation you desire.  Operators can come together and similar to rfc1998 write a document that says

a1:a2:666 is Blackhole
a1:a2:3 is Prepend 3x.

Right now we have everyone rolled their own, GROW or IDR could look at doing an operational practice document, but it need not be IETF work, it could be part of other BCOP efforts, a product of the RIPE routing-wg or otherwise.

I’m hoping this more clearly defines for you the thought process I share with others on this.

- Jared

---

From: Job Snijders <[email-masked]>
Date: Wed, 21 Sep 2016 16:59:30 +0300

On Wed, Sep 21, 2016 at 03:44:46PM +0200, Robert Raszuk wrote:
> Sure I can color the links then overload last 32 bits with
> color+action+parameter ... but is this really what we want to do here?

yes. and in doing so we continue to leverage 20 years, milions of
manhours which were spend to create this body of institutional knowledge
to work with 'the ugly hack'. there is great value in building upon what
operators already know and master. sometimes worse is better.

Kind regards,

Job

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 16:01:04 +0200

Jared,

Yes we could/should agree on list of common communities. In fact the
registered list I documented was an input from many operators providing
real input:

https://tools.ietf.org/html/draft-ietf-idr-registered-wide-bgp-communities-02

But the problem is they do not nicely fit into 32:32:32 fixed space we are
cooking here.

Cheers,
R.

On Wed, Sep 21, 2016 at 3:54 PM, Jared Mauch <[email-masked]> wrote:

> > Yet also no one provided me a solution how 32:32:32 will allow me to do
> any BGP policy (prepend, med insertion, suppression etc ...) based on the
> v6 prefix to my EBGP CEs peer.  Sure I can color the links then overload
> last 32 bits with color+action+parameter ... but is this really what we
> want to do here ?
> >
>
> The final action, can encode this Prepend, Med or other operation you
> desire.  Operators can come together and similar to rfc1998 write a
> document that says
>
> a1:a2:666 is Blackhole
> a1:a2:3 is Prepend 3x.
>
> Right now we have everyone rolled their own, GROW or IDR could look at
> doing an operational practice document, but it need not be IETF work, it
> could be part of other BCOP efforts, a product of the RIPE routing-wg or
> otherwise.
>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 16:04:38 +0200

>
>  sometimes worse is better.
>

​Well if this is an argument for IDR chairs to adopt a draft then I rest my
case .....​

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 10:10:12 -0400

> On Sep 21, 2016, at 10:01 AM, Robert Raszuk <[email-masked]> wrote:
> 
> Jared, 
> 
> Yes we could/should agree on list of common communities. In fact the registered list I documented was an input from many operators providing real input: 
> 
> But the problem is they do not nicely fit into 32:32:32 fixed space we are cooking here. 

I don’t think that if -large- progresses that IDR has completed, will close and BGP4+ will
be complete.

We should continue to document these cases, it may end up being a venue issue though.

- Jared

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 16:20:10 +0200

> ​>​

​Ask Jakob or other vendors ;-) ​I am sensing is that if they put -large on
the 2016/2017/2018 BGP roadmaps next time their PMs will take a look at BGP
communities topic will be at best 2025+.

Besides number of folks stated on the list that all they need is fixed
32:32:32, but when asked precise examples the truth comes out that 32:32:32
does not really cut it.

We should continue to document these cases, it may end up being a venue
> issue though.

​We may move this one to GROW as far as IETF goes. We also need to be more
active on those in Nanog/Ripe/*NOGs. ​

- - -

Honestly the way we should handle operational discussions like this one
would be to first agree in GROW WG on requirements document then come back
to IDR and discuss solutions. Now we are doing it completely ad-hoc and
backwords.

Cheers,
R.

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 10:23:59 -0400

> On Sep 21, 2016, at 10:04 AM, Robert Raszuk <[email-masked]> wrote:
> 
> ​Well if this is an argument for IDR chairs to adopt a draft then I rest my case .....​ 

I think you misread this two ways, namely intent.

Simple > complex

I get the impression you favor a well defined IETF driven structure of how networks are architected and organized.  Some people view that as complex and imposing.

opaque > well defined

Operators run the networks the way they do for the benefit of their business needs.  Why so few well defined communities years after rfc1998 was published?  I think it’s clear, there is not tooling consensus amongst the carriers nor the customer demand to push convergence at this point.  I await to be corrected or tapped on the shoulder in this regard, but the opaque nature of 1997 and of -large- where we could use that last variable as a proxy code point for something like blackhole, prepend, some new routing algebra seems feasible.

I’d take a close look at how we can do this in XR RPL, it’s quite good at tokenizing ($1,$2) out of 1997 communities and allowing an operator to write code around that.  Wish other vendors had such code, would make writing policies much easier.

- Jared

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 16:41:22 +0200

Jared,

Agreed with your last mail.

My conclusion of today's discussion leads me to believe that 4:4:4 has
number of use cases and will allow to use some of current operational
practices (which took million of manhours already and is likely to take
more millions of manhours to adjust 2:2 to 4:4:4 by rewriting current RPL
policies (or similar) and maintain those).

It is obvious just based on very simple example that 4:4:4:4 would simplify
number of policies and avoid to heavily overload last 4 octets in 3x4.

Yet in few more cases there is evidence for the need of 4:4:4:4:1 or :2 and
as indicated 4:4:4:4:16+1 would allow to use IPv6 prefix ... Likewise Jeff
mentioned his customers asking him for RFC5701 to embed text.

The easiest way to address all of the above without creating separate BGP
attribute for each is to encode them into a TLV format where TLV can be in
the front (aka common header) or at the back.

Maybe there are other ways to accommodate all of the above in efficient way
though ...

Best,
R.

On Wed, Sep 21, 2016 at 4:23 PM, Jared Mauch <[email-masked]> wrote:

> I get the impression you favor a well defined IETF driven structure of how
> networks are architected and organized.  Some people view that as complex
> and imposing.
>
> opaque > well defined
>
> Operators run the networks the way they do for the benefit of their
> business needs.  Why so few well defined communities years after rfc1998
> was published?  I think it’s clear, there is not tooling consensus amongst
> the carriers nor the customer demand to push convergence at this point.  I
> await to be corrected or tapped on the shoulder in this regard, but the
> opaque nature of 1997 and of -large- where we could use that last variable
> as a proxy code point for something like blackhole, prepend, some new
> routing algebra seems feasible.
>
> I’d take a close look at how we can do this in XR RPL, it’s quite good at
> tokenizing ($1,$2) out of 1997 communities and allowing an operator to
> write code around that.  Wish other vendors had such code, would make
> writing policies much easier.
>
> - Jared
>
>
>

---

From: Gert Doering <[email-masked]>
Date: Wed, 21 Sep 2016 16:43:00 +0200

Hi,

> you need to do is enable that with a single policy statement.

But this is not what we want.

We want a way to transport a 12-byte sequence of arbitrary bytes, attached
to a prefix, so my upstreams can offer feature sets based on these 12 bytes, 
according to our commercial needs.

Trying to define every possible aspect today that we might want to influence 
in our upstream's network in 5 years time with, and trying to encode that 
in amazingly complex code that runs on all routers even if never needed, 
this is where madness (and non-implementation) lies.

Gert Doering
        -- NetMaster

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 10:50:19 -0400

> On Sep 21, 2016, at 10:41 AM, Robert Raszuk <[email-masked]> wrote:
> 
> My conclusion of today's discussion leads me to believe that 4:4:4 has number of use cases and will allow to use some of current operational practices (which took million of manhours already and is likely to take more millions of manhours to adjust 2:2 to 4:4:4 by rewriting current RPL policies (or similar) and maintain those). 

I will be asking my friendly vendors to have a mapping option to map 1997 into -large-, so I can take my ($1,$2) and turn it to $1:$2:0 (or any other order we define) internally.

> It is obvious just based on very simple example that 4:4:4:4 would simplify number of policies and avoid to heavily overload last 4 octets in 3x4.

Yes, both the first and second words get overloaded in 1997, I expect this to be less the case in a 3x4 solution.

> Yet in few more cases there is evidence for the need of 4:4:4:4:1 or :2 and as indicated 4:4:4:4:16+1 would allow to use IPv6 prefix ... Likewise Jeff mentioned his customers asking him for RFC5701 to embed text. 

I take Jeff at his word and face value that people want these attributes attached to their routes.  I’m expecting to see a regex engine for this as well, which is other large work.

> The easiest way to address all of the above without creating separate BGP attribute for each is to encode them into a TLV format where TLV can be in the front (aka common header) or at the back. 

I’m in favor of the common header as the envelope for other attributes.  I don’t carry VPN routes, VRF targets but if others need them, lets let them (or their vendor by proxy) work with IDR/IETF to make the idea reality.

- Jared

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 16:57:12 +0200

> Yes, both the first and second words get overloaded in 1997
>

​Well frankly if you expect mapping option of overloaded both words in 2:2
​wouldn't it be easier to have even to even translation not even to odd ?

I will say something against me here but if -large is to go fwd in its own
attribute let do it: 4:4:4:4

Zero new complexity yet much more flexibility and even room for decoupled
actions and parameters.

Thx,
R.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Wed, 21 Sep 2016 15:05:23 +0000

> I will be asking my friendly vendors to have a mapping option to map 1997 into -large-, so I can take my ($1,$2) and turn it to $1:$2:0 (or any other order we define) internally.

Oh Jared. You are soooo piling it on.
Now I won't have time to code that extra header :)

Cheers,
Jakob.

---

From: Sander Steffann <[email-masked]>
Date: Wed, 21 Sep 2016 17:06:20 +0200

Hi Jared,

> I get the impression you favor a well defined IETF driven structure of how networks are architected and organized.  Some people view that as complex and imposing.

I think you hit the nail on the head. This whole discussion seems to circle around "a simple tool that operators can use (32:32:32)" vs "A method of signalling all kinds of useful behaviour in a nicely standardised way (TLV, 32:TLV, 32:32:TLV etc)".

Properties of "simple tool":
- Matches with what people are already used to (16:16)
- Simple concept
- Can be adapted in any way the operator wants
- Provides lots of flexibility for using the provided tools
- Everybody can and will use the tools in a different way
- Requires every operator to configure how to use the tool (route-maps etc)

Properties of "nicely standardised":
- Finally one way to express routing policy
- Very powerful expressions possible
- New ideas need standards and new code to be written
- Not everybody wants to offer all standardised options to everybody
- Feels complex and/or imposing to some

This is not a technical discussion anymore, this is about preference, simplicity, complexity, policy, politics, etc. I see the discussion (including myself) going in circles from "we can do so many useful things" to "it takes too long" and "it's too complex" to "please keep it simple and familiar" back to "but we can do so much better".

This circle can keep going for a long time :)  I think both sides are "right" and both sides need to respect that. I think we should do what has already been proposed several times:
- Continue with -large as-is as quickly as possible
- Don't hold -large back because we can do better. Yes, we can, but the perfect is the enemy of the good enough (for now)
- This will provide a much-needed solution quickly (running code/prototypes seem to be working)
- Let people get experience with deploying -large and start working on a best practice document on how to use it based on experience
- Continue working on a better long-term solution for standardised routing policy signalling
- Maybe -wide will provide that, maybe the best practice guidelines are enough, maybe some new proposal. But let -large continue without pulling it in with the effort to do better, because there seems to be too much controversion about that and the operators need a short-term solution. And we can have both.

Cheers,
Sander

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 21 Sep 2016 11:53:28 -0400

+grow, they got lost somewhere on the way...
(grow folk interested might want to just go look at the archive:
  https://www.ietf.org/mail-archive/web/idr/current/msg16066.html
...which is this thread...

sadly this discussion is spread over at least:
  https://www.ietf.org/mail-archive/web/idr/current/msg16102.html
  https://www.ietf.org/mail-archive/web/idr/current/msg16066.html
  https://www.ietf.org/mail-archive/web/idr/current/msg16034.html
  https://www.ietf.org/mail-archive/web/idr/current/msg16009.html
)

On Wed, Sep 21, 2016 at 10:20 AM, Robert Raszuk <[email-masked]> wrote:

> I am sensing is that if they put -large on the 2016/2017/2018 BGP roadmaps
> next time their PMs will take a look at BGP communities topic will be at
> best 2025+.

I think ops folk understand that given enough thrust (money) vendors will
do all manner of things on-demand. I think it's not productive to really
talk about this here though, what's helpful is:

1) is -large still interesting?
    (seems like lots of 'yes pls' already)
2) is there room for improvement?
    (always, almost anyway)
3) should we try to improve/change -large or move that work to -wide or
-tlv or -other ?
    (seems like today we want -large to progress, and we should gather
requirements/options for -wide or -tlv or -other-thing)

Can we get some requirements discussions/writng going in grow and once
we're not spitballing come back to IDR with a useful set of requirements
and uss-cases?

-chris

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 21 Sep 2016 13:05:25 +0100

Robert Raszuk wrote:
> shouldn't it be maintained by IANA registry rather then sit on private
> web sites ? 

if you are seriously suggesting this, then we look forward to your
Internet Drafts.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 18:02:40 +0200

But it already there ... it is even WG document ..

https://datatracker.ietf.org/doc/draft-ietf-idr-registered-wide-bgp-communities/

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 12:06:26 -0400

> On Sep 21, 2016, at 10:57 AM, Robert Raszuk <[email-masked]> wrote:
> 
>  
> ​Well frankly if you expect mapping option of overloaded both words in 2:2 ​wouldn't it be easier to have even to even translation not even to odd ? 

I’m not personally convinced there would be WG consensus on the following:

> I will say something against me here but if -large is to go fwd in its own attribute let do it: 4:4:4:4 
> 
> Zero new complexity yet much more flexibility and even room for decoupled actions and parameters. 

I’m willing to find out I’m wrong.

- Jared

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 21 Sep 2016 12:10:04 -0400

+grow

> https://datatracker.ietf.org/doc/draft-ietf-idr-registered-
i'm unclear how this is helping this dicussion.

>> if you are seriously suggesting this, then we look forward to your
I think nick/jared/gert/jzp/mikael (all operators) have all said that
registering the usecases today isn't helpful, that some of these usecases
are not public (by design) and that they have evolved over time.

I'd like to see the generalized set of these captured in a
document/reference though, which is something grow can / should do ... and
then use that as the requirements set going forward (updated as Gert/etc
find new requirements of use to their networks). Getting focused on
byzantine corner cases isn't helpful, so generally useful requirements
should be a goal. This should make the IDR side of life simpler and it
should help vendors stop telling me, mikael, jared all: "Hey you're the
first to ever ask for this, wow!" (because actually we've had to configure
these things between us... so we know we're all asking for this 'feature' -
insert feature-du-jour-discussion)

-chris

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 21 Sep 2016 12:45:27 -0400

> 	- and allow for parameters including at least one full ASN32

... and at most two full ASN32s

It's not clear to me why a restriction on the payload size should be a requirement, but evidently it is judging by both the conversation and the draft.

--John
  (with individual contributor hat on)

---

From: Matthew Ringel <[email-masked]>
Date: Wed, 21 Sep 2016 12:51:30 -0400

Presuming that the existing BGP community space is all we have to work
with right now, and that we're operating networks right now, so we
need something that works right now, it might be better to look at
what we can do as a level of abstraction right now.

Thinking about the data to be communicated, we're looking at verb/noun
sequences.

e.g.
Prepend once  (verb: prepend, noun: 1)
Export to Chile (verb: export, noun: Chile)
Deny All (verb: deny, noun: all)
etc.

Collect enough nouns and verbs, and it follows that you can have a
language, where the specific communities might be in a common database
(e.g. PeeringDB*), and networks can say that they support a (yet to be
specified) standard 10-15 verbs with some set of nouns.   Most
networks will not have a need for (say) "Export to Chile", but being
able to construct it in a standard way is the important part.**

Having sorted through more than a few BGP community documents from
large networks, constructing a set of operators that covers most cases
probably involves 5-10 networks.

Cheers
=Matt

* No affiliation beyond satisfied user.

**It necessarily follows that standard nouns and verbs need standard
definitions.  Not everyone means the same thing when they say
"Export."  That would likely be the subject of a different draft.

On Wed, Sep 21, 2016 at 12:10 PM, Christopher Morrow
<[email-masked]> wrote:
>>> > If something as you admit is critical to operators to run the network
> GROW mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/grow
>

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 13:06:15 -0400

> On Sep 21, 2016, at 10:20 AM, Robert Raszuk <[email-masked]> wrote:
> 
> ​Ask Jakob or other vendors ;-) ​I am sensing is that if they put -large on the 2016/2017/2018 BGP roadmaps next time their PMs will take a look at BGP communities topic will be at best 2025+. 
> 

the roadmap indications I’ve gotten show it will be in running code early 2017.

Anything beyond that will result in commercial negotiations which are not a topic for this list.

I have gotten one e-mail from a PM asking me why they didn’t see the request on their radar.  I’m expecting that to be an interesting dialogue as it plays out.

I’m guessing that BGP transported within QUIC will be the next proposal.</sarcasm=off>

- Jared

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 21 Sep 2016 13:15:54 -0400

On Sep 21, 2016, at 10:20 AM, Robert Raszuk <[email-masked]> wrote:
> Honestly the way we should handle operational discussions like this one would be to first agree in GROW WG on requirements document then come back to IDR and discuss solutions. Now we are doing it completely ad-hoc and backwords.

Chris and I have already talked about this and you'll have seen Chris's email about it. In short, I agree that's how it's supposed to work but as we all know, the difference between theory and practice is greater in practice than it is in theory. I hope we can learn from this experience and do better about avoiding such fire drills in the future. I'd rather not add "lessons learned" to this already overloaded thread, though.

--John

---

From: Mikael Abrahamsson <[email-masked]>
Date: Wed, 21 Sep 2016 19:20:19 +0200 (CEST)

On Wed, 21 Sep 2016, Jared Mauch wrote:

> I have gotten one e-mail from a PM asking me why they didn’t see the 
> request on their radar.  I’m expecting that to be an interesting 
> dialogue as it plays out.

This happens all the time.

Oh, you want to use *that* feature on IPv6 *too*? *baffeled look* Oh, 
you're the only one, so no.

Oh, you want to be able to use IPv6 only networks and think that all the 
MPLS control protocols should be able to run over an IPv6 only network? 
Wow, what a shocker. Oh, you're the only one, sorry, no.

Oh, you want feature parity between 16bit and 32bit ASNs? *shocked* You're 
the only one, sorry, no.

Yes, I am bitter.

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 21 Sep 2016 13:21:24 -0400

On Sep 21, 2016, at 9:20 AM, Jeffrey Haas <[email-masked]> wrote:
> But please stop treating the existence of one or more prototypes as making it a certainty that there will be no changes.  

I can say that from the chairs' point of view, there is indeed no "I have an implementation so publish it as a standard" card, although I agree that, as you say:

> Prototypes are great in proving a proposal is implementable.

No doubt!

Anyone who has been paying attention hopefully understands that a WG document belongs to the WG, not to the authors, and sometimes the WG has a consensus to change it that the author dissents with and that requires a change in the author's implementation. That's life and most of the implementors on this list have probably been through it a few times.

Thanks,

--John

---

From: Gert Doering <[email-masked]>
Date: Wed, 21 Sep 2016 19:24:13 +0200

Hi,

> able to construct it in a standard way is the important part.**

This is nice to have, and having a recommendation document that
shows operators "look, these are what people *typically* have been
doing with communities, and *that* is how to set the community
values to achieve things" would be a great help when building a new
policy framework.

Having this hard-coded in RFCs and code in routers is both far too 
complex to implement, and far too limited at the same time (what if
I only want "North Chile" but that isn't in the implementation?) - 
which is why I'm opting for the most simple approach: an opaque string
of bytes of a fixed length, displayed in 4:4:4 format, and all the
"what do I want to do with these?" flexibility and complexity goes
into the routing policy specified by the operator in their router
config.

We (AS5539) use about ~20 community values under 5539:<x> today, and
we do not see any need for much more complex policies (we're a fairly
leafish AS and only limited regional coverage) - so, I see no need to
ask my vendors for a wide list of automatic and all-encompassing features
in their automatic community / policy handling.  I want something they
can implement easily, and give me a if/match/set language to achieve
my policy, in a way that it works with 32bit ASes.

OTOH, my upstreams (2914) use a very complex and strongly regionalized 
framework ("blackhole outside the country that this route was 
received in"), and they seem to be happy with 4:4:4 as well and
not having "everything built-in and applied automatically", so the
-large draft seems to work for make small and large network operators
alike.

Gert Doering
        -- NetMaster

---

From: Matthew Ringel <[email-masked]>
Date: Wed, 21 Sep 2016 13:54:26 -0400

> into the routing policy specified by the operator in their router

My thoughts were more along the standardization of verbs, for exactly
the reason you specify.  "Export" remains a verb (and very susceptible
to being standardized), and nouns like "Chile", "North Chile",
"Santiago", or the "Las Condes district of Santiago", are nouns that
can be specified, provider by provider, and don't need to be put in an
RFC, except as examples.

We (AS 20940) work with a large amount of networks.  Knowing that
"Export" means the same thing to everyone, and I just have to look up
the right noun, means a lot less operational hassle, and a lot less
shuffling through BGP community documents.  "Do they support
Blackholing?" is a yes/no "Do they support the verb, and what nouns?"
question, as opposed to having to shuffle through 20 pages to cobble
something together that might end up being Blackholing.

In short, looking at the problem as a set of verbs/nouns, and then
defining and standardizing a "minimum supported subset" of verbs, and
recommending ways to do a taxonomy of nouns (e.g. integers for
prepending, how to state a geo region) is entirely standard-worthy.

Cheers
=Matt

[Words are mine, not my employer's.]

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 21 Sep 2016 20:12:01 +0200

> the roadmap indications I’ve gotten show it will be in running code early

​Well that may very well be for single vendor single OS or are you talking
multiple vendors multiple OSes.

The point being that the decision which will be made here will put
significant light or shade to the topic of BGP communities for years to
come.

Cheers,
R.

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 21 Sep 2016 14:19:07 -0400

> On Sep 21, 2016, at 10:50 AM, Jared Mauch <[email-masked]> wrote:
> 

The existing engine is sufficient, but not optimal, for such work.  And is being used exactly this way for existing 1997 style communities.

There had been discussion along the lines of an extended mechanism to permit the same matching on positional character tokens with underlying map actions.  Readily doable.  Just have to force it over product management mountain.

But the extended version of that discussion became why force things down to an ascii mapping - why not hex? And once hex, why not bit-ranges?  And thus leads to the ipv6 conversation which are natively represented in hex anyway.

And eventually leads down the discussion of if you're going to do all of that range mapping... why not just make it a first order of the feature.

-- Jeff (because knowing that the 3rd digit with a value of 5 is Europe is totally intuitive..)

---

From: Andrew Lange <[email-masked]>
Date: Wed, 21 Sep 2016 11:35:36 -0700

Mikael,

From the information theory point of view, who is in a better position to know this?  Vendors who track features requested by customers globally and aggregate them through a funnel into PLM, or a single vendor who may have asked their sales person or SE?  And likely asked that sales person for lots of other features too, as some requests get lost or reprioritized by the account based on what features you tell them are most important to you?

Andrew

> On Sep 20, 2016, at 8:39 PM, Mikael Abrahamsson <[email-masked]> wrote:
> 
> Just because a few vendor representatives (or people who used to work at vendors) keep saying this, doesn't mean it's true.
> 
> --

---

From: Andrew Lange <[email-masked]>
Date: Wed, 21 Sep 2016 11:36:25 -0700

Well, single implementor, whose beta code has not yet been approved or committed to any commercial release.

Andrew

> On Sep 21, 2016, at 12:47 AM, Gert Doering <[email-masked]> wrote:
> 
> Hi,
>

---

From: heasley <[email-masked]>
Date: Wed, 21 Sep 2016 18:44:49 +0000

> It's not clear to me why a restriction on the payload size should be a requirement, but evidently it is judging by both the conversation and the draft.

In my PoV, the expectation from many is that this results in
	- quicker publication as RFC - no need to bikeshed about the number
	  bits for flags, usw.
	- quicker implementation (just a size difference from rfc1997) in
	  wire representation and likely RPL and thus also less likely to have
	  bugs
	- solves the issues affecting operators WRT rfc6793 & rfc1997
	- because it is like rfc1997 in RPL and allows the removeal of weird
	  community mapping used to represent 32b ASNs, means less likelyhood
	  of operational errors

all culminating in (hopefully) publication and implementations appearing
before EOY.

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 14:48:16 -0400

> On Sep 21, 2016, at 2:12 PM, Robert Raszuk <[email-masked]> wrote:
> 
> ​Well that may very well be for single vendor single OS or are you talking multiple vendors multiple OSes.

Robert,

I’m not mistaking this list for my EBC or other regular contacts with my vendors and/or their developer teams.

I’ll let them speak for themselves thank you very much.  If you have your own implementation, I’d appreciate a pointer to it.

> The point being that the decision which will be made here will put significant light or shade to the topic of BGP communities for years to come. 

With the formal adoption complete, I will be communicating to my vendors our support of this draft.

- Jared

---

From: "i3D.net - Martijn Schmidt" <[email-masked]>
Date: Wed, 21 Sep 2016 21:05:50 +0200

Andrew,

>From an operator's perspective - when we go to $vendor and ask for a
feature that must inter-operate with other vendors' boxes but where no
official standard exists, we typically just get told "no such RFC" after
which the feature request is punted to /dev/null. I'm sure many of those
requests do not end up at your desk simply because the rest of your
organization filters out the noise.

So yeah, I'm really having a hard time believing that nobody asked for
32-bit support in BGP communities before. Another one of these
long-overdue items is having proper IPv6+MPLS support so we don't have
to resort to 6PE if we want to push IPv6 packets through an LSP, but
that's a subject for another draft.

Best regards,
Martijn

---

From: Jared Mauch <[email-masked]>
Date: Wed, 21 Sep 2016 15:12:11 -0400

> On Sep 21, 2016, at 3:05 PM, i3D.net - Martijn Schmidt <[email-masked]> wrote:
> 
> Andrew,

I was told to use extended or wide, which aren’t implemented.

eg:

(first draft in 2009):
https://tools.ietf.org/html/draft-ietf-idr-as4octet-extcomm-generic-subtype

(first draft in 2010):

https://tools.ietf.org/html/draft-ietf-idr-wide-bgp-communities

This disconnect I think has driven the operator frustration evident in this
thread.

Due to this gap, the market forces have caused us to be present here.

- Jared

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 21 Sep 2016 15:12:36 -0400

> that's a subject for another draft.

It was asked along with the 4-byte spec and its earlier I-Ds.  See RFC 4893 and its mention of extended communities in section 5.

As Jared notes, the breakdown is where it seemed to not only bypass the IETF community but apparently the entire operational community that the 2007 RFC recommends a format that only had a 2-byte local-admin field and didn't fit into some amount of desired practice to use a AS number there.

---

From: Jeffrey Haas <[email-masked]>
Date: Wed, 21 Sep 2016 15:47:40 -0400

> On Sep 21, 2016, at 3:12 PM, Jared Mauch <[email-masked]> wrote:
> 
> 
> https://tools.ietf.org/html/draft-ietf-idr-as4octet-extcomm-generic-subtype <https://tools.ietf.org/html/draft-ietf-idr-as4octet-extcomm-generic-subtype>

set policy-options community something-four members 0x0204:2514L:100

Although I find that there's a display bug on receipt.  Matches fine though.  

-- Jeff (sigh)

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 21 Sep 2016 16:23:43 -0400

On Wed, Sep 21, 2016 at 3:12 PM, Jeffrey Haas <[email-masked]> wrote:

> that the 2007 RFC recommends a format that only had a 2-byte local-admin
> field and didn't fit into some amount of desired practice to use a AS
> number there.
>
>
it's possible that requirements/usage changed as well from 2007 to 2015/16.

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 21 Sep 2016 16:53:06 -0400

> On Sep 21, 2016, at 2:44 PM, heasley <[email-masked]> wrote:
> 

Your list leads me to think you must be imagining I'm advocating for something I'm not. I actually didn't intend to advocate for a position, although if I did the only thing I think you can take from what I actually wrote is "arbitrary-sized payload".

However, I would prefer you interpret it as simply pointing out a fundamental property of the proposed spec.

I could go on at greater length, but enough electrons have been killed in service of this topic already.

--John (posting as a WG member which should be obvious from context but I suppose I should say so)

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 21 Sep 2016 17:10:59 -0400

Ha, my subconscious just divulged what bugged me about Ruediger's message enough to reply to it: In computing when we have a list of arbitrary things we usually have zero, one, or many of them. It is unusual to have exactly zero, one or two. The way Ruediger expressed his list of "requirements" led me to realize the proposal has this unusual property. I'm not saying it's bad, but it's notable, so I noted it.

$0.02 and thanks for triggering the thought process,

--John (same disclaimer)

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Wed, 21 Sep 2016 22:22:28 +0000

I thought about something like this with extended communities before I wrote my draft.
There is no order to communities. They can come in any order
and routers can change the order. Routers will remove duplicate communities.
So, you can only use any noun or verb once and, because there
is no order, you need another word type to bind the verbs and the nouns.
I quickly gave that up as way too hard.

Thanks,
Jakob.

> From: GROW [mailto:[email-masked]] On Behalf Of Matthew Ringel
> Sent: Wednesday, September 21, 2016 9:52 AM
> To: Christopher Morrow <[email-masked]>
> Cc: [email-masked]; [email-masked] [email-masked] <[email-masked]>
> Subject: Re: [GROW] [Idr] Request to adopt draft-heitz-idr-large-community - Working Group Adoption call (9/6 to
> 9/20)
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Wed, 21 Sep 2016 23:39:03 +0000

> I will be asking my friendly vendors to have a mapping option to map 1997 into -large-, so I can take my ($1,$2) and
> turn it to $1:$2:0 (or any other order we define) internally.

Should we specify rules for such mappings in the draft?
If different vendors offer different mappings, it could get quite difficult for you to write your policies to cover all your devices.

How about community (X:Y) shall map to large community (X:Y:0).
The large community shall be convertible into a community if X and Y can each fit into 16 bits.
You might want (X:0:Y) instead, but I can't see any other mapping being useful (even though it's possible).

The execution of such conversions should be configurable. I.e., unless an operator configures one community to be converted to another, they will still be treated separately.

Thanks,
Jakob.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Thu, 22 Sep 2016 08:02:04 +0200 (CEST)

On Wed, 21 Sep 2016, Andrew Lange wrote:

> From the information theory point of view, who is in a better position 
> to know this?  Vendors who track features requested by customers 
> globally and aggregate them through a funnel into PLM, or a single 
> vendor who may have asked their sales person or SE?  And likely asked 
> that sales person for lots of other features too, as some requests get 
> lost or reprioritized by the account based on what features you tell 
> them are most important to you?

You know what, us operators talk to each other. "you're the only one 
asking for this" is commonly used by account teams, even though it's 
obviously not true. I have never worked at a vendor so I don't know where 
in the chain this goes wrong. It might be that the feature request never 
makes it to PM. I have no idea. But, lots of people at vendors have never 
worked at ISPs either, so there is a lack of understanding how we're 
treated by the account teams.

---

From: Jared Mauch <[email-masked]>
Date: Thu, 22 Sep 2016 10:37:08 -0400

> On Sep 21, 2016, at 7:39 PM, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 

No, because these are opaque.  I may want $x:$y mapped to $x:$y:0 or 0:$y:$x or any of the other variants.  My needs may be different than Gert or another operator.  Giving flexibility in how one self-organizes the bitspace is a feature vs having to strictly constrain the results and ordering.

- Jared

---

From: "Simpson, Adam 1. (Nokia - CA)" <[email-masked]>
Date: Thu, 22 Sep 2016 16:16:59 +0000

OK, so the translation should be flexible. Two follow-on questions:

1. Is there a need for reverse mapping, from large to 16:16 (RFC 1997)?
2. Is it sufficient for mapping rules to be session level only? I.e. towards peer X always replace $x:$y with $x:$y:0. Or are there cases that need more complex policy logic? I.e. Only in BGP routes that match a particular prefix-list/route-filter then replace $x:$y with $x:$y:0.

-Adam

---

From: Jared Mauch <[email-masked]>
Date: Thu, 22 Sep 2016 12:53:41 -0400

> On Sep 22, 2016, at 12:16 PM, Simpson, Adam 1. (Nokia - CA) <[email-masked]> wrote:
> 

I would argue this should be done in the same way, with explicitly configured operator policy.

If I map 2914:9999 to no-export or local-as (for example) I could similarly map 2914:0:9999 to something else with a similar mapping ecosystem.

What I’ve seen (eg: I’ll keep using XR as the example) is take something and do ($1:$2) to tokenize it then I can use those variables within that sub-part of the policy.

> 2. Is it sufficient for mapping rules to be session level only? I.e. towards peer X always replace $x:$y with $x:$y:0. Or are there cases that need more complex policy logic? I.e. Only in BGP route…

I’d say if the policy is attached to a session, sure.

One place I might decide to do this is on an iBGP session or eBGP session on import/export.  Obviously anything that passes import would be propagated further in my iBGP or towards RR.

- Jared

---

From: "i3D.net - Martijn Schmidt" <[email-masked]>
Date: Thu, 22 Sep 2016 19:02:48 +0200

My $0.02 on mapping: don't do it. If I understand things correctly,
-large and RFC1997 communities can co-exist on the same prefix so there
is no reason to try and replace "old" functionality in the new standard.

On the other hand the downside of introducing a mapping scheme is that
you will have to reserve part of your new 32:32:32 space so the 16:16
can be mapped into that, which'll impact the opacity. The mapping
mechanism might also needlessly complicate the code, especially if it
must be implemented by vendors and may applied on a per-neighbor basis.

-Martijn

-- 
Met vriendelijke groet / Kindest regards,
Martijn Schmidt

i3D.net performance hosting 	
*Martijn Schmidt | Network Architect*
Email: [email-masked] <mailto://[email-masked]> | Tel:
+31 10 8900070

*i3D.net B.V. | Global Backbone AS49544*
Van Nelleweg 1, 3044 BC Rotterdam, The Netherlands
VAT: NL 8202.63.886.B01

Website
<http://www.i3d.net/?utm_source=emailsignature&utm_medium=email&utm_campaign=home>
| Case Studies
<http://www.i3d.net/partners/?utm_source=emailsignature&utm_medium=email&utm_campaign=case-studies>
| LinkedIn <https://www.linkedin.com/company/i3d-net>

---

From: heasley <[email-masked]>
Date: Thu, 22 Sep 2016 17:07:06 +0000

> I would argue this should be done in the same way, with explicitly configured operator policy.

gag.  leave the past behind.  use 1997 for well-knowns and for 16:16 - totally
avoid the complication of transition plans for well-knowns.  there is no need
to touch that.  in fact, any future well-knowns that can be 16:16 can be 1997
communities.

---

From: Gert Doering <[email-masked]>
Date: Thu, 22 Sep 2016 22:08:04 +0200

Hi,

> is no reason to try and replace "old" functionality in the new standard.

I could see reason to that - permit your customers to send you both
16:16 and 32:32:32 communities to achieve "things", and map on ingress,
so that the "achievement policy" will only have to handle a single set
of communities, not the old compatibility stuff...

As for mapping large -> 1997 - I can't see an obvious use case right now,
but that doesn't mean there is none.

Gert Doering
        -- NetMaster

---

From: John Scudder <[email-masked]>
Date: Thu, 22 Sep 2016 20:54:49 +0000

I'm trying to remind myself of why this kind of mapping is relevant to IDR, as such. Isn't it implementation specific? Is it something that requires standardization?

Thanks,

--John

> On Sep 22, 2016, at 4:08 PM, Gert Doering <[email-masked]> wrote:
> 
> Hi,
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Thu, 22 Sep 2016 23:51:38 +0000

I had the question. The responses indicate to me "no".

Thanks,
Jakob.

> From: Idr [mailto:[email-masked]] On Behalf Of John Scudder
> Sent: Thursday, September 22, 2016 1:55 PM
> To: Gert Doering <[email-masked]>
> Cc: [email-masked]
> I'm trying to remind myself of why this kind of mapping is relevant to IDR, as such. Isn't it implementation
> specific? Is it something that requires standardization?
> 
> Thanks,
> 
> --John
>

---

From: Gert Doering <[email-masked]>
Date: Fri, 23 Sep 2016 09:03:31 +0200

Hi,

> I'm trying to remind myself of why this kind of mapping is relevant to IDR, as such. Isn't it implementation specific? Is it something that requires standardization?

It came up in the discussion, people were wondering why it would make 
sense at all to do so.  So I shared perspective (IETF wants operator
feedback, I've been told).

... but I agree that it should not be done in any automatic or mandated-
by-standard way, so is out of the scope of this draft.

Gert Doering
        -- NetMaster