Subject: [Idr] Review of draft-ietf-large-community-06.txt

From: Geoff Huston <[email-masked]>
Date: Fri, 4 Nov 2016 11:14:29 +1100

I have reviewed this draft and have the following 9 suggestions. All of
these fall into the category of minor suggestions. I do not believe that
there is may major structural deficiency in this specification, and the
document is largely ready, in my view.

Here are my specific suggestions.

1. ----------------

Title: Large BGP Communities

to be consistent with previous attribute specifications, how about

"BGP Large Communities Attribute"

i.e. switch the order of "Large BGP" to "BGP Large" to be consistent with
earlier BGP Extended Communities specification in RFC4360

2. ----------------

"Network operators 
 attach BGP communities to routes to identify intrinsic properties of
 these routes."

I don't think community attributes are an intrinsic property of a route
advertisement - they are more appropriately expressed as an attached attribute 
that expresses some desired property.

how about:
   
"Network operators attach BGP communities to routes to associate
particular properties with these routes."

3. ----------------

"and may apply to an individual route or to a group of routes."

I am confused - surely the attributes in an Update BGP message apply to the
collection of routes contained in the Update. It cannot be applied to a 
single route when the update itself contains multiple routes. Why not
use the text:

"and is applied to all routes contained in a BGP Update Message where
the Communities Attribute is included."

4. ----------------

"[RFC1997] BGP Communities attributes are four-octet values split into
two two-octet words."

This is not the case - RFC1997 says: "Communities are treated as 32 bit values"
I think it would be more accurate to say:

"BGP Communities attributes are four-octet values [RFC1997]. Common use of
this attribute type splits this single 32-bit value field into two 16-bit values."

5. ----------------

"The BGP Extended Communities
attribute [RFC4360] is also unsuitable, as the protocol limit of six
octets cannot accommodate both a four-octet Global Administrator
value and a four-octet Local Administrator value, which precludes the
common operational practice of encoding a target ASN in the Local
Administrator field."

Thats a very long sentence. Try:

"The BGP Extended Communities attribute [RFC4360] is also unsuitable, as
the protocol limit of six octets for each community value cannot
accommodate both a four-octet Global Administrator value and a four-octet
Local Administrator value. This limitation precludes the common
operational practice of encoding a target ASN in the Local Administrator
field."

6. ----------------

The aggregation section contains fewer constraints then either RFC1997 or
RFC4360. It also contains a confusing non-normative “should".

For reference, RFC4360 states: By default if a range of routes is to be
aggregated and the resultant aggregates path attributes do not carry the
ATOMIC_AGGREGATE attribute, then the resulting aggregate should have an
Extended Communities path attribute that contains the set union of all the
Extended Communities from all of the aggregated routes.  The default
behavior could be overridden via local configuration, in which case
handling the Extended Communities attribute in the presence of route
aggregation becomes a matter of the local policy of the BGP speaker that
performs the aggregation.
   
Why not use this formulation? i.e.

"3. Aggregation

If a set of routes is to be aggregated and the resulting aggregate route's
path attributes do not include the ATOMIC_AGGREGATE attribute, then the
resulting aggregate route SHOULD have a Large Communities Attribute formed
from the set union of all the Large Community values from all of the
aggregated set of routes.  This behavior MAY be overridden via local
configuration, in which case handling the Large Communities attribute in
the presence of route aggregation is determined by the local policy of the
BGP speaker that performs the aggregation."

7. ----------------

4.  Canonical Representation

I am confused here - this section used an example with TWO canonical
representations:

   "For example: 64496:4294967295:2, 64496:0:0, or (64496, 111, 222)."

Conventionally, it's better to use a single canonical representation, so the
authors should pick either a colon-delimited list, or a bracketed comma+space
separated object.

8. ----------------
 
5.  Reserved Large BGP Community values

The text reads: 

"the Global Administrator values specified above could be
used if there is a future need for them."
   
This is entirely unclear. Is it referring to the values that the previous
paragraph specified as "SHOULD NOT use”? Also "could be used" is a poor
choice of words for a normative specification.

I suggest deleting completely the paragraph that contains this quote from
the document.

9. ----------------

The text reads: 

"The error handling of Large BGP Communities is as follows:

   o  A Large BGP Communities attribute SHALL be considered malformed if
      its length is not a non-zero multiple of 12."

I think the author is trying to dayL

"The error handling of Large BGP Communities is as follows:

   o  A Large Communities attribute SHALL be considered malformed if the
     length of the Large Communities value, expressed in octets, is not a
     non-zero multiple of 12."

thanks,

  Geoff

---

From: Geoff Huston <[email-masked]>
Date: Fri, 4 Nov 2016 11:17:24 +1100

I should’ve added:

In terms of the quality of the document I think this is sorely needed
and is a pivotal specification for the broad acceptance of four octet 
AS numbers. I support its timely progress to publication as a standards
track specification.

Geoff

> On 4 Nov. 2016, at 11:14 am, Geoff Huston <[email-masked]> wrote:
>

---

From: heasley <[email-masked]>
Date: Fri, 4 Nov 2016 00:47:25 +0000

is "particular" a useless word here?  I think the original text is fine, but
to consider your suggestion and follow the less-is-more mantra....

You are correct about a BGP Update msg, obviously, but that text is not
talking about an update, rather the utility of a community, which may
apply to a single route or a group of routes - some of which may not be
in a given BGP Update msg.  Yes?

Are you sure; a separator is not defined in the text.

I agree with the other suggestions; the authors may or not.

---

From: "Susan Hares" <[email-masked]>
Date: Thu, 3 Nov 2016 21:02:22 -0400

Geoff: 

Thank you for this amazing quick Review of the draft. 

Sue 

-----Original Message-----
From: Geoff Huston [mailto:[email-masked]] 
Sent: Thursday, November 3, 2016 8:14 PM
To: IETF IDR WG
Cc: [email-masked]; John G. Scudder; Susan Hares
Subject: Review of draft-ietf-large-community-06.txt

I have reviewed this draft and have the following 9 suggestions. All of these fall into the category of minor suggestions. I do not believe that there is may major structural deficiency in this specification, and the document is largely ready, in my view.

1. ----------------

i.e. switch the order of "Large BGP" to "BGP Large" to be consistent with earlier BGP Extended Communities specification in RFC4360

2. ----------------

"Network operators
 attach BGP communities to routes to identify intrinsic properties of  these routes."

I don't think community attributes are an intrinsic property of a route advertisement - they are more appropriately expressed as an attached attribute that expresses some desired property.

how about:
   
"Network operators attach BGP communities to routes to associate particular properties with these routes."

3. ----------------

I am confused - surely the attributes in an Update BGP message apply to the collection of routes contained in the Update. It cannot be applied to a single route when the update itself contains multiple routes. Why not use the text:

"and is applied to all routes contained in a BGP Update Message where the Communities Attribute is included."

4. ----------------

"[RFC1997] BGP Communities attributes are four-octet values split into two two-octet words."

"BGP Communities attributes are four-octet values [RFC1997]. Common use of this attribute type splits this single 32-bit value field into two 16-bit values."

5. ----------------

"The BGP Extended Communities
attribute [RFC4360] is also unsuitable, as the protocol limit of six octets cannot accommodate both a four-octet Global Administrator value and a four-octet Local Administrator value, which precludes the common operational practice of encoding a target ASN in the Local Administrator field."

"The BGP Extended Communities attribute [RFC4360] is also unsuitable, as the protocol limit of six octets for each community value cannot accommodate both a four-octet Global Administrator value and a four-octet Local Administrator value. This limitation precludes the common operational practice of encoding a target ASN in the Local Administrator field."

6. ----------------

The aggregation section contains fewer constraints then either RFC1997 or RFC4360. It also contains a confusing non-normative “should".

For reference, RFC4360 states: By default if a range of routes is to be aggregated and the resultant aggregates path attributes do not carry the ATOMIC_AGGREGATE attribute, then the resulting aggregate should have an Extended Communities path attribute that contains the set union of all the Extended Communities from all of the aggregated routes.  The default behavior could be overridden via local configuration, in which case handling the Extended Communities attribute in the presence of route aggregation becomes a matter of the local policy of the BGP speaker that performs the aggregation.

"3. Aggregation

If a set of routes is to be aggregated and the resulting aggregate route's path attributes do not include the ATOMIC_AGGREGATE attribute, then the resulting aggregate route SHOULD have a Large Communities Attribute formed from the set union of all the Large Community values from all of the aggregated set of routes.  This behavior MAY be overridden via local configuration, in which case handling the Large Communities attribute in the presence of route aggregation is determined by the local policy of the BGP speaker that performs the aggregation."

7. ----------------

Conventionally, it's better to use a single canonical representation, so the authors should pick either a colon-delimited list, or a bracketed comma+space separated object.

8. ----------------

The text reads: 

"the Global Administrator values specified above could be used if there is a future need for them."
   
This is entirely unclear. Is it referring to the values that the previous paragraph specified as "SHOULD NOT use”? Also "could be used" is a poor choice of words for a normative specification.

I suggest deleting completely the paragraph that contains this quote from the document.

9. ----------------

The text reads: 

thanks,

  Geoff

---

From: Geoff Huston <[email-masked]>
Date: Fri, 4 Nov 2016 12:18:20 +1100

> On 4 Nov. 2016, at 11:47 am, heasley <[email-masked]> wrote:
> 
> to consider your suggestion and follow the less-is-more mantra….
> 

I was taking exception to the word “intrinsic”

The route attributes attach properties to the route - they are not expressing
an “intrinsic” property of the route.

I suggested “particular” to propose that the properties expressed with 
communities are an association added by the BGP speaker.

But less is more could just drop the adjective completely and I’d be equally
happy!

Geoff

---

From: Geoff Huston <[email-masked]>
Date: Fri, 4 Nov 2016 12:21:12 +1100

> in a given BGP Update msg.  Yes?

So there is the mechanism of application (BGP) and the semantics of application.

It seemed to me that the text was implicitly referring to the mechanism
of application and gave the impression that a community value could be selectively
applied to only some routes contained in a single BGP update.

If the text is read as a semantic specification, then yes the same community
value can be applied to any number of routes.

Either way, the text could do with some small clarification, imho.

Geoff

---

From: Geoff Huston <[email-masked]>
Date: Fri, 4 Nov 2016 12:25:08 +1100

> Are you sure; a separator is not defined in the text.

If the “Canonical Representation” is to represent the Large Communities Attributes value 
as a sequence of triplets, where the triplet is three unsigned decimal values, but not to
define the representation of the delimiter between the elements of the triplet, nor the delineator
between successive triplets, then I think that this is an incomplete canonical
representation.

cheers,

  Geoff

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 4 Nov 2016 03:54:00 +0000

Thanks Geoff, I'll fix them.

One thing about the aggregation.
I first had the condition "do not include the ATOMIC_AGGREGATE".
Then someone commented on the list that ATOMIC_AGGREGATE was dead.
I reviewed the function of ATOMIC_AGGREGATE and what our code does.
ATOMIC_AGGREGATE means that information was lost by aggregation.
In our code (and other code that I have worked on), when we
aggregate, we always add the ATOMIC_AGGREGATE attribute.
There is no case when we aggregate without adding the ATOMIC_AGGREGATE
attribute. Being of no consequence, I removed that part of the text.
I mean, the text tells you how to aggregate if the ATOMIC_AGGREGATE
is absent, but says nothing about how to aggregate if it's present.

Personally, I'm fine either way. It will not change any code I write.

Jakob.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 4 Nov 2016 04:05:53 +0000

To explain this one, it was originally "Textual Representation"
and it was with colons only. Then we discovered that Bird uses
commas as a separator. Since that does not degrade the utility,
we allowed it. The real point is that it has to be exactly
3 positive decimal integers. If some implementations only offered hexadecimal
or used 6 int16's then it would become very difficult for ISPs
to communicate community settings to customers.
I can change it to a single representation and detail the allowed
deviations from it.

Jakob.

---

From: Job Snijders <[email-masked]>
Date: Fri, 4 Nov 2016 08:56:14 +0100

Dear Geoff,

Thank you for your time and review.

On Fri, Nov 04, 2016 at 12:25:08PM +1100, Geoff Huston wrote:
> >> Conventionally, it's better to use a single canonical
> >> representation, so the authors should pick either a colon-delimited
> >> list, or a bracketed comma+space separated object.
> > 
> If the “Canonical Representation” is to represent the Large
> Communities Attributes value as a sequence of triplets, where the
> triplet is three unsigned decimal values, but not to define the
> representation of the delimiter between the elements of the triplet,
> nor the delineator between successive triplets, then I think that this
> is an incomplete canonical representation.

The working group has debated long and furiously on this element of the
text. In the end we ended up with the text without specifying a
separator because the separator isn't what is important at all.

The 'Canonical Representation' section came to be because we want to
prevent a 'asdot vs asplain'-style time waste. It should be clear that
each Large Community is composed of three fields, separated by
something. If you read network documentation from another party and they
reference "111:333:555" it should be clear it is a Large community; even
if your router makes you type in "(111, 333, 555)". Or maybe your router
solely consumes XML or JSON as configuration: [111, 333, 555].

It was a conscious choice to make the representation liberal enough that
all implementors currently at the table felt comfortable with it, and
that the operators at the table also have guidance on what the large
community will look like in communication.

Given the above context, do you have a suggestion other then "pick eiher
a colon-delimited or bracketed"? My personal preference would be to
keep the original text.

Kind regards,

Job

---

From: Geoff Huston <[email-masked]>
Date: Fri, 4 Nov 2016 19:59:57 +1100

> On 4 Nov. 2016, at 2:54 pm, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 

I just noted that RFC1997 and RFC4360 had these constraints.

It seems strange to me that an implementation would handle aggregation differently, treating communities and extended communities one way and large communities in a subtly different manner.

Frankly I would prefer to see a consistent treatment of communities in the case of aggregation, and reproducxing the RFC4360 text kinda makes that clear (at least to me)

Omitting it invites different handling and that would be not good

Geoff

---

From: Geoff Huston <[email-masked]>
Date: Fri, 4 Nov 2016 20:04:47 +1100

> On 4 Nov. 2016, at 3:05 pm, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 

if you make a canonical a SHOULD not a MUST then you can permit variation
without breaking the standard.

So what you are saying is that the canonical representation of a single Large Community value
is three unsigned decimal integer values, separated by a ‘:’ (colon) character, representing 
the value as a triplet of unsigned 32-bit integer values. Implementations SHOULD accept this
representation as a valid form of representation of the value of a Large Community.

---

From: Geoff Huston <[email-masked]>
Date: Fri, 4 Nov 2016 20:10:23 +1100

> On 4 Nov. 2016, at 6:56 pm, Job Snijders <[email-masked]> wrote:
> 
> Dear Geoff,
> 

I think you make a clear SHOULD support for a particular representation of the
triplet, which makes other representations still ok, but there is a clear preference
for interoperability that says “clearly state a preferred format for representating
these values".

My previous response to Jakob proposes some candidate text.

regards,

   Geoff

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 4 Nov 2016 06:31:42 -0400

> On Nov 4, 2016, at 5:10 AM, Geoff Huston <[email-masked]> wrote:
> 
> 

I believe the intent of saying it's an ordered tuple of Global Admin, Local Admin 1, Local Admin 2 is sufficient.

If you want to define delimiters, go write a yang module.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 4 Nov 2016 06:34:27 -0400

> On Nov 4, 2016, at 4:59 AM, Geoff Huston <[email-masked]> wrote:
> 

The relevant point from the thread is that the atomic-aggregate attribute is largely protocol noise.  It's a vestigial organ from the BGP-3 to BGP-4 transition, and a poorly specified one at that.  We shouldn't include its use in specifications, particularly where discussing aggregation.

The only place its discussion would be relevant is as part of *de-*aggregation of a prefix.

-- Jeff

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 4 Nov 2016 13:50:32 +0000

The text in RFCs 1997 and 4360 is not a constraint:

If a range of routes is to be aggregated and the resultant aggregates
   attribute section does not carry the ATOMIC_AGGREGATE attribute, then
   the resulting aggregate should have a COMMUNITIES path attribute
   which contains all communities from all of the aggregated routes.

It does not describe how to aggregate if ATOMIC_AGGREGATE is present. The text is more constrained if ATOMIC_AGGREGATE is left out of it.

Thanks,
Jakob.

On Nov 4, 2016, at 4:24 AM, Jeffrey Haas <[email-masked]<mailto:[email-masked]>> wrote:

On Nov 4, 2016, at 4:59 AM, Geoff Huston <[email-masked]<mailto:[email-masked]>> wrote:

-- Jeff

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 4 Nov 2016 14:20:40 +0000

Geoff,

Here is my proposed revision. I made all the changes, except the ATOMIC_AGGREGATE. If after the latest discussion, you still want it, I'll make that change too.

Thanks,
Jakob.

Begin forwarded message:

>> From: Idr [mailto:[email-masked]] On Behalf Of Geoff Huston
>> Sent: Thursday, November 03, 2016 5:14 PM
>> To: IETF IDR WG <[email-masked]>
>> Cc: [email-masked]; Susan Hares <[email-masked]>
>> Subject: [Idr] Review of draft-ietf-large-community-06.txt
>> 
>> _______________________________________________
>> Idr mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/idr

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 4 Nov 2016 12:38:56 -0400

Jeff: 

Should atomic-aggregate be deemed "historical" at this point?  This can
happen in parallel to this work. 

Sue 

From: Idr [mailto:[email-masked]] On Behalf Of Jeffrey Haas
Sent: Friday, November 4, 2016 6:34 AM
To: Geoff Huston
Cc: IETF IDR WG; Sue Hares; [email-masked]
Subject: Re: [Idr] Review of draft-ietf-large-community-06.txt

It seems strange to me that an implementation would handle aggregation
differently, treating communities and extended communities one way and large
communities in a subtly different manner.

Frankly I would prefer to see a consistent treatment of communities in the
case of aggregation, and reproducxing the RFC4360 text kinda makes that
clear (at least to me)

The relevant point from the thread is that the atomic-aggregate attribute is
largely protocol noise.  It's a vestigial organ from the BGP-3 to BGP-4
transition, and a poorly specified one at that.  We shouldn't include its
use in specifications, particularly where discussing aggregation.

The only place its discussion would be relevant is as part of
*de-*aggregation of a prefix.

-- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Fri, 4 Nov 2016 12:49:23 -0400

Sue,

> happen in parallel to this work. 

I'd argue, yes.

You might recall some of the reasoning as fallout to my introduction to IETF
and the resultant churn in the 1771bis work as we tried to get the feature
to make sense. :-)

While I'm happy to write such a document at some point, it's something
that's very low priority for the WG.  In the meantime, I'll just keep
pushing back on things that refer to it.

-- Jeff

---

From: Jay Borkenhagen <[email-masked]>
Date: Fri, 4 Nov 2016 12:56:46 -0400

> pushing back on things that refer to it.

I agree with making atomic-aggregate 'historical'.  In the meantime,
the -large- work should not attempt to maintain parallels with
atomic-aggregate specifications for rfc1997 communities.

Thanks.
						Jay B.

---

From: Job Snijders <[email-masked]>
Date: Fri, 4 Nov 2016 18:18:34 +0100

On Fri, Nov 04, 2016 at 08:04:47PM +1100, Geoff Huston wrote:
> >> I am confused here - this section used an example with TWO
> >> canonical representations:
> >> 
> > To explain this one, it was originally "Textual Representation" and
> > it was with colons only. Then we discovered that Bird uses commas as
> > a separator. Since that does not degrade the utility, we allowed it.
> > The real point is that it has to be exactly 3 positive decimal
> > integers. If some implementations only offered hexadecimal or used 6
> > int16's then it would become very difficult for ISPs to communicate
> > community settings to customers.  I can change it to a single
> > representation and detail the allowed deviations from it.
> 
> if you make a canonical a SHOULD not a MUST then you can permit
> variation without breaking the standard.
> 
> So what you are saying is that the canonical representation of a
> single Large Community value is three unsigned decimal integer values,
> separated by a ‘:’ (colon) character, representing the value as a
> triplet of unsigned 32-bit integer values. Implementations SHOULD
> accept this representation as a valid form of representation of the
> value of a Large Community.

It appears the word "canonical" is maybe triggering something. The key
element is that its three separate values. Nobody cares whether it is a
colon, comma or a hypen.

Does removing the word "canonical" address the raised remark?

"""
4.  Representation

   Large BGP Communities MUST be represented as three separate unsigned
   integers in decimal notation in the following order: Global
   Administrator, Local Data 1, Local Data 2.  Numbers MUST NOT contain
   leading zeros; a zero value MUST be represented with a single zero.
   For example: 64496:4294967295:2, 64496:0:0, or (64496, 111, 222).
"""

Kind regards,

Job

---

From: "Susan Hares" <[email-masked]>
Date: Fri, 4 Nov 2016 13:27:54 -0400

Jeff: 

My understanding of the process - is that you do not need to have an RFC to
deem something historical.  You simply do a WG LC on historical and then
pass it to the IESG.  

Sue 

-----Original Message-----
From: Jeffrey Haas [mailto:[email-masked]] 
Sent: Friday, November 4, 2016 12:49 PM
To: Susan Hares
Cc: 'Geoff Huston'; 'IETF IDR WG'; [email-masked]
Subject: Re: [Idr] Review of draft-ietf-large-community-06.txt

Sue,

On Fri, Nov 04, 2016 at 12:38:56PM -0400, Susan Hares wrote:
> Should atomic-aggregate be deemed "historical" at this point?  This 
> can happen in parallel to this work.

I'd argue, yes.

-- Jeff

---

From: Geoff Huston <[email-masked]>
Date: Sat, 5 Nov 2016 05:03:40 +1100

> On 4 Nov. 2016, at 9:31 pm, Jeffrey Haas <[email-masked]> wrote:
> 
> 

For me thats a bit like saying that an IPv5 address is represented as a sequence of decimal values representing 8-bit values. Its not enough to be useful without the delimiter.

I like the way we are now spending time on the one part of the draft that is truly of least relevant.

So if all this generates too much angst then drop the entire canonical representation part from the draft and let coders do what they want. Like extended communities, no doubt there will be some canonical format emerge from this process in time.

---

From: Geoff Huston <[email-masked]>
Date: Sat, 5 Nov 2016 05:07:42 +1100

> On 4 Nov. 2016, at 9:34 pm, Jeffrey Haas <[email-masked]> wrote:
> 
> 

I have to disagree with this position. I am more than happy if you want to push out a spec that updates RFC1997 and RFC4360 and drops the atomic-aggregation conditions and applies the same considerations to this document as well. But having current specs that say “do <this> with communities and extended communities and do <that> with large communities” is bonkers! We should be consistent here and not just deliberately generate inconsistencies here - that is not doing anyone a favour.

---

From: Geoff Huston <[email-masked]>
Date: Sat, 5 Nov 2016 05:11:05 +1100

> On 5 Nov. 2016, at 12:50 am, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 
> If a range of routes is to be aggregated and the resultant aggregates attribute section does not carry the ATOMIC_AGGREGATE attribute, then the resulting aggregate should have a COMMUNITIES path attribute which contains all communities from all of the aggregated routes.
> 

My suggestion was a cut and paste from RFC4360. My motivation to propose this was along the grounds of consistent treatment of BGP communities irrespective of the size of the community value. If you want to specify a standard manner for BGP to treat ALL communities under aggregation and deaggregation of routes then please do so. But trying to achieve this by specifying inconsistent behaviours between the various community types is not the way to achieve a good outcome here, imho.

---

From: Job Snijders <[email-masked]>
Date: Fri, 4 Nov 2016 19:12:59 +0100

If we consider this as Option B:

>    For example: 64496:4294967295:2, 64496:0:0, or (64496, 111, 222).

Option C:

   Large BGP Communities SHOULD be represented as three separate
   unsigned integers in decimal notation, separated by a colon, in the
   following order: Global Administrator, Local Data 1, Local Data 2.
   Numbers MUST NOT contain leading zeros; a zero value MUST be
   represented with a single zero. For example: 64496:4294967295:2
   or 64496:0:0.

Option D:

    remove the section and let the market figure it out.

Some might argue "this is a minute detail" - however I do think that the
proliferation of ways to notate extended communities have not really
helped the extended communities effort. this is a good chance and a good
place to shortcut some of that proces and provide guidance.

Option C is basically as watered down as possible. Settle on that and
move on?

Kind regards,

Job

---

From: Geoff Huston <[email-masked]>
Date: Sat, 5 Nov 2016 05:22:59 +1100

> On 5 Nov. 2016, at 4:18 am, Job Snijders <[email-masked]> wrote:
> 
> “”

change the MUST to a SHOULD and drop the example that contains two different delimiters (as the text is fully functional and the example provides no further information, but adds confusion).

I suspect that the second sentence is overly normative (!!)

If you simply said that: “The decimal notation does not use leading zeros, and a zero value is represented as a single ‘0’.” then I suspect that you are consistent with a SHOULD, and adequately convey the minimal intent you are aiming at here

regards,

  Geoff

---

From: Geoff Huston <[email-masked]>
Date: Sat, 5 Nov 2016 05:26:20 +1100

Jeff, 

I, for one, would encourage you do write this and get atomic-aggregate declared historic. It would result in an update to RFC1997 and RFC4360 and thereby make them consistent with the current text in the Large Communities draft.

Geoff

> On 5 Nov. 2016, at 4:27 am, Susan Hares <[email-masked]> wrote:
> 
> Jeff: 
>

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 4 Nov 2016 14:31:53 -0400

Geoff and Job,

I note that the WG already discussed this topic in some detail while the document was being developed and the text as it stands represents a consensus reached with knowledge of the tradeoffs. Certainly, if you can improve the readability, that's great. Or if you've identified some fundamental flaw, then we have to fix it. But short of that, I would caution against making changes that go much past the editorial.

(It does seem as though you are spiraling in to agreeing on some helpful editorial changes that stop short of going against the previous consensus, which is a relief to me.)

Thanks!

--John

---

From: Geoff Huston <[email-masked]>
Date: Sat, 5 Nov 2016 06:41:32 +1100

Jakob,

Many thanks for sending this.

I have used Word to mark up my reactions to this proposed revisions for your further review as this tool has a good track changes option - the PDF of the marked up text is attached to this note.

My suggested changes to your proposed test are:

1) change to the Introduction to alter the description of Communities to conform more exactly to the language used in RFC 1997

2) change to the considerations of RFC4360 in the Introduction. (I also note that RFC4360 calls the Global Administrator and Local Administrator parts of the community value “sub-fileds” rather than “fields”, while this draft calls these elements just “fields” )

3) the set of values is explicitly called out to be “unordered” in the Introduction

4) The description of the attribute fields is altered to be consistent with RFC4360

5) the treatment of the Global Administrator value is unclear. I have proposed removing the last two sentences and propose changing the MUST to a SHOULD, as, later in the draft in Section 6, it is noted that the Global Administrator field can be any value!

6) I have explicitly called out that the values are in any order, and allowed for two or more individual Large Communities attributes to be contained in a single BGP Update message, again in any order. If this is not your intent then you should look at this part of section 2 and tighten up the wording.

7) you have not explicitly disambiguated between duplicate instances of the same attribute and duplicate instances of the same Community value. I have offerred some text to clarify this to the level of values.

8) I have proposed changing the MUST to a SHOULD in rthe canonical representation section

9) The sentences about the use of a colon make no sense given that you have not specified a colon separator in the preceeding paragraph. You probably need to either drop the reference to the colon, or add a colon as part of the Canonical Representation. I have proposed the former.

10) - in section 6 I changed the “may” to a “MAY” - seemed like the right thing to do.

11) I added a paragraph in the Security Considerations section explicitly that the values in this attribute are not specifically protected and then pointed to BGP security (just because an ASN is used in a community attribute, that does not mean that the ASN in question attached that attribute, or that the ASN will honour that attribute. - its just a signal without any integrity protection)

12) I have added my name as a reviewer - hope that’s ok! :-)

I _assume_ that the atomic aggregator mess will be fixed up by the action of making this an historic attribute, so I have left the text on aggregation unaltered.

Regards,

  Geoff

> On 5 Nov. 2016, at 1:20 am, Jakob Heitz (jheitz) <[email-masked]> wrote:
> 
> Geoff,
> 
> <draft-ietf-idr-large-community.xml><draft-ietf-idr-large-community-07.txt><draft-ietf-idr-large-community-07-diff-from-06.html>

---

From: Geoff Huston <[email-masked]>
Date: Sat, 5 Nov 2016 06:43:30 +1100

> On 5 Nov. 2016, at 5:12 am, Job Snijders <[email-masked]> wrote:
> 

I say “YES” - thanks Job

Geoff

---

From: Job Snijders <[email-masked]>
Date: Fri, 4 Nov 2016 20:53:46 +0100

Hi Geoff, Jakob,

I'll incorporate the proposed changes into our XML and email out an
rfcdiff for easier viewing. At first glance these look like good
suggestions!

Kind regards,

Job

ps. This may be the first time I receive a patch for a file packaged as
PDF written with word.exe - what ever happened to good ole' 'ed' and
'diff' ;-)

On Sat, Nov 05, 2016 at 06:41:32AM +1100, Geoff Huston wrote:
> I have used Word to mark up my reactions to this proposed revisions
> for your further review as this tool has a good track changes option -
> the PDF of the marked up text is attached to this note.
> 
> 1) change to the Introduction to alter the description of Communities
> to conform more exactly to the language used in RFC 1997
> 
> 2) change to the considerations of RFC4360 in the Introduction. (I
> also note that RFC4360 calls the Global Administrator and Local
> Administrator parts of the community value “sub-fileds” rather than
> “fields”, while this draft calls these elements just “fields” )
> 
> 5) the treatment of the Global Administrator value is unclear. I have
> proposed removing the last two sentences and propose changing the MUST
> to a SHOULD, as, later in the draft in Section 6, it is noted that the
> Global Administrator field can be any value!
> 
> 6) I have explicitly called out that the values are in any order, and
> allowed for two or more individual Large Communities attributes to be
> contained in a single BGP Update message, again in any order. If this
> is not your intent then you should look at this part of section 2 and
> tighten up the wording.
> 
> 7) you have not explicitly disambiguated between duplicate instances
> of the same attribute and duplicate instances of the same Community
> value. I have offerred some text to clarify this to the level of
> values.
> 
> 9) The sentences about the use of a colon make no sense given that you
> have not specified a colon separator in the preceeding paragraph. You
> probably need to either drop the reference to the colon, or add a
> colon as part of the Canonical Representation. I have proposed the
> former.
> 
> 11) I added a paragraph in the Security Considerations section
> explicitly that the values in this attribute are not specifically
> protected and then pointed to BGP security (just because an ASN is
> used in a community attribute, that does not mean that the ASN in
> question attached that attribute, or that the ASN will honour that
> attribute. - its just a signal without any integrity protection)
> 
> I _assume_ that the atomic aggregator mess will be fixed up by the
> action of making this an historic attribute, so I have left the text
> on aggregation unaltered.
> 
> Regards,
> 
>   Geoff
> 
> 
>

---

From: Job Snijders <[email-masked]>
Date: Fri, 4 Nov 2016 21:16:31 +0100

Hi everyone,

Attached is the rfcdiff html file that represents the current -06 -> -07
changes with the work from Jakob and Geoff and some nits that were
picked by John Scudder.

I should note the following, in "Reserved BGP Large Community values"
the 'reasons' behind each reserved value were removed. Some in the
working group made a case that it could be good to provide a
justification what makes these numbers special.

Other then that, i feel that these changes have improved the document's
flow and readability.

Kind regards,

Job

---

From: David Farmer <[email-masked]>
Date: Fri, 4 Nov 2016 15:39:51 -0500

>    For example: 64496:4294967295:2, 64496:0:0, or (64496, 111, 222).

I won't try to speak for Geoff, but it doesn't do it for me.

I think we should just say what you've said previously as a summary of the
intent, but use normative language.

I'd propose something like the following.

   The canonical representation for Large BGP Communities is three separate
unsigned
   integers in decimal notation separated by colons in the following order:
Global
   Administrator, Local Data 1, Local Data 2.  For example:
64496:4294967295:2,
   64496:0:0,  or 64496:111:222

   Implementations SHOULD use the above canonical representation,
especially when
   representing a Large BGP Community for output.   Implementations MAY use
other
   delimiters than colons, especially for compatibility with a
configuration syntax.  However,
   Large BGP Communities MUST be represented as three separate unsigned
integers
   in decimal notation, regardless of the delimiters used. Further, the
numbers MUST NOT contain
   leading zeros, and a zero value MUST be represented with a single zero.

-- 
===============================================
David Farmer               Email:[email-masked]
Networking & Telecommunication Services
Office of Information Technology
University of Minnesota
2218 University Ave SE        Phone: 612-626-0815
Minneapolis, MN 55414-3029   Cell: 612-812-9952
===============================================

---

From: Geoff Huston <[email-masked]>
Date: Sat, 5 Nov 2016 07:45:43 +1100

> On 5 Nov. 2016, at 7:16 am, Job Snijders <[email-masked]> wrote:
> 
> Hi everyone,
> 

Thanks indeed for this - my comments are all resolved at this point in the proposed -07 version.

regards,

   Geoff

---

From: Ignas Bagdonas <[email-masked]>
Date: Fri, 4 Nov 2016 20:46:44 +0000

Hi Geoff,

On your changes 6 and 7 - RFC7606 error handling requires to interpret 
only the first path attribute of the same type, ignoring the other 
instances. Within the path attribute duplicate identical community 
values are treated as one, and the case of multiple large community path 
attributes is handled by interpreting only the first path attribute. 
This has been discussed on the list around -02 or -03 revision 
timeframe. Making a deviation for multiple path attributes would be in 
violation of 7606 here.

Ignas

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 4 Nov 2016 21:55:49 +0000

All good, except one. You added:

   There is no significance to the order
   ...
   of the BGP Large Communities attributes if two or more such
   attributes are encoded in the BGP path attribute payload of a single
   BGP Update message.

This is in conflict with RFC 7606:

       If any other attribute (whether
       recognized or unrecognized) appears more than once in an UPDATE
       message, then all the occurrences of the attribute other than the
       first one SHALL be discarded and the UPDATE message will continue
       to be processed.

I would leave out any mention of multiple attributes.
What this is meant to say is that the order of community values within
the attribute is irrelevant. I put that in, because it has caused
confusion in the past. I was thinking to add that "a router may reorder
the community values when forwarding to the next router", because 
this point was the actual source of confusion in more than one instance.

Thanks,
Jakob.

> From: Idr [mailto:[email-masked]] On Behalf Of Job Snijders
> Sent: Friday, November 04, 2016 1:17 PM
> To: Geoff Huston <[email-masked]>
> Cc: [email-masked]
> > > > Here is my proposed revision. I made all the changes, except the ATOMIC_AGGREGATE. If after the latest
> discussion, you still want it, I'll make that change too.
> > > >
> > > > Thanks,
> > > > Jakob.
> > > >
> > > >
> > > > Begin forwarded message:
> > > >

---

From: Job Snijders <[email-masked]>
Date: Fri, 4 Nov 2016 23:10:30 +0100

On Fri, Nov 04, 2016 at 09:55:49PM +0000, Jakob Heitz (jheitz) wrote:
> I would leave out any mention of multiple attributes.  What this is
> meant to say is that the order of community values within the
> attribute is irrelevant. I put that in, because it has caused
> confusion in the past. I was thinking to add that "a router may
> reorder the community values when forwarding to the next router",
> because this point was the actual source of confusion in more than one
> instance.

Good catch, Ignas provided text to fix it. Attached are rfcdiff & txt.

Can you double check if that confirms your understanding too with regard
to duplicate attributes & duplicate attribute values.

Kind regards,

Job

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Fri, 4 Nov 2016 22:44:57 +0000

There has been enough confusion about whether it's the value or the attribute.
Change this:

   Duplicate BGP Large Community values SHOULD NOT be transmitted.  A
   receiving speaker SHOULD ignore duplicate BGP Large Community values
   from a BGP UPDATE message.

to this:

   Duplicate BGP Large Community values SHOULD NOT be transmitted.  A
   receiving speaker SHOULD silently remove duplicate BGP Large Community
   values from a BGP Large Community attribute.

Thanks,
Jakob.

> From: Job Snijders [mailto:[email-masked]]
> Sent: Friday, November 04, 2016 3:11 PM
> To: Jakob Heitz (jheitz) <[email-masked]>
> Cc: Geoff Huston <[email-masked]>; [email-masked]

---

From: Job Snijders <[email-masked]>
Date: Sat, 5 Nov 2016 00:05:36 +0100

>    values from a BGP Large Community attribute.

Thanks Jakob, this is helpful. I've updated the XML in github.

Kind regards,

Job

---

From: Zhuangshunwan <[email-masked]>
Date: Sat, 5 Nov 2016 10:28:58 +0000

Hi authors,

I have read this document and it looks good to me!

One comment:

In most of the network operating systems, we must enable neighbor/peer send-community command knob in order BGP speaker include community in routes announcement. 
By default, the BGP speaker does not send/transmit community attributes. Even if the COMMUNITIES path attribute is an optional transitive attribute. This is the current reality application.

So the following case describing in draft-ietf-idr-large-community-06 maybe cannot reflect the reality:
   "Because BGP communities are optional
   transitive BGP attributes, BGP communities may be acted upon or
   otherwise used by routing policies in other Autonomous Systems (ASes)
   on the Internet."

Some customers from IDC/OTT complain that ISPs can not transmit their community attributes, because ISPs do not transmit community attributes in most cases.

Should we clearly define whether LargeCommunity should act different from (or the same with) the RFC1997's Community?

Best Regards,
Shunwan

---

From: Job Snijders <[email-masked]>
Date: Sat, 5 Nov 2016 11:35:26 +0100

> I have read this document and it looks good to me!

Thank you for the kind words!

> One comment:
> 
> In most of the network operating systems, we must enable neighbor/peer
> send-community command knob in order BGP speaker include community in
> routes announcement. 
> By default, the BGP speaker does not send/transmit community
> attributes. Even if the COMMUNITIES path attribute is an optional
> transitive attribute. This is the current reality application.

Then change your default!

>    "Because BGP communities are optional transitive BGP attributes,
>    BGP communities may be acted upon or otherwise used by routing
>    policies in other Autonomous Systems (ASes) on the Internet."
> 
> Some customers from IDC/OTT complain that ISPs can not transmit their
> community attributes, because ISPs do not transmit community
> attributes in most cases.

I find it hard to make or accept quantative statements in this context,
I know some ISP that scrub all communities, and I know some ISPs that
pass on as much communities as possible. The market will decide.

> Should we clearly define whether LargeCommunity should act different
> from (or the same with) the RFC1997's Community?

I consider it a feature that the behaviour is exactly aligned with
RFC1997 communities. Some OTT operators might desire slightly different
behaviour, but that is really a commercial dicussion outside of IETF
scope. 

Thanks for the feedback.

Kind regards,

Job

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 5 Nov 2016 18:18:36 +0100

Hi Job,

> *​​Then change your default!*
>

​Are you making a recommendation here ​that BGP implementations should now
be modified to send BGP communities to EBGP (and obviously IBGP) peers by
default ?

Do you mean that only for large communities or also for standards and
extended communities ?

I think Shunwan's point is very valid and it seems by making above
recommendation to modify the BGP implementations default you agree with it.
If so draft should reflect this as this is no longer market nor EBGP peers
agreement.

Thx,
R.

---

From: heasley <[email-masked]>
Date: Sat, 5 Nov 2016 17:42:29 +0000

> If so draft should reflect this as this is no longer market nor EBGP peers

I do not believe Job made either recommendation.  He merely said that
Shunwan as an implementer (I am assuming) could change their implementation's
default.

Job did write that it was adventageous for -large- to have similar properties
as rfc1997.  neither docs make recommendations about the feature to which
Shunwan is referring.

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 5 Nov 2016 18:55:22 +0100

Heas,

All true .. I posted my question as perhaps it really makes sense to modify
the defaults for both 1997 and large.

To share the experience from a lot of global deployments of SAFI 128
propagation of extended communities was the biggest pain for 100s customers
to keep remembering to enable send-community extended|both knob on each
session.

And as you know SAFI 128 is pretty useless without RTs so keeping such
defaults there on both IBGP and EBGP sessions was just wrong.

Here in the case of Large Communities folks clearly indicated need to use
it across multiple ASNs. And that is cool. Except you have zero control how
peer of your provider will handle it.

So today non upgraded eBGP router will propagate it. When new OS is loaded
(maybe for completely different reason) which can recognize LC attribute
unless the knob is in place it will be dropped.

Perhaps it make sense to spell this out in the draft that large communities
attribute should be propagated by default unless otherwise suppressed by
policy ?

Thx,
r.

---

From: heasley <[email-masked]>
Date: Sat, 5 Nov 2016 18:35:17 +0000

> attribute should be propagated by default unless otherwise suppressed by

rfc4360 also does not make such a recommendation about a 'send-community
extended' (in cisco-ese) feature.  As I recall, it is one vendor who started
that, though I can not recall why, and many who copy their CLI have followed
suit.  One vendor has no such knob that I know of; communities are always
sent unless removed by RPL.  And, I believe there are a few open
implementations that also send communities by default.

I do wish that implementations that have this feature would default to
enabled/send, rather than disabled/filter.

However, given the heated discussion about prescribing a canonical
representation for -large-communities, is this a subject for an IDR rfc?
Isn't it a subject for a customer-vendor meeting?  Is changing the language
about propagating recognized transitive attributes going to change these
vendor's defaults?

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 5 Nov 2016 19:57:28 +0100

​​

> ​​

​IMO it is. Reason being that RFC will be read by someone who is tasked to
implement it and unless spec clearly at least recommends​ one way we will
have a mix of behaviors.

> Isn't it a subject for a customer-vendor meeting?

Honestly customer-vendor ​meetings are for new policy knobs. I really do
not recall any EBC where customer were successful in asking vendor to
change any deployed defaults. Defaults once set during implementation
usually stay for a long time.

> Is changing the language
> ​

​Well I believe so. Notice that as much as folks likes large to be "like"
1997 it is not 100%. Even by your cisco example .. there is already
send-community standard vs extended knob. Large ​will be new addition and
in addition to "both" they will likely add also "all" keyword.

So if we want to make large nicely transitive I think it is now a good time
to add this to draft/rfc just to have a common agreed way on the expected
outcome in any implementation.

As to why we have this knob in the first place from the early days I recall
it was based on ISPs who wanted to have RFC1997 local within their AS only
by default - such that they do not need any extra policy in order not to
propagate it over eBGP. Does the same design applies to Large ?

Note that I am not suggesting large must be propagate by default. But I
think it would be good to have mandated by the spec consistency across BGP
implementations - propagate or do not propagate.

Thx,
r.

---

From: Nick Hilliard <[email-masked]>
Date: Sat, 05 Nov 2016 21:14:56 +0100

Robert Raszuk wrote:
> All true .. I posted my question as perhaps it really makes sense to
> modify the defaults for both 1997 and large. 

I concur with Brian - if you're serious about this, you should post a
new I-D.  As you suggest, it's not specifically an LC issue, so
therefore it would not be helpful to address it in the LC draft.

Nick

---

From: heasley <[email-masked]>
Date: Sun, 6 Nov 2016 05:16:47 +0000

> usually stay for a long time.

i'd be shocked if operators dont remember a few that they've been requesting
for 2 decades (sorry to pick on cisco)

	no ip proxy-arp
	no ip redirects
	no ip directed-broadcast

iirc, all 3 are the default on ios-xr.

> outcome in any implementation.

shouldnt you clarify in 4271 - that which is transitive is transitive (by
default), recognized or not.

> propagate it over eBGP. Does the same design applies to Large ?

not imo; doesnt apply to 1997 either - not in year 1997 either, again - imo.

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Sun, 6 Nov 2016 06:26:53 +0000

Shunwan,

On Nov 5, 2016, at 3:35 AM, Job Snijders <[email-masked]<mailto:[email-masked]>> wrote:

Another reason is that some ISPs use private ASNs rather than their own ASN in the first 16 bits of the community, because there is not enough leftover bits to express the action. That makes it impossible for an ISP to transmit them to another ISP, because of name space clashes. That should no longer be a problem with large communities.
See https://onestep.net/communities<https://onestep.net/communities/>/ for examples of communities. BTW, I don't recognize any Chinese ISPs on that site. Can you share some community specifications for any Chinese ISPs, please?

谢谢,
Jakob.

---

From: Zhuangshunwan <[email-masked]>
Date: Mon, 7 Nov 2016 15:31:26 +0000

发件人: Jakob Heitz (jheitz) [mailto:[email-masked]]
发送时间: 2016年11月6日 14:27
收件人: Zhuangshunwan
抄送: [email-masked]; Job Snijders
主题: Re: [Idr] Review of draft-ietf-large-community-06.txt

Shunwan,

Another reason is that some ISPs use private ASNs rather than their own ASN in the first 16 bits of the community, because there is not enough leftover bits to express the action. That makes it impossible for an ISP to transmit them to another ISP, because of name space clashes. That should no longer be a problem with large communities.
[Shunwan] Thank you for your explanation!

See https://onestep.net/communities<https://onestep.net/communities/>/ for examples of communities. BTW, I don't recognize any Chinese ISPs on that site. Can you share some community specifications for any Chinese ISPs, please?
[Shunwan] Sorry, I looked through the entire list, but no Chinese ISPs were found on this list as far as I know.

谢谢,
[Shunwan] Your Chinese is very good!

Thanks,
Shunwan

Jakob.