Subject: Re: [Idr] AD Review of draft-ietf-idr-bgp-gr-notification-13

From: "John G. Scudder" <[email-masked]>
Date: Tue, 3 Apr 2018 16:02:48 -0400

Hi WG,

As co-chair, I want to point out that the AD asked the WG for comments ("I want to ask the WG to consider..."). Since Alvaro asked more than three months ago and nobody answered, maybe nobody has anything to say -- as he points out ("[After I wrote the text above…] I found that some of the points have been discussed on the list already"), this topic was beaten to death in March 2017 (https://www.ietf.org/mail-archive/web/idr/current/msg17679.html and the following thread, specifically in the context of session culling which Alvaro brings up). Also, for reference the original WGLC all the way back in May 2014 is here: https://www.ietf.org/mail-archive/web/idr/current/msg13202.html.

However, maybe some people have just missed Alvaro's request, so if you have comments you want considered, please send them by April 10. I will follow up myself (with cu-author hat).

Thanks,

--John

> On Dec 7, 2017, at 4:11 PM, Alvaro Retana <[email-masked]> wrote:
> 
> Dear authors:
> 
> I just finished reading this document — I have some comments, please see the list below.
> 
> I understand the intent of this document: instead of resetting a BGP session when a NOTIFICATION is received, use Graceful Restart; if the session is going to *really* be reset, then use the new Hard Reset sub-code.  That makes sense to me…but, is that the only code/sub-code for which it makes sense to do a hard reset?  The NOTIFICATION has always had the “stigma” of being something bad, so much that we (idr/IETF) have even worked on ways to reduce its use (rfc7606, for example).  I want to ask the WG to consider whether other code/sub-code NOTIFICATION combinations should also result in a hard reset.  I think there are several cases, for example:
> 
> (1) rfc4486 (Subcodes for BGP Cease Notification Message) defines “Administrative Shutdown” ("a BGP speaker decides to administratively shut down its peering with a neighbor”).  It seems to me that the sender of this NOTIFICATION would not want to "follow the rules for the Receiving Speaker” (as specified in Section 4).
> 
> (2) rfc4486 also defines "Administrative Reset” ("a BGP speaker decides to administratively reset the peering with a neighbor”); no more details are provided, but that sounds like a hard reset to me.
> 
> (3) …there’s probably others...
> 
> Having said all that, I note that Section 3.1. (Sending a Hard Reset) specifies the “encapsulation” (for lack of a better word) of the real reason for the Hard Reset.  If the consensus is to go forward with that, and not call out other exceptions, then I think that the text in 3.1 should expand more on the encapsulation operation and the rationale for doing it this way, and the document should also address other recent work that recommends the use of Administrative Shutdown, for example draft-ietf-grow-bgp-session-culling (a BCP currently in the RFC Editor’s Queue).
> 
> [After I wrote the text above…] I found that some of the points have been discussed on the list already — please include some of that discussion/analysis in the document.
> 
> 
> I’ll wait until the issue above and ones marked Major (below) are addressed before starting the IETF LC.
> 
> Thanks!
> 
> Alvaro.
> 
> 
> 
> Major:
> 
> M1. Unfortunately, rfc4724 failed to setup a registry for the Restat Flags (or the Flags for Address Family), which means that anyone is able to use the bits in there (assuming the receiver looks at them, of course).  Given that there are only a few bits, and to prevent conflicts, I would really like to see a registry set up.  This document is already tagged to Update rfc4724, so it seems like a good place to establish the registries.  [If for some reason you rather not include that information here, then we can take care of it elsewhere.  IOW, this request is not a requirement.]
> 
> 
> M2. Section 4.1. (Rules for the Receiving Speaker) has me a little confused.  Are the proposed changes contingent to setting the N bit?  The text starts by saying: "As part of this extension, routes from the peer previously marked as stale MUST NOT be deleted, until and unless the optional timer…expires…”…does that mean that the timer is no longer optional?   Then you also say: “...if the Graceful Notification ("N") bit is not set in the newly received Graceful Restart Capability, no new actions are triggered on the Receiving Speaker -- in particular, a clear "N” bit does not trigger deletion of stale routes.”  If I understand rfc4724 correctly, stale routes could be deleted — the text indicates changes in the behavior even if the N bit is not set, right?  If you are Updating this section of rfc4724, what would make it crystal clear is an “OLD/NEW” notation of the text (as in, this is the OLD text…and this is the NEW text…).
> 
> 
> M3. Security Considerations:  Maybe not a security issue, but something to think about.  Section 4.1 says that “routes...previously marked as stale MUST NOT be deleted, until and unless the optional timer...expires, or unless a Hard Reset is performed.  This supersedes the “consecutive restarts” requirement…”.  Not deleting the stale routes and not making the timer mandatory could result in stale routes that live forever if an attacker manages to create consecutive restarts (by simply sending NOTIFICATIONS before EoR) — stale routes are ok in the short term, but may point in the wrong direction eventually.  Is this an issue?  I think that it would be mitigated if the timer was made mandatory (with a nice default).
> 
> 
> 
> Minor:
> 
> P1. Section 4: “...receive and send BGP NOTIFICATION messages in Graceful mode...” What is “Graceful mode”?
> 
> 
> 
> Nits:
> 
> N1. In Section 2, please indicate that the first figure corresponds to the GR Capability from rfc4724.
> 
> N2. s/subcode is defined known as/subcode is defined as
> 
> N3. s/Graceful Notification flag/Graceful Notification bit   (For consistency)
> 
> N4. The operation is obviously per-AF; maybe it’s worth saying that somewhere just for completeness.

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 3 Apr 2018 17:02:31 -0400

(with co-author hat)

Hi Alvaro,

> [After I wrote the text above…] I found that some of the points have been discussed on the list already — please include some of that discussion/analysis in the document.

I'll draft something and we can discuss.

Just FYI, the encoding in question was adopted in version -03 as a result of Chris Hall's comments during the WGLC way back in the day (https://www.ietf.org/mail-archive/web/idr/current/msg13205.html)

> M1. Unfortunately, rfc4724 failed to setup a registry for the Restat Flags (or the Flags for Address Family), which means that anyone is able to use the bits in there (assuming the receiver looks a…

I'll do this and while I'm at it, set one up for Flags for Address Family too.

> M2. Section 4.1. (Rules for the Receiving Speaker) has me a little confused.  Are the proposed changes contingent to setting the N bit?  

No.

> The text starts by saying: "As part of this extension, routes from the peer previously marked as stale MUST NOT be deleted, until and unless the optional timer…expires…”…does that mean that the timer is no longer optional?   

No, it's just that we foolishly didn't give the timer a name in the base GR spec. Here's the paragraph from RFC 4272:

   To put an upper bound on the amount of time a router retains the
   stale routes, an implementation MAY support a (configurable) timer
   that imposes this upper bound.

In practice, all implementations have this and call it something like 'stale timer' or whatever. The lack of a defined name for this thing makes it a pain to write about it, which I think is what you've bumped into here. I'll propose a replacement in OLD/NEW style.

> Then you also say: “...if the Graceful Notification ("N") bit is not set in the newly received Graceful Restart Capability, no new actions are triggered on the Receiving Speaker -- in particular, a clear "N” bit does not trigger deletion of stale routes.”  If I understand rfc4724 correctly, stale routes could be deleted — the text indicates changes in the behavior even if the N bit is not set, right?  If you are Updating this section of rfc4724, what would make it crystal clear is an “OLD/NEW” notation of the text (as in, this is the OLD text…and this is the NEW text…).

Oh ouch, I see why you were confused. I believe we meant F bit, not N bit. I'll propose a replacement in OLD/NEW style.

> M3. Security Considerations:  Maybe not a security issue, but something to think about.  Section 4.1 says that “routes...previously marked as stale MUST NOT be deleted, until and unless the optiona…

That seems like a good idea. While we're at it, we can give it a name. 

> P1. Section 4: “...receive and send BGP NOTIFICATION messages in Graceful mode...” What is “Graceful mode”?

Changed to "according to the procedures of this document".

> N1. In Section 2, please indicate that the first figure corresponds to the GR Capability from rfc4724.

Actually unless someone objects I'll remove the figure entirely. I'm not sure why we put it in. It's not like this document stands alone anyway.

> N2. s/subcode is defined known as/subcode is defined as

OK

> N3. s/Graceful Notification flag/Graceful Notification bit   (For consistency)

OK

> N4. The operation is obviously per-AF; maybe it’s worth saying that somewhere just for completeness.

I don't think I agree -- the extension is NOT per-AF, it applies to all AFs for which GR is enabled (that's why the N flag goes in the Restart Flags field and not the Flags for Address Family field). The underlying operation of GR continues to be per-AF, it's true, but that's not affected by this spec so I think it might be confusing to add words about it?

Thanks,

--John

---

From: <[email-masked]>
Date: Wed, 4 Apr 2018 08:58:45 +0000

Hi John, Alvaro, WG,

Please find below a few comments (FWIW):

- First I do support this specification to move forward.

- Second this document has passed WGLC a long time ago, is technically valid (i.e. not flawed) and has at least 2 implementations
https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-gr-notification%20implementations
So it feels like a bit harsh to make non compatible change at this time.
(OTOH if we do a WGLC once we have 2 implementations, WGLC is never a good time to make technical changes to the document. So when is the good time?)

- Third, Alvaro’s comment seems legitimate.
I retrospect, may be this mechanism could (should?) have been applied only to error codes (i.e. all but the CEASE code). This would align with the motivation indicated in the introduction section. Alternatively, this could be defined on a per sub-code basis. As noted by Alvaro, using GR for some CEASE subcodes seem to never make sense. (I would add “Peer De-configured”)

Also, from a management standpoint, the error code is now dependent on dynamic session parameters (the setting of the “N” bit in the GR open capability) which makes it harder to track.

In all cases, I would welcome section 3.1 to indicate which Error Code and Sub code are to be indicated in log, syslog, yang models, SNMP data… The ones in the “Data” field (i.e. the true reason) or the ones sent in the Notification (cease, hard reset) ? This would avoid non-interoperable operational behaviors.

So no definite answer; but answer nonetheless, as requested.

Regards,
--Bruno

From: Idr [mailto:[email-masked]] On Behalf Of John G. Scudder
Sent: Tuesday, April 03, 2018 10:03 PM
To: Alvaro Retana; idr@ietf. org
Cc: [email-masked]; [email-masked]
Subject: Re: [Idr] AD Review of draft-ietf-idr-bgp-gr-notification-13

Hi WG,

Thanks,

--John

On Dec 7, 2017, at 4:11 PM, Alvaro Retana <[email-masked]<mailto:[email-masked]>> wrote:

Dear authors:

Thanks!

Alvaro.

Major:

Minor:

Nits:

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 4 Apr 2018 11:52:14 -0400

> Oh ouch, I see why you were confused. I believe we meant F bit, not N bit. I'll propose a replacement in OLD/NEW style.

Upon further consideration, I think the text was correct -- but confusing -- as written. I'll work on clarifying but won't change N to F. I believe the intent was to say "If the new session doesn't advertise the N bit, implementors shouldn't assume any special semantics other than 'gr-notification is no longer in effect'." One might argue this was an attempt to say "don't write bugs", so one solution would be to completely remove the offending paragraph if it can't be clarified acceptably.

--John

---

From: "John G. Scudder" <[email-masked]>
Date: Thu, 5 Apr 2018 12:47:21 -0400

(co-author hat)

Hi Bruno,

Thanks for your comments. I found them quite helpful, especially your restatement of Alvaro's point.

Now that I've had a few days of thinking about the question, I do actually have one motivation to offer for the way it's currently specced:

If we didn't have the Hard Reset notification, and relied on enumerating for each notification type (code + subcode) what reset semantics it should have, then whenever a new notification type was introduced, the spec author would have to define the semantics. But, the receiving implementation wouldn't necessarily know what the desired semantics were, because of the usual problem with introduction of new capabilities. So it would have to go with a default semantics, which might be wrong. To work around this we'd have to introduce some kind of encapsulation to indicate desired semantics... and presto, we have just reinvented Hard Reset. So this is a kind of proof by contradiction that we need the existing design.

It's true that we could specify both options -- Hard Reset (necessity argued above), plus specified semantics for existing ("legacy") notification types, but I've never been a fan of having two different code paths where one will do, especially when the one is what's already implemented.

One thing this discussion does make me think, is that it might be worth adding some advice to implementors to the spec, for which notification types should always be wrapped up in a Hard Reset. 

Regards,

--John

> On Apr 4, 2018, at 4:58 AM, [email-masked] wrote:
> 
> Hi John, Alvaro, WG,
>  
> https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-gr-notification%20implementations <https://urldefense.proofpoint.com/v2/url?u=https-3A__trac.ietf.org_trac_idr_wiki_draft-2Dietf-2Didr-2Dbgp-2Dgr-2Dnotification-2520implementations&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=LyyR87durQ_2Tdcp3v7xkJS4vNZCrnDFW44Eq4OeloU&s=ydo2omG1mRwd74ILAzDnNIN_GN_fWZe_4KBqxNA8R1U&e=>
> As co-chair, I want to point out that the AD asked the WG for comments ("I want to ask the WG to consider..."). Since Alvaro asked more than three months ago and nobody answered, maybe nobody has anything to say -- as he points out ("[After I wrote the text above…] I found that some of the points have been discussed on the list already"), this topic was beaten to death in March 2017 (https://www.ietf.org/mail-archive/web/idr/current/msg17679.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_idr_current_msg17679.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=LyyR87durQ_2Tdcp3v7xkJS4vNZCrnDFW44Eq4OeloU&s=jb6RyqGP3OGGQpKD0qizvkoIIyzy9bAU-9CcZnYCOv0&e=> and the following thread, specifically in the context of session culling which Alvaro brings up). Also, for reference the original WGLC all the way back in May 2014 is here: https://www.ietf.org/mail-archive/web/idr/current/msg13202.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_idr_current_msg13202.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=LyyR87durQ_2Tdcp3v7xkJS4vNZCrnDFW44Eq4OeloU&s=UM7yPXKLgDGANq0olGiItLwXTmgw_p6JuNx0uELOltM&e=>.
>  
> On Dec 7, 2017, at 4:11 PM, Alvaro Retana <[email-masked] <mailto:[email-masked]>> wrote:
>  
> Dear authors:
>

---

From: <[email-masked]>
Date: Fri, 6 Apr 2018 08:55:01 +0000

Hi John, Job, all

Hijacking the thread with a remotely related comment.

draft-ietf-idr-bgp-gr-notification and RFC8203/draft-snijders-idr-rfc8203bis-01 seems to ignore each other and as a result seems incompatible.

1)     It’s not crystal clear whether the use of RFC8203 is allowed with draft-ietf-idr-bgp-gr-notification given that with the latter, such NOTIFICATION message would be send with Error Code “Cease” and Error Subcode “Hard Reset” which is not covered by RFC 8203:

“   If a BGP speaker decides to terminate its session with a BGP
   neighbor, and it sends a NOTIFICATION message with the Error Code
   "Cease" and Error Subcode "Administrative Shutdown" or
   "Administrative Reset" [RFC4486<https://tools.ietf.org/html/rfc4486>], it MAY include an UTF-8 encoded
   string.  ”
https://tools.ietf.org/html/draft-snijders-idr-rfc8203bis-01#section-2

So either RFC8203/draft-snijders-idr-rfc8203bis-01 needs to cover both cases. Or draft-ietf-idr-bgp-gr-notification needs to state that the Hard Reset encapsulation behave as per the encapsulated Error Code, in all protocol aspects (but X, Y).

2)     In which order is the Notification data field highjacked?

https://tools.ietf.org/html/draft-snijders-idr-rfc8203bis-01#section-2  says:

   The Cease NOTIFICATION message with a Shutdown Communication is
   encoded as below:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | Error code 6  |    Subcode    |    Length     |     ...       \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               /
   \                                                               \
   /                 ... Shutdown Communication ...                /
   \                                                               \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

https://tools.ietf.org/html/draft-ietf-idr-bgp-gr-notification-13#section-3.1 says:

3.1<https://tools.ietf.org/html/draft-ietf-idr-bgp-gr-notification-13#section-3.1>.  Sending a Hard Reset

   A Hard Reset message is used to indicate to a peer with which the
   Graceful Notification flag has been exchanged, that the session is to
   be fully terminated.

   When sending a Hard Reset, the data portion of the NOTIFICATION is
   encoded as follows:

       +--------+--------+--------
       | ErrCode| Subcode| Data
       +--------+--------+--------

Given that no TLV encoding has been introduced, if both spec are enabled, should we first encode the encapsulated Code or the shutdown communication? (I would personally assume the former but fortunately my opinion is not normative)

Thanks,
Regards,
--Bruno

Hi WG,

Thanks,

--John

Dear authors:

Thanks!

Alvaro.

Major:

Minor:

Nits:

---

From: <[email-masked]>
Date: Fri, 6 Apr 2018 08:55:07 +0000

Hi John,

Thanks for your reply.
Please see inline [Bruno]

From: John G. Scudder [mailto:[email-masked]]
Sent: Thursday, April 05, 2018 6:47 PM
To: DECRAENE Bruno IMT/OLN
Cc: Alvaro Retana; [email-masked]; [email-masked]; idr@ietf. org
Subject: Re: [Idr] AD Review of draft-ietf-idr-bgp-gr-notification-13

(co-author hat)

Hi Bruno,

[Bruno] Many thanks for the crystal clear argumentation proving that we do need a signaling.
In theory, if we were at day 1 of the spec, this signaling could either use:
a) Encapsulation (Hard Reset sub code)
b) A flag (either part of the subcode, or added in the first octet of the data field of al code/subcode
c) Dedicated subcodes (specifying the behavior of existing subcodes, and defining two (unequal) IANA ranges for the futures subcodes)

But it’s clear that at this point, we are not at day 1 since we already have 2 implementations.

b) has the advantage of having a consistent encoding for all subcodes, avoid handling two codes/subcodes in one NOTIFICATION in all aspects of the protocol, adding a flag field which may be useful in the future (but in this case, it would probably be defined independently of draft-ietf-idr-bgp-gr-notification in order to be more widely available)

c) has the advantage of avoiding the 2 code paths ( [Code, subcode], [Cease, Hard Reset, Code, subcode]), the 2 codes/subcodes per message, not changing the message encoding.

[Bruno] IMHO, the introduction of the Hard Reset encapsulation already introduces the 2 code paths ( [Code, subcode] , [Cease, Hard Reset, Code, subcode]) and the question is only about the behavior for each path.

One thing this discussion does make me think, is that it might be worth adding some advice to implementors to the spec, for which notification types should always be wrapped up in a Hard Reset.
[Bruno] If “should always” translates as “MUST” then the draft needs to also specify the error handling when the original signaling is sent. Eventually, the error handling is keeping the original behavior (“Hard Reset”) for some subcodes, which brings us back to Alvaro’s comment.
Otherwise, there is no real change on the receiver side which needs to handle both cases/signaling.

Regards,
--Bruno

Regards,

--John

On Apr 4, 2018, at 4:58 AM, [email-masked]<mailto:[email-masked]> wrote:

Hi John, Alvaro, WG,

- Second this document has passed WGLC a long time ago, is technically valid (i.e. not flawed) and has at least 2 implementations
https://trac.ietf.org/trac/idr/wiki/draft-ietf-idr-bgp-gr-notification%20implementations<https://urldefense.proofpoint.com/v2/url?u=https-3A__trac.ietf.org_trac_idr_wiki_draft-2Dietf-2Didr-2Dbgp-2Dgr-2Dnotification-2520implementations&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=LyyR87durQ_2Tdcp3v7xkJS4vNZCrnDFW44Eq4OeloU&s=ydo2omG1mRwd74ILAzDnNIN_GN_fWZe_4KBqxNA8R1U&e=>
So it feels like a bit harsh to make non compatible change at this time.
(OTOH if we do a WGLC once we have 2 implementations, WGLC is never a good time to make technical changes to the document. So when is the good time?)

Regards,
--Bruno

Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>

Hi WG,

As co-chair, I want to point out that the AD asked the WG for comments ("I want to ask the WG to consider..."). Since Alvaro asked more than three months ago and nobody answered, maybe nobody has anything to say -- as he points out ("[After I wrote the text above…] I found that some of the points have been discussed on the list already"), this topic was beaten to death in March 2017 (https://www.ietf.org/mail-archive/web/idr/current/msg17679.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_idr_current_msg17679.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=LyyR87durQ_2Tdcp3v7xkJS4vNZCrnDFW44Eq4OeloU&s=jb6RyqGP3OGGQpKD0qizvkoIIyzy9bAU-9CcZnYCOv0&e=> and the following thread, specifically in the context of session culling which Alvaro brings up). Also, for reference the original WGLC all the way back in May 2014 is here: https://www.ietf.org/mail-archive/web/idr/current/msg13202.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_idr_current_msg13202.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=LyyR87durQ_2Tdcp3v7xkJS4vNZCrnDFW44Eq4OeloU&s=UM7yPXKLgDGANq0olGiItLwXTmgw_p6JuNx0uELOltM&e=>.

Thanks,

--John

Dear authors:

Thanks!

Alvaro.

Major:

Minor:

Nits:

Thank you.

---

From: <[email-masked]>
Date: Fri, 6 Apr 2018 08:57:27 +0000

Hi John,

Sorry to turn this thread into a general call for comment while the document is post WGLC…

When:

1)     a BGP session is established (OPEN1)

2)     then GR or draft-ietf-idr-bgp-gr-notification triggers a new BGP open (OPEN2)

3)     this BGP OPEN2 triggers a Notification OPEN Message Error

4)     draft-ietf-idr-bgp-gr-notification is used to handle this Notification using GR

It’s not crystal clear to me whether the BGP speakers should behave (in term of OPEN capability/parameters) as per OPEN1 or as per OPEN2.
Indeed, OPEN2 is the latest but was erroneous and explicitly refused.
Then again, the answer may be subcode specific… (with “Unacceptable Hold Time” it’s unlikely to accept the (Hold Time) parameter(s) from OPEN2).

Thanks,
--Bruno

Hi WG,

Thanks,

--John

Dear authors:

Thanks!

Alvaro.

Major:

Minor:

Nits:

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 6 Apr 2018 09:24:39 -0400

Hi Bruno

No comments on your a/b/c options which as you point out would be more applicable if we had a clean sheet instead of existing implementations. A few other remarks below.

> On Apr 6, 2018, at 4:55 AM, [email-masked] wrote:
> 

Sorry, I meant "two code paths to achieve the same semantics".
 
> [Bruno] If “should always” translates as “MUST” then the draft needs to also specify the error handling when the original signaling is sent.

I meant "should always" in a much weaker sense, and the draft language I have in my edit buffer (hopefully) makes that clear. I think it should be an implementation choice when to send a hard reset vs. a regular one (maybe we should call the latter a "graceful" reset), but it does seem useful to offer guidance to implementors when we have something useful to say.

I'll share a proposed revision once I've finished coming up to date with the mail thread.

Thanks,

--John

---

From: Job Snijders <[email-masked]>
Date: Fri, 6 Apr 2018 13:49:26 +0000

> Hijacking the thread with a remotely related comment.

Yes, you are. You raise valid points, thank you.

What would you suggest as a solution?

Kind regards,

Job

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

---

From: <[email-masked]>
Date: Fri, 6 Apr 2018 14:13:23 +0000

Hi Job,

> From: Job Snijders [mailto:[email-masked]]  > Sent: Friday, April 06, 2018 3:49 PM

In general, any option allowing the use of both solutions works for me.

For the first point, I proposed two, which are not exclusive.
Regarding, draft-snijders-idr-rfc8203bis the easiest path seems to be

OLD:
   If a BGP speaker decides to terminate its session with a BGP
   neighbor, and it sends a NOTIFICATION message with the Error Code
   "Cease" and Error Subcode "Administrative Shutdown" or
   "Administrative Reset" [RFC4486],

NEW:
   If a BGP speaker decides to terminate its session with a BGP
   neighbor, and it sends a NOTIFICATION message with the Error Code
   "Cease" and Error Subcode "Administrative Shutdown" or
   "Administrative Reset" [RFC4486],
   eventually encapsulated in the data field of the "Hard Reset message" [draft-ietf-idr-bgp-gr-notification] 

For the second point, we may want to define a general rule. But for draft-snijders-idr-rfc8203bis the easiest path seems to be

OLD:

    0                   1                   2                   3

                                 Figure 1

   Subcode:  the Error Subcode value MUST be one of the following
      values: 2 ("Administrative Shutdown") or 4 ("Administrative
      Reset").

NEW:
   The Shutdown Communication is encoded in the data portion of the NOTIFICATION message, immediately following the Error Code and Sub Code.
 If [draft-ietf-idr-bgp-gr-notification] is used, it's encoded following the _second_ (Error Code, Sub Code) pair. The Shutdown Communication has the following format:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    Length     |     ...       \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               /
   \                                                               \
   /                 ... Shutdown Communication String...                /
   \                                                               \
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                                 Figure 1

But really, it's up to you.

Thanks,
Kind regards,
--Bruno

 > Kind regards,
 > 
 > Job
 > 
 > 
 > > draft-ietf-idr-bgp-gr-notification and RFC8203/draft-snijders-idr-rfc8203bis-01 seems to ignore
 > each other and as a result seems incompatible.
 > >
 > >
 > > 1)     It’s not crystal clear whether the use of RFC8203 is allowed with draft-ietf-idr-bgp-gr-
 > notification given that with the latter, such NOTIFICATION message would be send with Error
 > Code “Cease” and Error Subcode “Hard Reset” which is not covered by RFC 8203:
 > >
 > >
 > >
 > > So either RFC8203/draft-snijders-idr-rfc8203bis-01 needs to cover both cases. Or draft-ietf-idr-
 > bgp-gr-notification needs to state that the Hard Reset encapsulation behave as per the
 > encapsulated Error Code, in all protocol aspects (but X, Y).
 > >
 > >
 > >
 > > Given that no TLV encoding has been introduced, if both spec are enabled, should we first
 > encode the encapsulated Code or the shutdown communication? (I would personally assume the
 > former but fortunately my opinion is not normative)
 > >
 > >
 > > Thanks,
 > > Regards,
 > > --Bruno
 > >
 > > As co-chair, I want to point out that the AD asked the WG for comments ("I want to ask the WG
 > to consider..."). Since Alvaro asked more than three months ago and nobody answered, maybe
 > nobody has anything to say -- as he points out ("[After I wrote the text above…] I found that some
 > of the points have been discussed on the list already"), this topic was beaten to death in March
 > 2017 (https://www.ietf.org/mail-archive/web/idr/current/msg17679.html and the following thread,
 > specifically in the context of session culling which Alvaro brings up). Also, for reference the original
 > WGLC all the way back in May 2014 is here: https://www.ietf.org/mail-
 > archive/web/idr/current/msg13202.html.
 > >
 > > However, maybe some people have just missed Alvaro's request, so if you have comments you
 > want considered, please send them by April 10. I will follow up myself (with cu-author hat).
 > >
 > > Thanks,
 > >
 > > --John
 > >
 > > On Dec 7, 2017, at 4:11 PM, Alvaro Retana
 > <[email-masked]<mailto:[email-masked]>> wrote:
 > >
 > > Dear authors:
 > >
 > > I understand the intent of this document: instead of resetting a BGP session when a
 > NOTIFICATION is received, use Graceful Restart; if the session is going to *really* be reset, then
 > use the new Hard Reset sub-code.  That makes sense to me…but, is that the only code/sub-code
 > for which it makes sense to do a hard reset?  The NOTIFICATION has always had the “stigma” of
 > being something bad, so much that we (idr/IETF) have even worked on ways to reduce its use
 > (rfc7606, for example).  I want to ask the WG to consider whether other code/sub-code
 > NOTIFICATION combinations should also result in a hard reset.  I think there are several cases, for
 > example:
 > >
 > > (1) rfc4486 (Subcodes for BGP Cease Notification Message) defines “Administrative Shutdown”
 > ("a BGP speaker decides to administratively shut down its peering with a neighbor”).  It seems to
 > me that the sender of this NOTIFICATION would not want to "follow the rules for the Receiving
 > Speaker” (as specified in Section 4).
 > >
 > > (2) rfc4486 also defines "Administrative Reset” ("a BGP speaker decides to administratively reset
 > the peering with a neighbor”); no more details are provided, but that sounds like a hard reset to me.
 > >
 > > Having said all that, I note that Section 3.1. (Sending a Hard Reset) specifies the “encapsulation”
 > (for lack of a better word) of the real reason for the Hard Reset.  If the consensus is to go forward
 > with that, and not call out other exceptions, then I think that the text in 3.1 should expand more on
 > the encapsulation operation and the rationale for doing it this way, and the document should also
 > address other recent work that recommends the use of Administrative Shutdown, for example
 > draft-ietf-grow-bgp-session-culling (a BCP currently in the RFC Editor’s Queue).
 > >
 > > [After I wrote the text above…] I found that some of the points have been discussed on the list
 > already — please include some of that discussion/analysis in the document.
 > >
 > >
 > > M1. Unfortunately, rfc4724 failed to setup a registry for the Restat Flags (or the Flags for
 > Address Family), which means that anyone is able to use the bits in there (assuming the receiver
 > looks at them, of course).  Given that there are only a few bits, and to prevent conflicts, I would
 > really like to see a registry set up.  This document is already tagged to Update rfc4724, so it seems
 > like a good place to establish the registries.  [If for some reason you rather not include that
 > information here, then we can take care of it elsewhere.  IOW, this request is not a requirement.]
 > >
 > >
 > >
 > >
 > > M2. Section 4.1. (Rules for the Receiving Speaker) has me a little confused.  Are the proposed
 > changes contingent to setting the N bit?  The text starts by saying: "As part of this extension, routes
 > from the peer previously marked as stale MUST NOT be deleted, until and unless the optional
 > timer…expires…”…does that mean that the timer is no longer optional?   Then you also say: “...if
 > the Graceful Notification ("N") bit is not set in the newly received Graceful Restart Capability, no
 > new actions are triggered on the Receiving Speaker -- in particular, a clear "N” bit does not trigger
 > deletion of stale routes.”  If I understand rfc4724 correctly, stale routes could be deleted — the text
 > indicates changes in the behavior even if the N bit is not set, right?  If you are Updating this section
 > of rfc4724, what would make it crystal clear is an “OLD/NEW” notation of the text (as in, this is the
 > OLD text…and this is the NEW text…).
 > >
 > >
 > >
 > >
 > >
 > > M3. Security Considerations:  Maybe not a security issue, but something to think about.  Section
 > 4.1 says that “routes...previously marked as stale MUST NOT be deleted, until and unless the
 > optional timer...expires, or unless a Hard Reset is performed.  This supersedes the “consecutive
 > restarts” requirement…”.  Not deleting the stale routes and not making the timer mandatory could
 > result in stale routes that live forever if an attacker manages to create consecutive restarts (by
 > simply sending NOTIFICATIONS before EoR) — stale routes are ok in the short term, but may
 > point in the wrong direction eventually.  Is this an issue?  I think that it would be mitigated if the
 > timer was made mandatory (with a nice default).
 > >
 > >
 > >
 > >
 > >
 > > Minor:
 > >
 > >
 > ________________________________________________________________________________
 > _________________________________________
 > >

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 6 Apr 2018 11:15:50 -0400

Hi Bruno,

I've just posted a -14 which I believe addresses all comments so far (yours and Alvaro's) other than this one, for which some discussion below.

> On Apr 6, 2018, at 4:57 AM, [email-masked] wrote:
> 
> Hi John,
>  

Better now than when it gets to IETF Last Call.
 
> Then again, the answer may be subcode specific… (with “Unacceptable Hold Time” it’s unlikely to accept the (Hold Time) parameter(s) from OPEN2).

I think if the OPEN2 still exchanges the "N" bit it's already sufficiently specified, or maybe the point is moot. 

OTOH if OPEN2 doesn't exchange the "N" bit (either because it removes the GR capability entirely or because it removes the bit) then maybe there is some clarification required. Unfortunately RFC 5492 doesn't seem to say when a capability is said to have been successfully exchanged, probably because at time of writing (or at least at time of writing the predecessor) the idea of keeping around previous-session state didn't exist. 

Tentatively, my answer is to say "a capability is only considered to have been exchanged when the session carrying it transitions to ESTABLISHED state". In the situation under discussion, that wouldn't happen, so gr-notification type semantics would continue to remain in effect.

--John

>  
> Thanks,
> --Bruno
>  
> From: Idr [mailto:[email-masked] <mailto:[email-masked]>] On Behalf Of John G. Scudder
> Cc: [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>
> As co-chair, I want to point out that the AD asked the WG for comments ("I want to ask the WG to consider..."). Since Alvaro asked more than three months ago and nobody answered, maybe nobody has anything to say -- as he points out ("[After I wrote the text above…] I found that some of the points have been discussed on the list already"), this topic was beaten to death in March 2017 (https://www.ietf.org/mail-archive/web/idr/current/msg17679.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_idr_current_msg17679.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=7AX9lsHCz9FSlFK6feRmlnAwToPRMqMdbsjdQNxre98&s=Eb2l4KGMYRyULZHgUquSdC0EFBsq78qFVgQH_0svDSU&e=> and the following thread, specifically in the context of session culling which Alvaro brings up). Also, for reference the original WGLC all the way back in May 2014 is here: https://www.ietf.org/mail-archive/web/idr/current/msg13202.html <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_idr_current_msg13202.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=7AX9lsHCz9FSlFK6feRmlnAwToPRMqMdbsjdQNxre98&s=Cj0Pme9WuS9-M7h-jLQRDlyUlQT70fb2qCojXaqenYQ&e=>.
>

---

From: <[email-masked]>
Date: Tue, 10 Apr 2018 08:52:24 +0000

Hi John,

Thank you for the updated draft.
In general -14 looks good to me, but I have 2 comments below, and more inline [Bruno] in your email.

1)
In case of Hard Reset “encapsulating” another notification, I had a comment about which Error (sub)Code is to be indicated to the network operator (e.g. in Yang, SNMP, Syslog…) I’m not seeing an answer nor clarification in the draft. (I’m not sure there is a good answer, but I’d prefer a consistent answer from all implementations)

2)
Following the change introduced, I wish that the meaning of the N bit were slightly clarified: does it enable/disable the gr-notification capability for the subsequent/future event (à la BGP capability) or for the past one (à la GR "F" bit)? More below

§2:
"   If a BGP speaker which previously advertised the "N" bit opens a new
   session without advertising that bit, normal BGP Graceful Restart
   procedures documented in [RFC4724] apply."

FYI when first reading this new sentence, I have been wondering whether this was meant as aborting the gr-notification procedures underway (a la "F" bit). Then I tough probably not. If so may be :s/apply/will apply      (or apply from this Open till the next Open; or will apply for subsequent Notification). But really up to you.

But then section 4.1 is changed with the addition of
"The sentence "To deal with possible consecutive restarts, a route
   (from the peer) previously marked as stale MUST be deleted" only
   applies when the "N" bit has not been exchanged with the peer:"

Which may be understood as saying that not sending the N bit will result in aborting the gr-notification procedure.

Especially given that §4 says
"   When such a BGP speaker has received the "N" bit from its peer, and
   receives from that peer a BGP NOTIFICATION message other than a Hard
   Reset, it MUST follow the rules for the Receiving Speaker mentioned
   in Section 4.1."

While section 4.1 only explicitly covers "detects termination of the TCP session for a BGP session" which may be understood as only TCP “issues” and not the termination of the BGP session following the reception of a Notification.

From: John G. Scudder [mailto:[email-masked]]
Sent: Friday, April 06, 2018 5:16 PM
To: DECRAENE Bruno IMT/OLN
Cc: [email-masked]; [email-masked]; idr@ietf. org; Alvaro Retana
Subject: Re: [Idr] AD Review of draft-ietf-idr-bgp-gr-notification-13

Hi Bruno,

On Apr 6, 2018, at 4:57 AM, [email-masked]<mailto:[email-masked]> wrote:

Hi John,

When:

[Bruno] To be more specific, at step “4”, does the BGP speaker uses the GR parameters such as “Restart Time”, “GR AFI/SAFI” from OPEN1 or OPEN2? Same question for the “Forwarding State (F) » bits. Same question for parameters in LLGR capability.

From your answer, you seem so say that the “N” bit from OPEN2 needs to be considered and taken into account. I’d like this to be clarified.

OTOH if OPEN2 doesn't exchange the "N" bit (either because it removes the GR capability entirely or because it removes the bit) then maybe there is some clarification required. Unfortunately RFC 5492 doesn't seem to say when a capability is said to have been successfully exchanged, probably because at time of writing (or at least at time of writing the predecessor) the idea of keeping around previous-session state didn't exist.
[Bruno] Exactly. That’s why I believe the burden to clarify/specify this falls on draft-ietf-idr-bgp-gr-notification

Tentatively, my answer is to say "a capability is only considered to have been exchanged when the session carrying it transitions to ESTABLISHED state". In the situation under discussion, that wouldn't happen, so gr-notification type semantics would continue to remain in effect.
[Bruno] Any answer works for me. I particularly like your proposition being clear and very general. However, it seems to say that the BGP capability advertised in OPEN2 should not be considered. While in your above answer, you seem to take it into consideration (“if the OPEN2 still exchanges the "N"”)

Thanks,
Regards,
--Bruno

--John

Thanks,
--Bruno

Hi WG,

As co-chair, I want to point out that the AD asked the WG for comments ("I want to ask the WG to consider..."). Since Alvaro asked more than three months ago and nobody answered, maybe nobody has anything to say -- as he points out ("[After I wrote the text above…] I found that some of the points have been discussed on the list already"), this topic was beaten to death in March 2017 (https://www.ietf.org/mail-archive/web/idr/current/msg17679.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_idr_current_msg17679.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=7AX9lsHCz9FSlFK6feRmlnAwToPRMqMdbsjdQNxre98&s=Eb2l4KGMYRyULZHgUquSdC0EFBsq78qFVgQH_0svDSU&e=> and the following thread, specifically in the context of session culling which Alvaro brings up). Also, for reference the original WGLC all the way back in May 2014 is here: https://www.ietf.org/mail-archive/web/idr/current/msg13202.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_idr_current_msg13202.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=7AX9lsHCz9FSlFK6feRmlnAwToPRMqMdbsjdQNxre98&s=Cj0Pme9WuS9-M7h-jLQRDlyUlQT70fb2qCojXaqenYQ&e=>.

Thanks,

--John

Dear authors:

Thanks!

Alvaro.

Major:

Minor:

Nits:

Thank you.

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 10 Apr 2018 10:18:52 -0400

Bruno,

> In case of Hard Reset “encapsulating” another notification, I had a comment about which Error (sub)Code is to be indicated to the network operator (e.g. in Yang, SNMP, Syslog…) I’m not seeing an an…

I'm not sure there's something to be consistently done here.

I believe your point is that if you receive a hard reset, you want to make
sure the underlying reason in the tunneled code/subcode is bubbled up the
management layer so you can see the WHY of it.  E.g. max prefix exceeded.
This is a good goal.  However, the implementation issues make trying to
provide normative behavior for this difficult.  

We provide no guidance for display of notifications in syslog in any BGP spec.

The RFC 4273 MIB doesn't provide any access to the notification DATA field.

The BGP MIBv2 work does provide a human readable text string where this
could go, but the contents of this field was intentionally not specified.

The IDR yang module (stale) doesn't have notifications in it.

The openconfig BGP module covers code/subcode, but not any human or machine
readable element for the DATA field.

My employer's implementation will grab the inner code/subcode and provide
that in the UI, but appears to lose the semantics that it was a hard reset.
(Sigh. Minor bug to file.)

The above mostly is intended to agree that there's valid management
considerations to be dealt with, but I don't see a good path forward.

-- Jeff

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 10 Apr 2018 10:28:57 -0400

(as co-author and WG member)

Hi Bruno,

Discussion in line below.

> On Apr 10, 2018, at 4:52 AM, [email-masked] wrote:
> 
> Hi John,
>  

Part of my takeaway from the comments Jeff just sent is that there are limits on how much we can specify in this document since the various management layers that might be used have different semantics and constraints. How about something like this, as a new management considerations section?

"When reporting a hard reset to network management, the error code and subcode reported MUST be CEASE, Hard Reset. If the network management layer in use permits it, the information carried in the Data portion SHOULD be reported as well." 

While it would be nice to tell the management layer that it has to report the encapsulated code and subcode in a useful, structured way, I can't see that as being in-scope for this document.
 
> Which may be understood as saying that not sending the N bit will result in aborting the gr-notification procedure.

These are good points and underscore the subtlety of the problem. You might recall that we introduced the new text and structure to deal with Alvaro's objection to the previous formulation, which said something to the effect of "if the N bit goes away, you shouldn't use that as a trigger to abort a GR in progress". As you point out, the GR capability has two functions -- to say what actions should be applied to routes from the previous session (or potentially previous sessionS in the case of this draft), and what actions should be applied at the termination of the current session. Trying to cover both of these in a way both clear and precise is proving challenging.

As a practical matter, I almost don't care about whether "a route (from the peer) previously marked as stale MUST be deleted" because the difference would only come into force if there were two or more resets in quick succession, *and* the final one removed the N bit. And even then, the difference in behaviors would necessarily be transient. So this is a very edgy edge case. 

However, of course it's still desirable to be clear, which requires us to decide what exactly we want. It seems to me that a desirable way to spec and implement is that up until the GR capability has been exchanged, sans F bit, the previous F bit based semantics should apply, and after it has been exchanged, the new semantics should apply. What this comes down to is,

- Don't try to retrospectively go back and say "oh because the previous session was reset due to a holdtime expiration, and now I don't have the F bit, I'd better act as though it had been hard reset". 
- Do apply the quoted text to any already-stale detritus of previous sessions.

Perhaps a sufficient clarifying change would be:

OLD:
   If a BGP speaker which previously advertised the "N" bit opens a new
   session without advertising that bit, normal BGP Graceful Restart
   procedures documented in [RFC4724] apply."

NEW:
   If a BGP speaker which previously advertised the "N" bit opens a new
   session without advertising that bit, normal BGP Graceful Restart
   procedures documented in [RFC4724] apply once the session has 
   transitioned into ESTABLISHED state".

Unless there's objection or more discussion I'll update the draft accordingly.

> While section 4.1 only explicitly covers "detects termination of the TCP session for a BGP session" which may be understood as only TCP “issues” and not the termination of the BGP session following…

Yes, as discussed above, creativity about this clause was largely what the text is trying to prevent.

> From: John G. Scudder [mailto:[email-masked] <mailto:[email-masked]>] 
> Cc: [email-masked] <mailto:[email-masked]>; [email-masked] <mailto:[email-masked]>; idr@ietf. org; Alvaro Retana
> On Apr 6, 2018, at 4:57 AM, [email-masked] <mailto:[email-masked]> wrote:
>  
> Hi John,
>  
> From your answer, you seem so say that the “N” bit from OPEN2 needs to be considered and taken into account. 

That wasn't what I meant to say. Rather I meant that if OPEN1 carried the N bit and OPEN2 also carries the N bit, then it's immaterial whether we're considering the N bit status of OPEN1 or OPEN2 since they're the same anyway. But if pressed, I'd say the status of OPEN1 is the one that applies. Possibly the new text I propose above (that adds "once the session has transitioned into ESTABLISHED state" is sufficient?

> I’d like this to be clarified.  
>  

I hope my comments above clarify this point.

[rest trimmed]

Regards,

--John

---

From: <[email-masked]>
Date: Tue, 10 Apr 2018 15:37:54 +0000

Jeff,

Thanks for your reply.
More inline [Bruno]

> From: Jeffrey Haas [mailto:[email-masked]]  > Sent: Tuesday, April 10, 2018 4:19 PM
> 
 > Bruno,
 > 
 > > In case of Hard Reset “encapsulating” another notification, I had a comment about which Error
 > (sub)Code is to be indicated to the network operator (e.g. in Yang, SNMP, Syslog…) I’m not seeing
 > an answer nor clarification in the draft. (I’m not sure there is a good answer, but I’d prefer a
 > consistent answer from all implementations)
 > >
 > 

So let's take the RFC 4273 MIB example.

        bgpPeerLastError OBJECT-TYPE
            SYNTAX     OCTET STRING (SIZE (2))
            MAX-ACCESS read-only
            STATUS     current
            DESCRIPTION
                    "The last error code and subcode seen by this
                     peer on this connection.

My question is whether the "The last error code and subcode seen by this peer on this connection." is either:
a) the one encoded in the BGP Notification fields i.e. always (Cease, Hard Reset) whatever the original reason is. i.e. gr-notification is now masking the true reason.
b) the one encapsulated in the Data field, i.e. the original one/true reason/one which used to be advertised when gr-notification is not enabled

My reading of your email, is that "a" applies. This works for me.
Eventually the problematic of masking the true reason could be mentioned in the draft. (e.g. in a Manageability Considerations section)

 > readable element for the DATA field.
 
So a priori the same question applies.
 
 > considerations to be dealt with, but I don't see a good path forward.
 
IMHO, draft-ietf-idr-bgp-gr-notification indicating which is the preferred option (between "a" and "b") would be better than nothing.
But I can certainly live with the current draft.

Thanks
--Bruno

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 10 Apr 2018 11:49:25 -0400

Bruno,

> My reading of your email, is that "a" applies. This works for me.

Especially for implementations that do not implement this draft.  So, a
would be consistent even if less helpful.

> Eventually the problematic of masking the true reason could be mentioned in the draft. (e.g. in a Manageability Considerations section)

This may be the best thing to do.  

> But I can certainly live with the current draft.

I think John's most recent suggestion covers most of the ground with regard
to suggested normative behavior.  If the implementation could display it, it
should.  I do worry slightly that this creates precedence for other similar
logging of DATA.

-- Jeff

---

From: <[email-masked]>
Date: Tue, 10 Apr 2018 15:51:15 +0000

Hi John,

More inline [Bruno2]

From: Idr [mailto:[email-masked]] On Behalf Of John G. Scudder
Sent: Tuesday, April 10, 2018 4:29 PM
To: DECRAENE Bruno IMT/OLN
Cc: [email-masked]; idr@ietf. org; [email-masked]
Subject: Re: [Idr] AD Review of draft-ietf-idr-bgp-gr-notification-13

Hi Bruno,

On Apr 10, 2018, at 4:52 AM, [email-masked]<mailto:[email-masked]> wrote:

Hi John,

1)

[Bruno2] Perfect.
Thanks for taking my comment into account.

[Bruno2] Agreed *2.

--Bruno

2)

§2:

OLD:

NEW:

Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; idr@ietf. org; Alvaro Retana

Hi Bruno,

Hi John,

When:

[rest trimmed]

Regards,

--John

---

From: <[email-masked]>
Date: Tue, 10 Apr 2018 16:12:10 +0000

Hi John,

Please see inline [Bruno2]

Hi Bruno,

Hi John,

[… skipping the easy point]

2)

§2:

[Bruno2] Another case is a first reset (following the erroneous OPEN  i.e. my example) followed by some time e.g. around GR Restart Time. So I’m not sure that this is necessary quick enough not to care about (although I agree that this is very unlikely)

OLD:

NEW:

[Bruno2] Looks good to me. Thanks.
Yet this text only applies to the “N” bit and does not specify the behavior for other GR parameters. e.g. change in Restart Time. May be the following text would cover all GR parameters:
NEW2:
   If a BGP speaker which previously advertised the "N" bit opens a new
   session with different Graceful Restart parameters, these new parameters
 apply once the session has
   transitioned into ESTABLISHED state".

But I leave this to you.

Thanks,
Regards,
--Bruno

Hi Bruno,

Hi John,

When:

[rest trimmed]

Regards,

--John

---

From: "John G. Scudder" <[email-masked]>
Date: Tue, 10 Apr 2018 13:20:24 -0400

>    transitioned into ESTABLISHED state".

I adopted an even-more-general version of this:

   If a BGP speaker that previously advertised a given set of Graceful
   Restart parameters opens a new session with a different set of
   parameters, these new parameters apply once the session has
   transitioned into ESTABLISHED state.

I just posted -15 with this and the new Management Considerations section we discussed.

--John

---

From: Alvaro Retana <[email-masked]>
Date: Tue, 10 Apr 2018 18:24:36 -0400

On April 10, 2018 at 2:26:23 PM, John G. Scudder ([email-masked]) wrote:

John:

Hi!

I just posted -15...

I looked at this new update.  Just a couple of comments:

C1. S5.1 sounds very hand-wavy: “...guiding principle for the implementor
should be that if there is no realistic hope that forwarding can
continue…”.   I’m not a programmer, but I have no idea how hope can be
coded in…  I think that unless someone has a significant objection later on
(IETF LC, IESG), we can leave the text as is, given that it is correctly
qualified as suggestions… I don’t have a suggestion to make the text
better, it may be the best we can do…it just doesn’t give me a warm fuzzy
feeling…

C2. Even though S5.2. (Interaction With Other Specifications) qualifies
that it applies to “when [RFC8203] is used in conjunction with Hard Reset”,
I think that is the general case (not the exception), so I think that this
document Updates rfc8203 (and should be tagged as such).  Please make that
change on the next revision (you can wait to collect IETF LC comments).

I’m starting the IETF Last Call.

Thanks!

Alvaro.

---

From: <[email-masked]>
Date: Fri, 13 Apr 2018 10:05:15 +0000

Thank you John.
-15 looks good to me.

--Bruno

Sent: Tuesday, April 10, 2018 7:20 PM

On Apr 10, 2018, at 12:12 PM, [email-masked]<mailto:[email-masked]> wrote:

--John