Subject: [Idr] WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14

From: John Scudder <[email-masked]>
Date: Fri, 6 Oct 2017 13:30:16 -0400

Hi All,

(This is the first of a series of WGLCs as we work to clear backlog. We will stagger the WGLCs somewhat.)

A working group last call has been requested for draft-ietf-idr-bgp-optimal-route-reflection-14. Please reply to the list with your comments. As usual note we cannot advance the draft without participation from the group. Please get your comments in before October 20, 2017.

Authors, please confirm that any relevant IPR has been disclosed. 

https://tools.ietf.org/html/draft-ietf-idr-bgp-optimal-route-reflection-14

Thanks,

--John

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 6 Oct 2017 20:23:22 +0200

I am not aware about any IPR associated
with draft-ietf-idr-bgp-optimal-route-reflection-14.

Thx,
Robert Raszuk.

On Fri, Oct 6, 2017 at 7:30 PM, John Scudder <[email-masked]> wrote:

> A working group last call has been requested for
> draft-ietf-idr-bgp-optimal-route-reflection-14. Please reply to the list
> with your comments. As usual note we cannot advance the draft without
> participation from the group. Please get your comments in before October
> 20, 2017.
>
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
>

---

From: <[email-masked]>
Date: Mon, 9 Oct 2017 06:44:22 +0000

Hi John,

> From: Idr [mailto:[email-masked]] On Behalf Of John Scudder
 > Sent: Friday, October 06, 2017 7:30 PM
> 
 > Hi All,
 > 
 > A working group last call has been requested for draft-ietf-idr-bgp-optimal-route-reflection-14.
 > Please reply to the list with your comments. As usual note we cannot advance the draft without
 > participation from the group. Please get your comments in before October 20, 2017.
 > 

I'm not aware of IPR.

Thanks,
Regards,
--Bruno

 > https://tools.ietf.org/html/draft-ietf-idr-bgp-optimal-route-reflection-14

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Job Snijders <[email-masked]>
Date: Mon, 9 Oct 2017 10:11:40 +0200

Dear all,

On Fri, Oct 06, 2017 at 05:30:16PM +0000, John Scudder wrote:
> draft-ietf-idr-bgp-optimal-route-reflection-14. Please reply to the
> list with your comments. As usual note we cannot advance the draft
> without participation from the group. Please get your comments in
> before October 20, 2017.

I'd like to thank the authors for their hard work on this document. ORR
is a very clever improvement for deployment scenarios with centralized
route reflection. Kudos!

I have one remark at this point, I think the document would benefit from
an "Implementation Considerations" section or at least a clarification
on route recursion.

We've come across a number of implementations where there either minute
or quite large differences in how the route reflector choose the best
path on behalf of the client. One particular implementation stood out
because the effective 'route selection behalf of the client' was
very different from what the client would've chosen itself (at least in
our topology). 

Let's consider the following: we have BGP prefix 192.0.2.0/24 with BGP
next-hop of 198.51.100.101, 198.51.100.101 recurses to IGP next-hop
203.0.113.203, which has metric value of 1000. In a scenario without
route reflection the client will observe that the IGP cost to
198.51.100.101 is 1000. Note that the 198.51.100.101 BGP next-hop is not
present in the IGP.

With Optimal Route reflection (as user) I would assume the route
reflector recurses similarly as the client would, and takes into
consideration that from the client's point-of-view the metric to
203.0.113.203 is 1000 because of the recursion from BGP next-hop to IGP
next-hop. We should not end up in a situation where the route-reflector
does not offer a path for 192.0.2.0/24 to the client if the next-hop was
not found in the IGP.

It may be good to emphasize the need to do full route recursion from the
ORR client's point-of-view. Perhaps add this somewhere:

    "If the next-hop is not found in the IGP, the route reflector MUST
    recursively lookup the next-hop until an IGP next-hop is found."

However, herein lies the crux, we can only hope that the route-reflector
and the client have a similar enough routing table in which route
recursion will give the same result. At least one vendor thought "if the
next-hop is not in the IGP we'll simply not offer that path", but I
suspect that oftentimes errs to the side of breakage and I'd prefer "a
path is better than no path".

Kind regards,

Job

---

From: <[email-masked]>
Date: Mon, 9 Oct 2017 08:53:43 +0000

I'm not aware of any IPR related to this draft.

Regards,
Erik Åman

-----Original Message-----
From: Idr [mailto:[email-masked]] On Behalf Of John Scudder
Sent: den 6 oktober 2017 19:30
To: [email-masked]
Subject: [Idr] WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14

Hi All,

Thanks,

--John

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 9 Oct 2017 16:25:27 +0200

Hi Job,

In general I agree that implementations should be consistent in their
behaviour, but the reality is that they are now. But let's focus on your
observation for a moment ... With that few questions come up:

1. Are you using the same implementation on both client and RR with ORR ?

2. Can you send output from both ORR context and client for:

show (ip) route 192.0.2.0 det
show (ip) route 198.51.100.101 det
show (ip) route 203.0.113.203 det

The problem usually is related here not to BGP implementation but to RIB.
Not all RIBs reference a metric of N-level recursive route to the child.

When BGP registers its next hop with RIB for state/metric tracking
(assuming that implementation even does that rathere then a single call to
RIB) the metric it get's is the one attached to the specific next hop which
is being queried.

Bottom line is that mandating suggested text to the current spec may very
likely result in number of implementations to become non-compliant and what
is worse the proper fix may require in some cases complete RIB redesign
which is far from BGP and IDR area.

For your valid case I would suggest perhaps to solve it via
draft-ietf-idr-bgp-nh-cost-02.

If you find vendors buying into your case we will be more then happy to
update the NH cost draft and progress it.

Kind regards,
Robert.

---

From: Job Snijders <[email-masked]>
Date: Mon, 9 Oct 2017 17:20:51 +0200

> 1. Are you using the same implementation on both client and RR with

Sometimes.

> The problem usually is related here not to BGP implementation but to
> RIB. Not all RIBs reference a metric of N-level recursive route to
> the child.

No, this is an optimal route reflection specific challenge. A vendor
read the document, and didn't consider the case where a BGP next-hop
may recurse through another BGP route to an IGP route. This is a simple
oversight that probably would not have happened if the document
specified how to consider route recursion from via one or more
protocols.

> (assuming that implementation even does that rathere then a single
> call to RIB) the metric it get's is the one attached to the specific
> next hop which is being queried.
> 
> Bottom line is that mandating suggested text to the current spec may
> very likely result in number of implementations to become
> non-compliant and what is worse the proper fix may require in some
> cases complete RIB redesign which is far from BGP and IDR area.

I think you are overestimating the impact of the proposed change.

> If you find vendors buying into your case we will be more then happy
> to update the NH cost draft and progress it.

I disagree, draft-ietf-idr-bgp-optimal-route-reflection is the right
place for this.

The specific vendor I was referring to told us that they'll change their
behaviour: they'll fully recurse the BGP routes at route reflector,
before chosing which route to reflect. The fact that they released
software without this capacity, implies draft has ambiguity which causes
operational inconsistency in best path algortihm (with possibility for
blackholing).

Kind regards,

Job

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 9 Oct 2017 17:56:44 +0200

Job,

> I think you are overestimating the impact of the proposed change.

It would be helpful to hear from vendors how propagating to BGP an IGP
metric in the scenario
of N-level of recursion is easy or hard for them.

> The fact that they released software without this capacity, implies draft
has ambiguity
> which causes operational inconsistency in best path algortihm (with
possibility for blackholing).

Ok in such a case I don't see any problem of addressing the ambiguity with
the below sentence
added to section 4.1:

Current text:

   "In this document we refer to optimal as the decision made during best
   path selection at the IGP metric to BGP next hop comparison step."

New text:

   "In this document we refer to optimal as the decision made during best
   path selection at the IGP metric to BGP next hop comparison step.

   In the scenario of BGP route's next hop resolving via N level recursion
   the metric used for given path's comparison (provided implementation
   supports it) should be equal to IGP metric of the final IGP route the
   N level recursion resolves via. Similarly next hop reachability
validation
   should be based on the state of such IGP route."

Would WG and yourself be fine with this change ?

The only problem is what to do if implementation can not propagate such
metric .. should the N-level recursive path be less preferred and assumed
having max metric in such cases ?

Thx,
R.

---

From: Job Snijders <[email-masked]>
Date: Mon, 9 Oct 2017 21:12:36 +0200

On Mon, Oct 09, 2017 at 05:56:44PM +0200, Robert Raszuk wrote:
> metric in the scenario of N-level of recursion is easy or hard for
> them.

I think the number of BGP implementations that can't do BGP to BGP to
IGP recursion is extremely small, it'll be mostly the implementations
that don't support the IGP or already operate without RIB.

> > The fact that they released software without this capacity, implies
> > draft has ambiguity which causes operational inconsistency in best
> > path algortihm (with possibility for blackholing).
> 
> Ok in such a case I don't see any problem of addressing the ambiguity
> with the below sentence added to section 4.1:
> 
> Current text:
> 
>    "In this document we refer to optimal as the decision made during
>    best path selection at the IGP metric to BGP next hop comparison
>    step."
> 
> New text:
> 
>    "In this document we refer to optimal as the decision made during
>    best path selection at the IGP metric to BGP next hop comparison
>    step.
> 
>    In the scenario of BGP route's next hop resolving via N level
>    recursion the metric used for given path's comparison (provided
>    implementation supports it) should be equal to IGP metric of the
>    final IGP route the N level recursion resolves via. Similarly next
>    hop reachability validation should be based on the state of such
>    IGP route."
> 

I think some wordsmithing may be needed to improve the readability. The
word 'recursion' or 'recursive lookup' should probably be used.

NEW:

    """
    In this document we refer to optimal as the decision made during
    best path selection at the IGP metric to BGP next hop comparison
    step.

    In the scenario where a path's BGP next-hop can only be resolved
    through multi-level recursion, the metric used for a given path's
    comparison SHOULD be equal to the IGP metric of the final IGP route
    the recursive lookup resulted in. Similarly, next-hop reachability
    validation should be based on the state of such IGP route.
    """

> The only problem is what to do if implementation can not propagate
> such metric .. should the N-level recursive path be less preferred and
> assumed having max metric in such cases ?

Good question, 'assuming max metric' will at least prevent certain
blackholes compared to 'not announcing'. Vendors may want to chip in
here on what their implementation does. The document will be better off
if we highlight this problem and make a recommendation (such as 'treat
as less preferred').

Kind regards,

Job

---

From: Adam Chappell <[email-masked]>
Date: Mon, 9 Oct 2017 21:15:54 +0200

Perhaps resolve/recurse as far as is possible/feasible and eventually fall
back to max metric for a comparison in absence of anything else?  Would
certainly be principle of least astonishment in Job's case where an NLRI
was simply omitted because the NH wasn't comparable. Optimal should be
better, but not really any worse than classic, right?  :)

I don't think the recursion needs to be mandated though, much as the
supported IGP protocols aren't mandated. It is putting the the ORR's
Loc-RIB and selection policy into play which is what we'd avoided.

On 9 October 2017 at 17:56, Robert Raszuk <[email-masked]> wrote:

> draft has ambiguity

-- 

-- Adam.

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 9 Oct 2017 21:24:58 +0200

Hi Adam,

> where an NLRI was simply omitted because the NH wasn't comparable

If I have single path to a net there is nothing to compare it with so this
step of best path does not matter regardless if you use regular RR,
add-paths or ORR. So none of those means that we are not sending such NLRI
in iBGP.

The way I understood Job's case it is not the "NLRI itself which was
ommited", but one recursive path to such NLRI was not considered in the
comparison with other paths (next hops) as metric to its recursive next hop
was not there.

Thx,
R.

On Mon, Oct 9, 2017 at 9:15 PM, Adam Chappell <[email-masked]>
wrote:

>> recursion

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Mon, 9 Oct 2017 20:41:32 +0000

I don't understand the phrase " if implementation can not propagate such metric ".

Suppose we have a BGP nexthop NH1.
There are two BGP routes to NH1: BGP1 and BGP2.
BGP1 has nexthop NNH1.
BGP2 has nexthop NNH2.
BGP1 is the BGP best path, however, NNH2 has a lower (better) metric.
Using the stated algorithm would choose NNH1, whereas NNH2 is actually better.
Networks with BGP nexthops out of reach of the IGP are susceptible to this issue.
I'm sure this is fine in most cases, but it is good to be aware.
You could add it to a "deployment considerations" section.

Thanks,
Jakob

From: Idr [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Monday, October 9, 2017 12:25 PM
To: Adam Chappell <[email-masked]>
Cc: idr wg <[email-masked]>; [email-masked]
Subject: Re: [Idr] WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14

Hi Adam,

> where an NLRI was simply omitted because the NH wasn't comparable

If I have single path to a net there is nothing to compare it with so this step of best path does not matter regardless if you use regular RR, add-paths or ORR. So none of those means that we are not sending such NLRI in iBGP.

The way I understood Job's case it is not the "NLRI itself which was ommited", but one recursive path to such NLRI was not considered in the comparison with other paths (next hops) as metric to its recursive next hop was not there.

Thx,
R.

On Mon, Oct 9, 2017 at 9:15 PM, Adam Chappell <[email-masked]<mailto:[email-masked]>> wrote:
Perhaps resolve/recurse as far as is possible/feasible and eventually fall back to max metric for a comparison in absence of anything else?  Would certainly be principle of least astonishment in Job's case where an NLRI was simply omitted because the NH wasn't comparable. Optimal should be better, but not really any worse than classic, right?  :)

I don't think the recursion needs to be mandated though, much as the supported IGP protocols aren't mandated. It is putting the the ORR's Loc-RIB and selection policy into play which is what we'd avoided.

On 9 October 2017 at 17:56, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:
Job,

> I think you are overestimating the impact of the proposed change.

It would be helpful to hear from vendors how propagating to BGP an IGP metric in the scenario
of N-level of recursion is easy or hard for them.

> The fact that they released software without this capacity, implies draft has ambiguity
> which causes operational inconsistency in best path algortihm (with possibility for blackholing).

Ok in such a case I don't see any problem of addressing the ambiguity with the below sentence
added to section 4.1:

Current text:

New text:

   N level recursion resolves via. Similarly next hop reachability validation

The only problem is what to do if implementation can not propagate such metric .. should the N-level recursive path be less preferred and assumed having max metric in such cases ?

Thx,
R.

On Mon, Oct 9, 2017 at 5:20 PM, Job Snijders <[email-masked]<mailto:[email-masked]>> wrote:

Sometimes.

Kind regards,

Job

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

--

-- Adam.

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 9 Oct 2017 23:03:09 +0200

Jakob,

Let me try to describe it in a bit different way ...

Prefix 1.1.1.1/24 has 3 paths each with different next hop: NH1, NH2, NH3

NH1 is an NLRI in BGP1 and it's next hop is NNH1 (IGP metric 100).

NH2 is an NLRI in BGP2 with next hop NNH2 which is again a NLRI of BGP3 and
next hop NNNH3 (IGP metric 90)

NH3 is native IGP next hop with IGP metric 110

So when you run (regular or ORR) best path on 1.1.1.1/24 you should have
all three metrics 90, 100, 110 there to pick the best path. If next hop is
not in IGP with or without recursion such path is invalid so we should not
worry about it here.

Thx,
R.

On Mon, Oct 9, 2017 at 10:41 PM, Jakob Heitz (jheitz) <[email-masked]>
wrote:

> *From:* Idr [mailto:[email-masked]] *On Behalf Of *Robert Raszuk
> *Sent:* Monday, October 9, 2017 12:25 PM
> *To:* Adam Chappell <[email-masked]>
> *Cc:* idr wg <[email-masked]>; [email-masked]
> *Subject:* Re: [Idr] WGLC for draft-ietf-idr-bgp-optimal-
> route-reflection-14
>
>
>
> Hi Adam,
>
>
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Mon, 9 Oct 2017 21:11:11 +0000

We have been assuming that the metric of a BGP hop is 0. Accordingly:
If it fails to resolve during the recursion, then it is unreachable.
If the recursion ends at the SPF root, without touching an IGP reachable NH, then the metric is 0.

Thanks,
Jakob

From: [email-masked] [mailto:[email-masked]] On Behalf Of Robert Raszuk
Sent: Monday, October 9, 2017 2:03 PM
To: Jakob Heitz (jheitz) <[email-masked]>
Cc: Adam Chappell <[email-masked]>; idr wg <[email-masked]>; [email-masked]
Subject: Re: [Idr] WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14

Jakob,

Prefix 1.1.1.1/24<http://1.1.1.1/24> has 3 paths each with different next hop: NH1, NH2, NH3

NH2 is an NLRI in BGP2 with next hop NNH2 which is again a NLRI of BGP3 and next hop NNNH3 (IGP metric 90)

So when you run (regular or ORR) best path on 1.1.1.1/24<http://1.1.1.1/24> you should have all three metrics 90, 100, 110 there to pick the best path. If next hop is not in IGP with or without recursion such path is invalid so we should not worry about it here.

Thx,
R.

On Mon, Oct 9, 2017 at 10:41 PM, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:
I don't understand the phrase " if implementation can not propagate such metric ".

Thanks,
Jakob

From: Idr [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Robert Raszuk
Sent: Monday, October 9, 2017 12:25 PM
To: Adam Chappell <[email-masked]<mailto:[email-masked]>>
Cc: idr wg <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [Idr] WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14

Hi Adam,

Thx,
R.

Current text:

New text:

Thx,
R.

Sometimes.

Kind regards,

Job

--

-- Adam.

---

From: Nick Hilliard <[email-masked]>
Date: Mon, 09 Oct 2017 22:16:28 +0100

Robert Raszuk wrote:
> So when you run (regular or ORR) best path on 1.1.1.1/24
> <http://1.1.1.1/24> you should have all three metrics 90, 100, 110 there
> to pick the best path. If next hop is not in IGP with or without
> recursion such path is invalid so we should not worry about it here. 

correct me if I'm wrong, but I don't think you're understanding what
Jakob is saying.

When you have multiple orr clients (BGP1 and BGP2 in Jakob's example),
their IGP view of the next-next-hops might be different from each other.
 This might happen if the NH address was anycasted in the IGP, for
example, causing the NNH to be resolved different on different bgp clients.

In this situation, the behaviour of the ORR client is not defined
clearly in the spec.  Because of the ambiguity, some implementations
appear to have steered clear of handling recursion because it can cause
non deterministic resolution problems.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 9 Oct 2017 23:20:02 +0200

Nick,

We are talking about single client ...

Thx,
R.

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 9 Oct 2017 23:37:44 +0200

What if after say N levels of recursion it ends up on IGP leaf with metric
X to it from the logical placement perspective ? Do you pass this metric X
to BGP to be used in NH metric comparison among other paths (which may be
also N level recursive) ?

Ending up on SPF root in the logical RR placement would be the remote
client node so 0 is OK, but the crux of the question here is really above.

Thx,
R.

On Mon, Oct 9, 2017 at 11:11 PM, Jakob Heitz (jheitz) <[email-masked]>
wrote:

> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *Robert
> Raszuk
> *Sent:* Monday, October 9, 2017 2:03 PM
> *To:* Jakob Heitz (jheitz) <[email-masked]>
> *Cc:* Adam Chappell <[email-masked]>; idr wg <[email-masked]>;
> [email-masked]
>

---

From: "Jakob Heitz (jheitz)" <[email-masked]>
Date: Mon, 9 Oct 2017 21:59:28 +0000

Yes.

Thanks,
Jakob

Sent: Monday, October 9, 2017 2:38 PM

What if after say N levels of recursion it ends up on IGP leaf with metric X to it from the logical placement perspective ? Do you pass this metric X to BGP to be used in NH metric comparison among other paths (which may be also N level recursive) ?

Ending up on SPF root in the logical RR placement would be the remote client node so 0 is OK, but the crux of the question here is really above.

Thx,
R.

On Mon, Oct 9, 2017 at 11:11 PM, Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>> wrote:
We have been assuming that the metric of a BGP hop is 0. Accordingly:
If it fails to resolve during the recursion, then it is unreachable.
If the recursion ends at the SPF root, without touching an IGP reachable NH, then the metric is 0.

Thanks,
Jakob

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Robert Raszuk
Sent: Monday, October 9, 2017 2:03 PM
To: Jakob Heitz (jheitz) <[email-masked]<mailto:[email-masked]>>
Cc: Adam Chappell <[email-masked]<mailto:[email-masked]>>; idr wg <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>

Jakob,

Thx,
R.

Thanks,
Jakob

Hi Adam,

Thx,
R.

Current text:

New text:

Thx,
R.

Sometimes.

Kind regards,

Job

--

-- Adam.

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 13 Oct 2017 13:51:56 -0400

Hi All,

It seems to me as though Robert and Job agreed on the proposed text below, or something like it. If so, I suggest the authors update the draft and publish as -15, so the WG knows what we're talking about.

Speaking for myself, I don't object to the proposed clarification but it seems to me this draft isn't strictly speaking the right place for it since it amounts to a correction of an underspecification in RFC 4271. That underspecification affects all BGP implementations, not just ORR. Of course, most implementors "just knew" to do the obviously right thing, but apparently Job has encountered one implementor who didn't. It's tempting to criticize the implementor, but IMO the right target of criticism is the spec. 

Specifically I think the underspecification in RFC 4271 is in 9.1.2.2 (e), the definition of interior cost. 9.1.2.1 is actually quite careful to talk about recursive resolution, but unfortunately that care wasn't extended to talking about the interior cost. (The final "similarly..." sentence in the proposed text is strictly speaking redundant with 9.1.2.1.)

Even though this draft isn't the right place to correct the weakness in RFC 4271, since every BGP implementation should get this right, not just ORR implementations, I don't see harm in doing it here.

--John

> On Oct 9, 2017, at 3:12 PM, Job Snijders <[email-masked]> wrote:
>

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 13 Oct 2017 13:55:55 -0400

Paging through the discussion so far, I see exactly one person, Job Snijders, has expressed what I interpret as support for publishing the document. I presume the authors support it, and nobody has expressed opposition. However, it would be quite helpful if there were more contributions, whether to say "ship it", or "needs more work", or whatever.

Thanks,

--John

> On Oct 6, 2017, at 1:30 PM, John Scudder <[email-masked]> wrote:
> 
> Hi All,
> 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Didr-2Dbgp-2Doptimal-2Droute-2Dreflection-2D14&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=2T8sWKXOuk1R-CTDDPFM8HO_a6AbOHGOseJ81Ht9_4Q&s=yunFv1zXtd-bLp5IQGspvPZQYpLtAFtcXdjv_vA4-rU&e= 
> 
> Thanks,
> 
> --John

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 13 Oct 2017 19:13:39 +0100

John G. Scudder wrote:
> Paging through the discussion so far, I see exactly one person, Job
> Snijders, has expressed what I interpret as support for publishing
> the document. I presume the authors support it, and nobody has
> expressed opposition. However, it would be quite helpful if there
> were more contributions, whether to say "ship it", or "needs more
> work", or whatever.

We are being slack.

ORR is ridiculously useful and we've needed it for years. Full support
for the doc (+ implementation clarification).

Nick

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Fri, 13 Oct 2017 18:15:17 +0000

Support.

-- 
Cheers, 
Rajiv

-----Original Message-----
From: Idr <[email-masked]> on behalf of Nick Hilliard <[email-masked]>
Date: Friday, October 13, 2017 at 2:13 PM
To: John Scudder <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [Idr] WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14

    We are being slack.

    Nick

---

From: "Serpil Bayraktar (serpil)" <[email-masked]>
Date: Fri, 13 Oct 2017 18:37:22 +0000

I support it.

Serpil

From: Idr <[email-masked]> on behalf of "John G. Scudder" <[email-masked]>
Date: Friday, October 13, 2017 at 10:56 AM
To: "[email-masked]" <[email-masked]>
Subject: Re: [Idr] WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14

Thanks,

--John

On Oct 6, 2017, at 1:30 PM, John Scudder <[email-masked]<mailto:[email-masked]>> wrote:
Hi All,

---

From: "Manish Bhardwaj (manbhard)" <[email-masked]>
Date: Fri, 13 Oct 2017 19:07:15 +0000

Support.

-Manish.

On 10/13/17, 10:55 AM, "Idr on behalf of John G. Scudder"
<[email-masked] on behalf of [email-masked]> wrote:

>Snijders, has expressed what I interpret as support for publishing the
>document. I presume the authors support it, and nobody has expressed
>opposition. However, it would be quite helpful if there were more
>contributions, whether to say "ship it", or "needs more work", or
>whatever.
>
>Thanks,
>
>--John
>
>>https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_
>>draft-2Dietf-2Didr-2Dbgp-2Doptimal-2Droute-2Dreflection-2D14&d=DwICAg&c=H
>>AkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=2T8
>>sWKXOuk1R-CTDDPFM8HO_a6AbOHGOseJ81Ht9_4Q&s=yunFv1zXtd-bLp5IQGspvPZQYpLtAF
>>tcXdjv_vA4-rU&e= 
>> 
>> Thanks,
>> 
>> --John
>

---

From: "John G. Scudder" <[email-masked]>
Date: Fri, 13 Oct 2017 16:52:58 -0400

Hi Everyone,

> Authors, please confirm that any relevant IPR has been disclosed. 

I was just told by Serpil Bayraktar that Cisco has IPR related to this spec. She is working with their legal department to disclose it, and of course as soon as the disclosure is available we'll notify the WG.

Naturally we will keep the WGLC open until the IPR disclosure has been published and the WG has had a chance to consider it.

I want to thank Serpil for her diligence in bringing this to my attention as soon as she became aware of it. This is as good a time as any to remind everyone of the Note Well we all see multiple times per meeting as well as when we sign up for a mailing list: https://www.ietf.org/about/note-well.html. In particular, see https://www.rfc-editor.org/rfc/rfc8179.txt, "Intellectual Property Rights in IETF Technology", which is linked from the Note Well, which describes the who, what and when in detail. Let me quote the beginning of the Note Well here:

"Any submission to the IETF intended by the Contributor for publication as all or part of an IETF Internet-Draft or RFC and any statement made within the context of an IETF activity is considered an "IETF Contribution". Such statements include oral statements in IETF sessions, as well as written and electronic communications made at any time or place..."

and so on.

While we're at it, I will note that four of the authors -- Robert Raszuk, Bruno Decraene, Eric Aman, and Adam Chappell -- have said they didn't know of any IPR, so presumably this comes as a surprise to them. I have not seen replies to this question from Stephane Litkowski, Christian Cassar or Kevin Wang yet, I've cc'd them. 

Regards,

--John

---

From: Kevin F Wang <[email-masked]>
Date: Fri, 13 Oct 2017 20:54:47 +0000

Hi John,

I am not aware of any IPR related to this draft from our side.

Thanks,
Kevin

On 10/13/17, 4:52 PM, "John Scudder" <[email-masked]> wrote:

    Hi Everyone,

    

    
    and so on.

    Regards,
    
    --John

---

From: Keyur Patel <[email-masked]>
Date: Fri, 13 Oct 2017 20:57:51 +0000

Support.
Regards,
Keyur

On 10/13/17, 10:55 AM, "Idr on behalf of John G. Scudder" <[email-masked] on behalf of [email-masked]> wrote:

    Thanks,
    
    --John

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 13 Oct 2017 23:31:43 +0200

Hi John and WG,

Thank you for your note.

While I am still in process of getting full clarification from Cisco on
what has been filed without authors knowledge I think the message you got
from Serpil is incorrect.

To the best of my current understanding cisco filed some patents with
claims describing implementation solutions used to implement ideas from ORR
document not to claim IPR rights to such ideas itself.

Both are very different and I do not understand while the former would in
any way impact the IETF process.

Kind regards,
Robert.

On Oct 13, 2017 4:53 PM, "John G. Scudder" <[email-masked]> wrote:

Hi Everyone,

> Authors, please confirm that any relevant IPR has been disclosed.

I was just told by Serpil Bayraktar that Cisco has IPR related to this
spec. She is working with their legal department to disclose it, and of
course as soon as the disclosure is available we'll notify the WG.

Naturally we will keep the WGLC open until the IPR disclosure has been
published and the WG has had a chance to consider it.

I want to thank Serpil for her diligence in bringing this to my attention
as soon as she became aware of it. This is as good a time as any to remind
everyone of the Note Well we all see multiple times per meeting as well as
when we sign up for a mailing list: https://www.ietf.org/about/not
e-well.html. In particular, see https://www.rfc-editor.org/rfc/rfc8179.txt,
"Intellectual Property Rights in IETF Technology", which is linked from the
Note Well, which describes the who, what and when in detail. Let me quote
the beginning of the Note Well here:

"Any submission to the IETF intended by the Contributor for publication as
all or part of an IETF Internet-Draft or RFC and any statement made within
the context of an IETF activity is considered an "IETF Contribution". Such
statements include oral statements in IETF sessions, as well as written and
electronic communications made at any time or place..."

and so on.

While we're at it, I will note that four of the authors -- Robert Raszuk,
Bruno Decraene, Eric Aman, and Adam Chappell -- have said they didn't know
of any IPR, so presumably this comes as a surprise to them. I have not seen
replies to this question from Stephane Litkowski, Christian Cassar or Kevin
Wang yet, I've cc'd them.

Regards,

--John

---

From: "Serpil Bayraktar (serpil)" <[email-masked]>
Date: Fri, 13 Oct 2017 22:09:54 +0000

Hey Robert,

I think there is a big misunderstanding here. I have the data to back up what I said. The question is how do we go about discussing and resolving it gracefully. I am happy to provide the data and work with you and John towards a resolution.

Serpil

From: Idr <[email-masked]> on behalf of Robert Raszuk <[email-masked]>
Date: Friday, October 13, 2017 at 2:32 PM
To: "John G. Scudder" <[email-masked]>
Cc: idr wg <[email-masked]>, Kevin Wang <[email-masked]>
Subject: Re: [Idr] IPR in draft-ietf-idr-bgp-optimal-route-reflection-14 [was: Re: WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14]

Hi John and WG,

Thank you for your note.

While I am still in process of getting full clarification from Cisco on what has been filed without authors knowledge I think the message you got from Serpil is incorrect.

To the best of my current understanding cisco filed some patents with claims describing implementation solutions used to implement ideas from ORR document not to claim IPR rights to such ideas itself.

Both are very different and I do not understand while the former would in any way impact the IETF process.

Kind regards,
Robert.

On Oct 13, 2017 4:53 PM, "John G. Scudder" <[email-masked]<mailto:[email-masked]>> wrote:
Hi Everyone,

and so on.

Regards,

--John

---

From: John Scudder <[email-masked]>
Date: Fri, 13 Oct 2017 22:18:53 +0000

On Oct 13, 2017, at 5:31 PM, Robert Raszuk <[email-masked]<mailto:[email-masked]>> wrote:

The validity of a given IPR claim is explicitly not the responsibility of the IETF and the WG (as a group) to determine. Individual WG contributors may wish to make their own determination, possibly in consultation with legal counsel.

If you consult the IETF IPR page (https://datatracker.ietf.org/ipr/about<https://datatracker.ietf.org/ipr/about/>) you will see "The IETF takes no position regarding the validity or scope of any intellectual property rights or other rights that might be claimed to pertain to the implementation or use of the technology described in any IETF documents or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights." And so on.

As an individual contributor, I don't plan to draw any conclusions until I've seen the specifics of the Cisco disclosure.

--John

---

From: Robert Raszuk <[email-masked]>
Date: Sat, 14 Oct 2017 22:54:59 +0200

Hi Serpil,

To me this looks very clear.

You have filed 2 patents in 2015 describing an idea which was invented by
completely different set of inventors in 2010-2011 without even sharing
this information with the original authors.

https://patents.google.com/patent/US20160248663A1/

https://patents.google.com/patent/US20170104676A1/

You have also filed a 3rd patent in 2015 which documents your
implementation innovation to automatically allocate a logical IGP root
based on the client address. Cool innovation, but as this is not part of
the current document it is irrelevant to this discussion.

https://patents.google.com/patent/US20160248658A1/

To me the next steps are obvious:

Option 1 - Cisco formally discloses the first two patents and IDR WG will
decided if this is of any barrier to progress the document further

Option 2 - Cisco withdraws those two patents

Thx,
Robert.

On Sat, Oct 14, 2017 at 12:09 AM, Serpil Bayraktar (serpil) <
[email-masked]> wrote:

>
>
> Hey Robert,
>
>
>
> I think there is a big misunderstanding here. I have the data to back up
> what I said. The question is how do we go about discussing and resolving it
> gracefully. I am happy to provide the data and work with you and John
> towards a resolution.
>
>
>
> Serpil
>
>
>
>
>
>
>
> *From: *Idr <[email-masked]> on behalf of Robert Raszuk <
> [email-masked]>
> *Date: *Friday, October 13, 2017 at 2:32 PM
> *To: *"John G. Scudder" <[email-masked]>
> *Cc: *idr wg <[email-masked]>, Kevin Wang <[email-masked]>
> *Subject: *Re: [Idr] IPR in draft-ietf-idr-bgp-optimal-route-reflection-14
> [was: Re: WGLC for draft-ietf-idr-bgp-optimal-route-reflection-14]
>
>
>
> Hi John and WG,
>
>
>
> Thank you for your note.
>
>
>
> when we sign up for a mailing list: https://www.ietf.org/about/
> note-well.html. In particular, see https://www.rfc-editor.org/
> rfc/rfc8179.txt, "Intellectual Property Rights in IETF Technology", which
> is linked from the Note Well, which describes the who, what and when in
> detail. Let me quote the beginning of the Note Well here:
>

---

From: Jeff Tantsura <[email-masked]>
Date: Sun, 15 Oct 2017 13:15:21 -0700

yes/support

Cheers,
Jeff

        
        Thanks,
        
        --John

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Mon, 16 Oct 2017 15:08:34 +0000

Support.

---

From: Randy Bush <[email-masked]>
Date: Mon, 16 Oct 2017 09:28:51 -0700

> While I am still in process of getting full clarification from Cisco
> on what has been filed without authors knowledge I think the message
> you got from Serpil is incorrect.
> 
> claims describing implementation solutions used to implement ideas
> from ORR document not to claim IPR rights to such ideas itself.
> 

perhaps we should let the lawyers be lawyers and we stick to
engineering?  

randy

---

From: Christian Cassar <[email-masked]>
Date: Mon, 16 Oct 2017 18:04:33 +0100

Hello John,

I was not aware of any IPR. I will work with Serpil and we will get back.

Thanks
Christian

On 13/10/2017 21:52, John G. Scudder wrote:
> --John.

---

From: Randy Bush <[email-masked]>
Date: Mon, 16 Oct 2017 15:56:59 -0700

robert,

i am becoming more confused.  (no snark needed :)

the document refs 2119 (which has been replaced, btw) but does not seem
to use MUST et alia.  is this document informational or standards track?

randy

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 17 Oct 2017 09:16:04 +0200

Hello Randy,

> to use MUST et alia.  is this document informational or standards track?

​The way I see the docs there are MUSTs and SHOULDs ...

​   ​
 However, the solutions
*​​MUST NOT* prevent
   reflecting more than one optimal path to the client as path diversity
   may be desirable for load balancing or fast restoration.  In cases
   where add-path and optimal route reflection are configured together,
   the route reflector
*​​MUST* reflect n optimal paths to a client, where n

   is the add-path count.​

​Point taken that some SHOULDs are lower case - will fix.

As far as 2119 replacement happy to update too.

​Last I kept standards track as base BGP RFC or RR RFC is not really
allowing policy based

best path selection on RRs. He we are clearly introducing such
possibility. So while no new

bits in BGP protocol are introduced BGP behavior can be considered
departure/extension

of current
standard. However to me personally it can be informational or even
BCP. I am happy to adjust

it per WG and chairs direction.

Thx,
Robert

---

From: Randy Bush <[email-masked]>
Date: Tue, 17 Oct 2017 01:56:44 -0700

apologies, i was a bit quick

% grep MUST draft-ietf-idr-bgp-optimal-route-reflection-15.txt 
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   optimal route to the client.  However, the solutions MUST NOT prevent
   the route reflector MUST reflect n optimal paths to a client, where n

---

From: Randy Bush <[email-masked]>
Date: Tue, 17 Oct 2017 14:11:15 -0700

of course, the normative language is not in the actual protool
semantics, but rather in some operational advice in

   4.3.  Solution Interactions

is this docco info or a bcp?

randy

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 18 Oct 2017 18:57:09 +0300

An update on this WGLC.

Tl;dr: Before we proceed forward we need to see the promised IPR declaration from Cisco, we need to hear back from Stephane about IPR, and we need to close on the question Randy raised regarding document status.

1. IPR

Serpil Bayraktar let me know a few days ago that Cisco intends to disclose IPR. The disclosure hasn't been filed yet, we will not progress the document until it has been filed and the WG has an opportunity to consider the filing. I would like to encourage the WG to wait until the filing is done before discussing further -- while we wait you may want to pass the time reviewing the IETF's policy on what is, and is not, in-scope for us to consider when discussing IPR in our work. 

Authors Robert Raszuk, Bruno Decraene, Eric Aman, Kevin Wang, and Adam Chappell have said they didn't know of any IPR. Christian Cassar said "I was not aware of any IPR. I will work with Serpil and we will get back." Stephane Litkowski hasn't answered.

https://datatracker.ietf.org/ipr/search/?submit=draft&id=draft-ietf-idr-bgp-optimal-route-reflection

2. Other

Support from Job Snijders, Nick Hilliard, Rajiv Asati, Serpil Bayraktar, Manish Bhardwaj, Keyur Patel, Jeff Tantsura, Acee Lindem

Review by Job Snijders, largely about route recursion, led to a discussion and update to the document. 

Note that much of the support was expressed before the IPR issue came up, and it's the individuals' prerogative to change their minds if they see fit. Support was for version -15 of the document, though, or at least was expressed subsequent to the discussion and publication of -15.

There is an unresolved question from Randy Bush as to whether the document should really be Standards Track or some other status (Informational and BCP were both mentioned).

Thanks,

--John

> On Oct 6, 2017, at 8:30 PM, John Scudder <[email-masked]> wrote:
> 
> Hi All,
>

---

From: Robert Raszuk <[email-masked]>
Date: Wed, 18 Oct 2017 18:11:15 +0200

Hi,

> There is an unresolved question from Randy Bush as to whether the
document should really be
> Standards Track or some other status (Informational and BCP were both
mentioned).

I explained the rationale for keeping it as Standards Track document.

There is also recent outstanding question from Randy supposedly questioning
the name of section 4.3. I personally do not see anything wrong with the
current name.

Thx,
R.

On Wed, Oct 18, 2017 at 5:57 PM, John G. Scudder <[email-masked]> wrote:

> An update on this WGLC.
>
> Tl;dr: Before we proceed forward we need to see the promised IPR
> declaration from Cisco, we need to hear back from Stephane about IPR, and
> we need to close on the question Randy raised regarding document status.
>
> 1. IPR
>
> Serpil Bayraktar let me know a few days ago that Cisco intends to disclose
> IPR. The disclosure hasn't been filed yet, we will not progress the
> document until it has been filed and the WG has an opportunity to consider
> the filing. I would like to encourage the WG to wait until the filing is
> done before discussing further -- while we wait you may want to pass the
> time reviewing the IETF's policy on what is, and is not, in-scope for us to
> consider when discussing IPR in our work.
>
> Authors Robert Raszuk, Bruno Decraene, Eric Aman, Kevin Wang, and Adam
> Chappell have said they didn't know of any IPR. Christian Cassar said "I
> was not aware of any IPR. I will work with Serpil and we will get back."
> Stephane Litkowski hasn't answered.
>
> https://datatracker.ietf.org/ipr/search/?submit=draft&id=
> draft-ietf-idr-bgp-optimal-route-reflection
>
> 2. Other
>
> Support from Job Snijders, Nick Hilliard, Rajiv Asati, Serpil Bayraktar,
> Manish Bhardwaj, Keyur Patel, Jeff Tantsura, Acee Lindem
>
> Note that much of the support was expressed before the IPR issue came up,
> and it's the individuals' prerogative to change their minds if they see
> fit. Support was for version -15 of the document, though, or at least was
> expressed subsequent to the discussion and publication of -15.
>
> There is an unresolved question from Randy Bush as to whether the document
> should really be Standards Track or some other status (Informational and
> BCP were both mentioned).
>
> Thanks,
>
> --John
>
>
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.
> ietf.org_html_draft-2Dietf-2Didr-2Dbgp-2Doptimal-2Droute-
> 2Dreflection-2D14&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-
> ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=2T8sWKXOuk1R-CTDDPFM8HO_
> a6AbOHGOseJ81Ht9_4Q&s=yunFv1zXtd-bLp5IQGspvPZQYpLtAFtcXdjv_vA4-rU&e=
> >
> > Thanks,
> >
> > --John
>

---

From: Job Snijders <[email-masked]>
Date: Wed, 18 Oct 2017 18:26:48 +0200

I am fine with standards track.

Kind regards,

Job

On Wed, Oct 18, 2017 at 6:11 PM, Robert Raszuk <[email-masked]> wrote:
>> should really be
>> declaration from Cisco, we need to hear back from Stephane about IPR, and we
>> need to close on the question Randy raised regarding document status.
>>
>> 1. IPR
>>
>> IPR. The disclosure hasn't been filed yet, we will not progress the document
>> until it has been filed and the WG has an opportunity to consider the
>> filing. I would like to encourage the WG to wait until the filing is done
>> before discussing further -- while we wait you may want to pass the time
>> reviewing the IETF's policy on what is, and is not, in-scope for us to
>> Chappell have said they didn't know of any IPR. Christian Cassar said "I was
>> not aware of any IPR. I will work with Serpil and we will get back."
>> and it's the individuals' prerogative to change their minds if they see fit.
>> Support was for version -15 of the document, though, or at least was
>> should really be Standards Track or some other status (Informational and BCP
>> were both mentioned).
>>
>> Thanks,
>>
>> --John
>>
>>

---

From: "John G. Scudder" <[email-masked]>
Date: Wed, 18 Oct 2017 20:40:38 +0300

Cisco's IPR filing is now available at the datatracker link below.

I propose a one week extension to the WGLC period (until next Wednesday, October 25) to ensure WG participants have ample time to consider this information. If anyone is concerned that it's not enough time, feel free to unicast me (or Sue, or Alvaro).

If you've previously expressed support for publication, I will assume that you still support it unless you say otherwise.

Thanks,

--John

On Oct 18, 2017, at 6:57 PM, John G. Scudder <[email-masked]> wrote:
...

---

From: Randy Bush <[email-masked]>
Date: Thu, 19 Oct 2017 21:37:23 -0700

>> document should really be Standards Track or some other status
>> (Informational and BCP were both mentioned).
> 

you did not understand my point.  i was fine with the title of that
section.  i was pointing out that it was an operational guidance
section, not a protocol section.  and it had the only occurrences of
normative language.  hence my suggestion that this was an informative
bcp.

randy

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 20 Oct 2017 09:29:50 +0200

Randy,

This is first time I hear that use of not use of MUSTs and SHOULDs
classifies an I-D as standards track, informational or bcp. To me spec
content means a bit more then use or not use of those words.

But as I said personally I completely do not care how it is
progressed/shipped.

It is shipping in major routing implementations today and is being used in
few networks and evaluated for use in few others and this is what really
matters the most to me.

Cheers,
R.

---

From: "John G. Scudder" <[email-masked]>
Date: Thu, 26 Oct 2017 18:57:32 +0300

Hi All,

This WGLC has concluded. Based on feedback from the WG, we will forward the draft for publication as an RFC. A few notes:

- I have some editorial comments I'll forward as part of the document shepherd process.
- I think the discussion about document status between Robert and Randy didn't conclude other than with implicit agreement to disagree. There was no WG consensus to change the intended status, but I'll note it as part of the shepherd writeup for the IESG to be aware of.
- No objections to publication were raised related to the late IPR disclosure by Cisco.
- Stephane Litkowski, listed in the "authors" section (Section 2 of the draft) hasn't replied regarding IPR. I'll note this in the shepherd writeup.

Thanks,

--John

> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_idr&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=hLt5iDJpw7ukqICc0hoT7A&m=2T8sWKXOuk1R-CTDDPFM8HO_a6AbOHGOseJ81Ht9_4Q&s=t3ZEK22Jly_SUaRFAJK66Qs8r-lyPunM9_dcnx2hcNo&e=