Subject: Re: [Idr] draft-ietf-idr-rs-bfd-05 - 2 Week WG LC from 4/13 to 4/27

From: Randy Bush <[email-masked]>
Date: Sat, 02 Jun 2018 19:15:31 -0700

excuse top post

as we left the discussion at lunch

  o this does not change bgp

  o when the router knows the link to an nlri goes down, the rs
    needs to know so it can give the router the next best path's
    nlri

randy

> The WGLC is officially over, but not announced as over. I meant to
> comment on this ID sooner, but here are some thoughts anyway.
> 
> The IDR WG needs to take on board the Camel presentation from dnsops.
> 
> BGP is already a hugely convoluted protocol, and this convolution
> breeds both complexity and fragility.  It's not that any individual
> chunk of the protocol is especially complicated in itself: it's more
> that in order to produce a working BGP stack, it's necessary to
> transcode voluminous tomes of RFCs and then figure out all the
> implementation subtleties required for reasonable interoperability
> with other stacks.
> 
> Occasionally I get to talk to people who attempt to write new BGP
> stacks and their opinion is that it's a daunting prospect.  The sheer
> volume of features in BGP is now such that building new BGP
> implementations is prohibitively expensive, and the possibility of
> starting again with a new protocol (bgp5?) is receding further into
> the distance due to the scale of what it would take to replicate the
> bits of BGP that people need and use.
> 
> There's no doubt all these protocol developments are useful, but the
> cost of implementing them is not linear.  Probably as a working group,
> we need to take stock of this and try to decide where we want BGP to
> be in 20 years?  Tacking on more arms and legs?  Or do we ever
> envisage taking out the secateurs or a saw?
> 
> Regarding draft-ietf-idr-rs-bfd implementations, or lack thereof,
> there isn't a compelling reason why the normal guidelines for WG LC
> should be waived.  In fact, John's argument suggests the opposite -
> there have been some major protocol design changes in what's been
> suggested since the draft was first mooted, and there's still no code
> to prove that it will work well in real life situations, or that the
> spec is defined well enough that one implementation might interoperate
> with another.
> 
> Interoperability in this case is more important than usual because the
> ID is targetted at ixp route servers, which often have weird, ancient
> and tragically broken devices connected.  On AS43760, which is a small
> RS, we still have ~5% of devices not supporting the AS4
> extension. Think about what that means in terms of code rot.
> 
> Regarding the content of the ID, there are two aspects to it: bfd
> brokerage and NH reachability propagation.
> 
> The draft states that both problems need to be fixed, but do they
> really?  Is BFD brokerage not sufficient?  Putting it another way, is
> the complexity required by the NH reachability propagation really
> justified by the gain in benefit?  I am having trouble understanding
> why.  Although I can see what it does and how it works, it looks like
> it's optimising an edge case rather than dealing with the meat of the
> issue which is bfd brokerage.
> 
> It would dramatically simplify the proposal if the NH reachability
> propagation part could be stripped out.  If it's left in, then the
> overall mechanism seems to be a very complicated and very new
> departure for BGP, which gets back to the point above about the point
> at which the straw breaks the camel's back.  I really think the
> authors need to make it clear why this part of the specification
> cannot be done without.
> 
> Regarding WGLC, I can understand the desire to get this ID into the
> bag, get it on customer feature spec lists and get it out the door
> from a vendor point of view, but I query the wisdom of pushing a draft
> like this through IETF WGLC without any running code in the general
> case, and a good deal more in this specific case.
> 
> Right now, I'm inclined to think that it shouldn't proceed until there
> is consensus that the extenuating circumstances are sufficient that it
> should get special treatment, or, as John says below, whether we
> should open up a wider discussion on what part running code plays in
> protocol development.  We may also wish to discuss protocol
> over-development and the balance between complexity vs gain.
> 
> Nick
> 
> John G. Scudder wrote:
> > (As a co-author, yes, really. Maybe the co-chair would even disagree
> > with some of these points, who knows.[*])
> > 
> >> On Apr 13, 2018, at 1:31 PM, Job Snijders <[email-masked]
> >> <mailto:[email-masked]>> wrote:
> >> 
> >> Dear all,
> >> 
> >> Are there implementations? Is there an implemention report as per
> >> IDR tradition?
> >> 
> >> If not, I don’t support progressing this draft at this moment.
> >> 
> >> The machinery described in this draft is not entirely straight
> >> forward, without running code we should hold off on progressing.
> > 
> > For those who missed it, there is a bit more context in Job's
> > earlier note,
> > https://www.ietf.org/mail-archive/web/idr/current/msg19261.html,
> > including
> > 
> > "ps. yes, i know IDR can have WGLC without running code, but given that
> > this is a non-trivial extension to BGP; I'd really like to see multiple
> > implementations that are able to interact with each other in the correct
> > way and a report detailing the interopability."
> > 
> > In this case the authors are, exactly and without pretense,
> > requesting WGLC without running code. Which seems a funny thing
> > coming from me -- sometimes all these hats I wear get too tight and
> > give me a headache. Anyway, in this case we think it's worth doing
> > the WGLC because it indicates as clearly as we can to potential
> > implementors that the spec is "done" and will not continue to
> > mutate. As you might recall, this particular spec has changed quite
> > a lot over its history, more than many, and I wouldn't blame an
> > implementor for standing back and waiting for the dust to settle
> > before writing code.
> > 
> > As we all know (and have a history of doing, as a WG) if post-WGLC
> > implementation experience indicates a need for changes, the draft
> > can and will be updated to take them onboard, and of course if those
> > changes are substantive enough, I would expect a new WGLC would be
> > needed at that point. Other groups do this same
> > review/implementation cycle by progressing things to RFC, then
> > implementing, then bis-ing or otherwise updating the RFCs. I am not
> > (again, as a WG member) a fan of that, and I'm glad we don't do
> > things that way, although our way is also not perfect as witness
> > this discussion.
> > 
> > Fundamentally a WGLC is a way of asking the WG, "the authors think
> > this is done, do you agree?" We are asking. "We have
> > implementations" is a strong argument to support the "we are done"
> > argument, but it's neither necessary nor sufficient and so far the
> > WG has deemed it desirable but not required for WGLC. We think there
> > would be no benefit, and some harm, from letting the document sit in
> > "active WG draft" status when it's *not* in fact active and the
> > authors have moved on to other things. Moving it to "waiting for
> > implementation" status, if the results of the WGLC support that, is
> > more honest and more clear. I suppose another way of doing this
> > would be to invent some IDR pre-WGLC-WGLC that feeds into the
> > "waiting for implementation" document state, but right now we don't
> > have that.
> > 
> > Of course you and any other WG member are free to use all the facts
> > at your disposal (including the document itself, and including any
> > reported implementation) to decide if you want to support a WGLC,
> > and that's fine.
> > 
> > Finally I'd like to ask (and this *is* with the co-chair hat on)
> > that if we continue a wide-ranging philosophical discussion of the
> > IDR implementation tradition as opposed to a discussion of whether
> > the doc is ready for WGLC, that we change the subject line
> > accordingly.
> > 
> > Thanks!
> > 
> > --John
> > 
> > [*] "Do I contradict myself? Very well, then I contradict myself, I
> > am large, I contain multitudes." --Walt Whitman
> > 
> > 
> > _______________________________________________
> > Idr mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/idr
> >

---

From: Robert Raszuk <[email-masked]>
Date: Mon, 4 Jun 2018 10:38:33 +0200

All,

Last year between April and July 2017 there was over 250 emails exchanged
on this topic and none of the problems pointed out nor alternative
solutions were addressed (leave alone incorporated) by the authors.

Is this some sort of new IDR strategy that if a draft receives a push back
from WG it needs to go on slience for 1 year then comes back hoping that no
one cares any more and it will smoothly go through the last call ?

To bring back some points:

* It has been demonstrated with real RS data that vast majority of RS carry
single path for given prefix. This draft will not help in such cases.

* If there is second path available for a given prefix the simple solution
is to observe that most IXes offer two Route Servers. Therefor it is
trivial to make such secondary RS to distribute second best path ahead of
any failure (knob for this does exist already in number of shipping
implementations) - so this would be one line config change on RS and no
upgrade(s) to the IX clients needed.

* It has also been pointed out that distributing 2 or even 3 or more paths
with add-paths will not cause any customer CE meltdown. So simply configure
"add-paths 2" on those clients who need two paths from single RS and be
done.

* Let's not forget the bigger picture here. IX peering is an optimization.
To provide redundancy across IX with failing fabric connectivity, different
IX peering can be used or destinations can be reached over native Internet
path.

* Last getting different path does not guarantee any level  success if IX
fabric failure location is close to the client.

Kind regards,
RR

---

From: Nick Hilliard <[email-masked]>
Date: Tue, 5 Jun 2018 22:06:30 +0100

Robert,

Robert Raszuk wrote on 04/06/2018 09:38:
> * It has been demonstrated with real RS data that vast majority of RS 
> carry single path for given prefix. This draft will not help in such cases.

the bfd brokerage aspect of draft-ietf-idr-rs-bfd will help enormously 
in this case.

> * If there is second path available for a given prefix the simple 
> solution is to observe that most IXes offer two Route Servers. Therefor 
> it is trivial to make such secondary RS to distribute second best path 
> ahead of any failure (knob for this does exist already in number of 
> shipping implementations) - so this would be one line config change on 
> RS and no upgrade(s) to the IX clients needed.

offering inconsistent views from a route server cluster is difficult. 
The best thing for an ixp to do is to provide a consistent and 
predictable service.

> * It has also been pointed out that distributing 2 or even 3 or more 
> paths with add-paths will not cause any customer CE meltdown. So simply 
> configure "add-paths 2" on those clients who need two paths from single 
> RS and be done.

In theory, yes.  In practice, there are very few ebgp add-path 
implementations.  Also in practice, if you want fail failover, you need bfd.

> * Let's not forget the bigger picture here. IX peering is an 
> optimization. To provide redundancy across IX with failing fabric 
> connectivity, different IX peering can be used or destinations can be 
> reached over native Internet path.

yes, but in order to use that alternative path, the client network needs 
to detect the failure at the IXP, which is what this draft is trying to 
achieve.

> * Last getting different path does not guarantee any level  success if 
> IX fabric failure location is close to the client.

this point is out of scope for this discussion, which deals with 
arbitrary failures of the ixp fabric.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 5 Jun 2018 23:24:35 +0200

Hi Nick,​​

> In theory, yes.  In practice, there are very few ebgp add-path
implementations.

Are you saying that enabling add-paths for eBGP especially for
route-server-client neighbors is going to be any harder then coming with
new SAFI  ? This draft requires OS upgrade and most if not all router
vendors support iBGP add-paths today. If the draft would spell such
alternative solution it would be brilliant.

> Also in practice, if you want fail failover, you need bfd.

And what is stopping you to use it since you have next hops anyway ? Ref:
https://tools.ietf.org/html/draft-chen-bfd-unsolicited-02

Thx,
R.

On Tue, Jun 5, 2018 at 11:06 PM, Nick Hilliard <[email-masked]> wrote:

> the bfd brokerage aspect of draft-ietf-idr-rs-bfd will help enormously in
> this case.
>
> offering inconsistent views from a route server cluster is difficult. The
> best thing for an ixp to do is to provide a consistent and predictable
> service.
>
> this point is out of scope for this discussion, which deals with arbitrary
> failures of the ixp fabric.
>
> Nick
>
>

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Tue, 5 Jun 2018 22:43:03 +0000

Dear Authors,

I was suggested to refresh ietf-idr-bgp-bestpath-selection-criteria draft, which should be a normative reference in this draft. Could you please confirm that next version would reflect that correctly?

--
Cheers,
Rajiv Asati
Distinguished Engineer, Cisco

From: Idr <[email-masked]> on behalf of Nick Hilliard <[email-masked]>
Date: Tuesday, June 5, 2018 at 5:07 PM
To: "[email-masked]" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, Hares Susan <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Idr] draft-ietf-idr-rs-bfd-05 - 2 Week WG LC from 4/13 to 4/27

Robert,

Nick

_______________________________________________
Idr mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/idr

---

From: Nick Hilliard <[email-masked]>
Date: Wed, 6 Jun 2018 11:27:33 +0100

Robert,

I can't help feeling that you're throwing things at this draft to see 
what sticks.

Nick

Robert Raszuk wrote on 05/06/2018 22:24:
> Hi Nick,​​
> 
>> In theory, yes.  In practice, there are very few ebgp add-path implementations.  
> 
> And what is stopping you to use it since you have next hops anyway ? 
> Ref: https://tools.ietf.org/html/draft-chen-bfd-unsolicited-02
> 
> Thx,
> R.
> 
> 
> 
> On Tue, Jun 5, 2018 at 11:06 PM, Nick Hilliard <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     Robert,
> 
>         * It has been demonstrated with real RS data that vast majority
>         of RS carry single path for given prefix. This draft will not
>         help in such cases.
> 
> 
>     the bfd brokerage aspect of draft-ietf-idr-rs-bfd will help
>     enormously in this case.
> 
>         * If there is second path available for a given prefix the
>         simple solution is to observe that most IXes offer two Route
>         Servers. Therefor it is trivial to make such secondary RS to
>         distribute second best path ahead of any failure (knob for this
>         does exist already in number of shipping implementations) - so
>         this would be one line config change on RS and no upgrade(s) to
>         the IX clients needed.
> 
> 
>     offering inconsistent views from a route server cluster is
>     difficult. The best thing for an ixp to do is to provide a
>     consistent and predictable service.
> 
>         * It has also been pointed out that distributing 2 or even 3 or
>         more paths with add-paths will not cause any customer CE
>         meltdown. So simply configure "add-paths 2" on those clients who
>         need two paths from single RS and be done.
> 
> 
>         optimization. To provide redundancy across IX with failing
>         fabric connectivity, different IX peering can be used or
>         destinations can be reached over native Internet path.
> 
> 
>     yes, but in order to use that alternative path, the client network
>     needs to detect the failure at the IXP, which is what this draft is
>     trying to achieve.
> 
>         * Last getting different path does not guarantee any level 
>         success if IX fabric failure location is close to the client.
> 
> 
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/idr>
> 
>