Subject: Re: [Idr] I-D Action: draft-ietf-idr-rs-bfd-02.txt

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Sun, 16 Apr 2017 20:12:41 +0000

Hi Jeff,

>   But more importantly, you'll stop sending it toward your own peering and
>   attracting blackholed traffic.
 
Indeed. That’s key. 

And it could rely on the route resolvability condition to be appropriately modified, as described in  
https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria.

>    RS-BFD basically reinvents the same procedure in your draft, simply being
>    specific about the use of BFD as the dataplane liveness check mechanism.
>    However, as you note in the RS-BFD draft, we do leave the option for other
>    mechanisms.
>
>    I suspect the other authors of RS-BFD aren't particular where we pick up our
>    text for the resolvability condition.  However, if the suggestion is to make
>   a reference to your draft, you'll need to bring it back from zombie state
>    and progress it. :-)

Agreed. It is alive (-07 version). https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria-07
It can be used as a normative reference in your draft.	

-- 
Cheers,
Rajiv Asati
Distinguished Engineer, Cisco

-----Original Message-----
From: Jeffrey Haas <[email-masked]>
Date: Tuesday, March 14, 2017 at 5:36 PM
To: Rajiv Asati <[email-masked]>
Cc: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-ietf-idr-rs-bfd-02.txt

    Rajiv,
    
    On Tue, Mar 14, 2017 at 09:08:24PM +0000, Rajiv Asati (rajiva) wrote:
    > Is the assumption here that the client routers have routing view limited to what’s provided by the Route Server? If not, then wouldn’t Client Routers benefit from having to invalidate the path learned from the remote client router as soon as the connectivity check failed?
    
    This is what I believe the procedure says.  See section 6.
    
    > Of course, Client Routers conveying the lack of NLRI reachability per NH to the Route Server, and expecting Route Server to provide a different NHs of the NLRIs, and expecting it to be functional, while still attracting the traffic for unreachable destinations since the Loc-RIB is still pointing to the unreachable NH for the affected NLRIs.
    
    The thing that is somewhat different for a IXP environment running a route
    server than normal eBGP is the low (to zero) likelihood of having a backup
    path.  If 10/8 was learned from the route server for nexthop 192.0.2.1, and
    you stop being able to reach that nexthop, removing it from your forwarding
    (unreachable) is your only choice.
    
    You *might* have a source of that path internally.  In that case, you can
    use it.
    
    But more importantly, you'll stop sending it toward your own peering and
    attracting blackholed traffic.
    
    > I wonder whether  https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria be useful here.
    
    In a sense of good timing, John Scudder had brought this to my attention
    about an hour ago. 
    
    RS-BFD basically reinvents the same procedure in your draft, simply being
    specific about the use of BFD as the dataplane liveness check mechanism.
    However, as you note in the RS-BFD draft, we do leave the option for other
    mechanisms.
    
    I suspect the other authors of RS-BFD aren't particular where we pick up our
    text for the resolvability condition.  However, if the suggestion is to make
    a reference to your draft, you'll need to bring it back from zombie state
    and progress it. :-)
    
    -- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Sun, 16 Apr 2017 22:34:49 +0200

Rajiv,

I think that everyone fully agrees that in order to consider prefix as
valid the next hop to it should be reachable.

The only open question is how to detect it when you have no direct protocol
adj. established.

Jeff seems stuck with BFD .. but there is number of folks who see S-BFD as
much better fit to the problem. Yet UDP echo in some implementations
already does it today.

I think we need to evaluate pros and cons of all options before making any
more formal recommendation. At this point however I think 7880 fits best.

Kind regards,
R.

On Apr 16, 2017 10:12 PM, "Rajiv Asati (rajiva)" <[email-masked]> wrote:

> Agreed. It is alive (-07 version). https://tools.ietf.org/html/
> draft-ietf-idr-bgp-bestpath-selection-criteria-07
>     > Is the assumption here that the client routers have routing view
> limited to what’s provided by the Route Server? If not, then wouldn’t
> Client Routers benefit from having to invalidate the path learned from the
> remote client router as soon as the connectivity check failed?
>
>     > Of course, Client Routers conveying the lack of NLRI reachability
> per NH to the Route Server, and expecting Route Server to provide a
> different NHs of the NLRIs, and expecting it to be functional, while still
> attracting the traffic for unreachable destinations since the Loc-RIB is
> still pointing to the unreachable NH for the affected NLRIs.
>
>     > I wonder whether  https://tools.ietf.org/html/
> draft-ietf-idr-bgp-bestpath-selection-criteria be useful here.
>

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Tue, 18 Apr 2017 04:09:29 +0000

Hi Robert,

>    I think that everyone fully agrees that in order to consider prefix as valid the next hop to it should be reachable.

Indeed. 

> but there is number of folks who see S-BFD as much better fit to the problem. Yet UDP echo in some implementations already does it
> today.

UDP echo as in IP SLA?

>    I think we need to evaluate pros and cons of all options before making any more formal recommendation. At this point however I
>  think 7880 fits best.

RFC7881 to be exact (since it defines procedures for using S-BFD usage in IPv4, IPv6, and MPLS environments).
RFC7880 describes a generalized mechanism.
-- 
Cheers,
Rajiv Asati
Distinguished Engineer, Cisco

-----Original Message-----
From: <[email-masked]> on behalf of "[email-masked]" <[email-masked]>
Date: Sunday, April 16, 2017 at 4:34 PM
To: Rajiv Asati <[email-masked]>
Cc: PFRC - jhaas <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: Re: [Idr] I-D Action: draft-ietf-idr-rs-bfd-02.txt

    Rajiv,

    I think that everyone fully agrees that in order to consider prefix as valid the next hop to it should be reachable.

    The only open question is how to detect it when you have no direct protocol adj. established.

    Jeff seems stuck with BFD .. but there is number of folks who see S-BFD as much better fit to the problem. Yet UDP echo in some implementations already does it today.

    I think we need to evaluate pros and cons of all options before making any more formal recommendation. At this point however I think 7880 fits best.

    Kind regards,
    R.

    
    
    Hi Jeff,

    Indeed. That’s key.

    >   a reference to your draft, you'll need to bring it back from zombie state
    
    Agreed. It is alive (-07 version). 
    https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria-07 <https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria-07>
    It can be used as a normative reference in your draft.

    -----Original Message-----
    From: Jeffrey Haas <[email-masked]>
    Date: Tuesday, March 14, 2017 at 5:36 PM
    To: Rajiv Asati <[email-masked]>
    Cc: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
    Subject: Re: [Idr] I-D Action: draft-ietf-idr-rs-bfd-02.txt
    
        Rajiv,

    
        > Of course, Client Routers conveying the lack of NLRI reachability per NH to the Route Server, and expecting Route Server to provide a different NHs of the NLRIs, and expecting it to be functional, while still attracting the traffic for unreachable destinations
     since the Loc-RIB is still pointing to the unreachable NH for the affected NLRIs.

    
    
        > I wonder whether  
    https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria <https://tools.ietf.org/html/draft-ietf-idr-bgp-bestpath-selection-criteria> be useful here.

    
    
        -- Jeff

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 18 Apr 2017 16:31:08 -0400

Robert,

On Sun, Apr 16, 2017 at 10:34:49PM +0200, Robert Raszuk wrote:
> much better fit to the problem.

Since you brought it up to the main mailing list, who are the other folks
besides yourself that thinks S-BFD is a better fit?

You're going to make me write that largeish e-mail that doesn't help the
discussion much about why S-BFD is not a good fit, aren't you?

> Yet UDP echo in some implementations
> already does it today.

As noted previously, the draft does permit for alternate means beyond BFD.
However, we have to pick one.  Standardizing ping is likely a bad idea. :-)

-- Jeff

---

From: Robert Raszuk <[email-masked]>
Date: Tue, 18 Apr 2017 22:36:56 +0200

Hi Jeff,

> besides yourself that thinks S-BFD is a better fit?

​I will let them speak themselves if they care ...​

> discussion much about why S-BFD is not a good fit, aren't you?

​I am not going to make you do anything however since we are discussing
how best to detect NH liveness in NBMA env. we should examine all options
vs requirements for those.  ​

And one of the requirements as you have heard from at least one customer is
to test MTU of the path to such BGP next hops. Is RFC5880 BFD really best
tool for that ?

> However, we have to pick one.  Standardizing ping is likely a bad idea. :-)

​What is there to standardize ? RFC862 seems like pretty good standard
already.

Ref: https://tools.ietf.org/html/rfc862

And implementing it in hardware should not be that hard too for those
concerned
not to go to RE/RP with each packet. .

Thx,
R.
​

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 20 Apr 2017 12:31:19 +0100

> to test MTU of the path to such BGP next hops. Is RFC5880 BFD really best 

All IXPs have published MTUs and someone attempts to connect to an IXP
with the expectation that using a different MTU is going to cause
anything other than complete brokenness, then I'd tactfully suggest an
alternative career path.

> ​What is there to standardize ? RFC862 seems like pretty good standard

With sufficient thrust, pigs fly just fine.

ICMP is the wrong tool in the same way that ARP request/reply is also
the wrong tool for this.  It's rubbish for this purpose because it's
usually highly deprioritised on routers, unlike bfd which is often
fast-pathed and carefully controlled.  BFD is fit for this purpose
because it's designed specifically and is supported by router vendors
for exactly this purpose.

Fast liveliness detection cannot be pawned off to an arbitrary protocol
just because that protocol replies to packets, in the same way that we
don't exchange routes over xml in https or implement ssh using udp/53,
or plough fields using a modified toyota yaris.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 21 Apr 2017 00:17:41 +0200

Nick,

I am very happy for you that your experience with published static MTU
values is so great. Well if you go via your own switches with dark fiber I
would not expect this to be any different.

If however you operate a distributed IX and use leased circuits from other
carriers between your access switches you may be very surprised that
suddenly your MTU get's reduced due to underlying carrier reroute to
different links or applying say MPLS link protection.

We do see this occurring more and more these days and it is nasty to
troubleshoot too if you have no proper tool running. So tactfully I
recommend we do not close our minds to only those problems which one have
seen in his own shop.

On the topic of ICMP or ARP it is your choice. I prefer to know that my
upstream is down in say 2 sec rather then suffer from broken Internet link
forever as my upstream provider will not run BFD session with an office
fiber tail.

Cheers,
R.

---

From: "i3D.net - Martijn Schmidt" <[email-masked]>
Date: Fri, 21 Apr 2017 00:32:13 +0200

Robert, 

Distributed IX operators typically use dark fiber or DWDM wavelengths as underlying circuits in their VPLS fabric for exactly this reason. If they do cobble together their network with various 3rd party, say, "ethernet solutions", I doubt they'll have a good reputation in the market for long. 

Best regards, 
Martijn 

On 21 April 2017 00:17:41 CEST, Robert Raszuk <[email-masked]> wrote:
>> > And one of the requirements as you have heard from at least one
>customer
>> is
>> >     However, we have to pick one.  Standardizing ping is likely a
>bad
>> >     idea. :-)
>> >

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 21 Apr 2017 00:38:39 +0200

Yes within a city you can use it the way you describe, But between
countries it would be rather expensive option don't you think.

Moreover I have 15000 CEs WAN and do see those issues in real. So the point
is that it would be great to come with one good solution for such problems
rather then again zoo of new features - especially since we would like
those to be implemented in line card's hardware.

Yes I know the draft name is rs-bfd ... but the problem at hand is bigger
then IX environments.

Thank you,
R.

On Fri, Apr 21, 2017 at 12:32 AM, i3D.net - Martijn Schmidt <
[email-masked]> wrote:

> Robert,
>
> Distributed IX operators typically use dark fiber or DWDM wavelengths as
> underlying circuits in their VPLS fabric for exactly this reason. If they
> do cobble together their network with various 3rd party, say, "ethernet
> solutions", I doubt they'll have a good reputation in the market for long.
>
> Best regards,
> Martijn
>
>> If however you operate a distributed IX and use leased circuits from
>> other carriers between your access switches you may be very surprised that
>>> customer is

---

From: Nick Hilliard <[email-masked]>
Date: Thu, 20 Apr 2017 23:50:24 +0100

Robert Raszuk wrote:
> values is so great. Well if you go via your own switches with dark fiber
> I would not expect this to be any different. 
> 
> other carriers between your access switches you may be very surprised
> that suddenly your MTU get's reduced due to underlying carrier reroute
> to different links or applying say MPLS link protection. 
> 
> recommend we do not close our minds to only those problems which one
> have seen in his own shop.

So what you're saying is that if someone builds infrastructure which is
identifiably unfit for its intended purpose, it might break in nasty but
entirely predictable ways.

At this stage, I think it's important to say that this draft is not
about MTUs, and that if you have core MTU problems on your inter-domain
l2 network, you have problems which are both bigger and not particularly
related to this draft.

> upstream is down in say 2 sec rather then suffer from broken Internet
> link forever as my upstream provider will not run BFD session with an
> office fiber tail.

We're talking about bfd session brokering via route servers in
inter-domain routing situations, not residential / b2b access models.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 21 Apr 2017 00:54:46 +0200

Yes I do realize that.

But my point is that we can come up with single solution to both problems
which would work in a hardware accelerated way equally well. I am also
observing that MTU issue is applicable to long distance stretched IXes
which as I am sure you are aware is a real IX service offering by number of
IX operators today.

//R

---

From: Nick Hilliard <[email-masked]>
Date: Fri, 21 Apr 2017 09:57:02 +0100

Robert Raszuk wrote:
> But my point is that we can come up with single solution to both
> problems which would work in a hardware accelerated way equally well.

Then I suggest you author documents to:

- add a new icmp type to define reachability / mtu detection / etc mechanism
- replicate whatever bits of BFD you want to port over
- get widespread vendor support

Once you've done all this, and not before, then please feel free to
write an update to draft-ietf-idr-rs-bfd rfc to include your new
reachability mechanism.

Or you could update bfd to support mtu detection.

> I
> am also observing that MTU issue is applicable to long distance
> stretched IXes which as I am sure you are aware is a real IX service
> offering by number of IX operators today. 

IXP core MTU monitoring is orthogonal to draft-ietf-idr-rs-bfd.  IXP
edge MTU monitoring is the responsibility of the ixp edge participant.

If you are operating a private infrastructure in your company which
amalgamates these roles and where you have a need to monitor for MTU
changes, then this is a private problem which can be easily solved using
any one of a plethora of off-the-shelf tools and there is no reason
whatsoever for the ietf to standardise a new protocol to solve this
corner case.

I'm not going further down this rat-hole because it has nothing to with
this draft.

Nick

---

From: Robert Raszuk <[email-masked]>
Date: Fri, 21 Apr 2017 11:01:20 +0200

The document is written, accepted, last called and published as Standard
Track RFC.

Ref: RFC 7880.

> If you are operating a private infrastructure

I am building private secure infra over public IXes. Hint: SD-WAN MTU
stability over IX matters to me.

Best,
R.

On Fri, Apr 21, 2017 at 10:57 AM, Nick Hilliard <[email-masked]> wrote:

> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr
>

---

From: Mikael Abrahamsson <[email-masked]>
Date: Fri, 21 Apr 2017 11:08:38 +0200 (CEST)

> - add a new icmp type to define reachability / mtu detection / etc mechanism

https://tools.ietf.org/html/draft-van-beijnum-multi-mtu-05

I am interested in working in this area, however IDR doesn't seem to be 
the ideal place to do that. I'd prefer to do it in INT-AREA (where I 
presented I van Beijnums work a few IETFs back.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Jeffrey Haas <[email-masked]>
Date: Tue, 25 Apr 2017 16:38:08 -0400

> Or you could update bfd to support mtu detection.

Note that there's been prior art in this problem space:
https://tools.ietf.org/html/draft-haas-xiao-bfd-echo-path-mtu-01

However, I do suggest it's worth having that conversation in
[email-masked].

-- Jeff