Subject: [Idr] 2 Week WG Adoption call for draft-ymbk-idr-rs-bfd-00 (3/2/2015 to 3/16/2015)

From: "Jerome Durand (jerduran)" <[email-masked]>
Date: Sun, 15 Mar 2015 20:56:52 +0000

Dear all,

I think draft-ymbk-idr-rs-bfd-00.txt addresses real issues that need to be solved somehow and I’m happy to see work conducted on the topic.

However the approach chosen implies that IXP members rely on a mechanism to be implemented on the route-server, while we believe a direct approach could be simpler and quicker to implement, and much more reliable. David Freedman and I have worked on finding a solution for quite some time now and we are proposing an approach based on BFD, which can be found here:

https://tools.ietf.org/id/draft-jdurand-auto-bfd-00.txt

The Idea is simple and it only requires a basic BFD capability between IXP members. Based on the BFD session state of the learnt BGP next-hop, routes can be accepted and BFD backed by a member, the decision remains local.

While we elaborated this document we discovered that there are real challenges in the BFD implementation, for example when there is asymmetric routing. This is documented in our draft. I believe on the other hand that there is a lack of this information in draft-ymbk-idr-rs-bfd-00 at this stage. we would like to see more information on that in future versions.

Thank you,

Jerome Durand
and 
David Freedman

---

From: Thomas Mangin <[email-masked]>
Date: Mon, 16 Mar 2015 08:56:15 +0000

Hello Jerome,
Hello David,

Thank you for this work.

Many networks will not use Route Servers due to this problem which did
hit quite a few operators over the years on large internet exchanges.
This is a reason why some large operators refuse to peer with smaller
ones via RS.

The first time I heard about the idea, I thought that this mechanism was
an elegant solution to prevent traffic black-holing.

This draft means that a network should be able to prevent loss on
outgoing traffic. However should the peer not have the feature enabled,
return traffic could still suffer: BFD notice the session is down, the
routes are removed from the FIB, traffic goes via another path, but
still tries to come back via the dead L2 as the peer does not implement
the feature. Was it what you had in mind when speaking of asymmetric
routing, or was it something else ?

This case does not remove any of the merit of this draft, operators are
still better with it than without, but seems to indicate that other
mechanisms could be considered to complement it.
Naively, I would thing that something which would for example allow the
tagging of routes with no-export to the RS(or a removal) should a number
of peers on a same L2 suddenly became unreachable due to BFD be
acceptable in a RS scenario.

Sincerely,

Thomas
> _______________________________________________
> Idr mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/idr

-- 

Thomas Mangin
Director - Exa Networks
http://exa.net.uk/about/contact-us

---

From: David Freedman <[email-masked]>
Date: Mon, 16 Mar 2015 09:20:15 +0000

>This is a reason why some large operators refuse to peer with smaller

Yes, I can speak as such an operator, our default policy is not to use RS after
suffering repeated L2 partitioning on a particular exchange many years ago.

>return traffic could still suffer: BFD notice the session is down, the

The idea in this draft requires both sides to support it, the BFD sessions
are set up from both sides, toward the other side (i.e really bi-directional) 
we don't care about the RS other than for a BGP session to bind the next
hop discovery to. 

>acceptable in a RS scenario.

I know of an operator which has set up a number of static BFD sessions
across an exchange, and has a script, which, if a number of these
are seen to go down, shuts down the RS session.

Dave.

---

From: Randy Bush <[email-masked]>
Date: Mon, 16 Mar 2015 11:59:51 +0100

hi david,

clue bat, please.  how does your scheme tell the rs to give me next best
paths instead of those over the failing link?

how likely are we to get the bfd hacks from C, J, A, B, ...?  in our
draft, we are trying to minimize dependence on vendor hacks for a
reason.  their frelling bfds are not even fully compatible today.

but i definitely appreciate this discussion as opposed to reading the
completely engineering-free discussions on the ietf list etc.  and i
look forward to discussing this with you in dallas.

randy

---

From: David Freedman <[email-masked]>
Date: Mon, 16 Mar 2015 11:08:38 +0000

>paths instead of those over the failing link?

It doesn't , but then, if the exchange is having issues, I won't be
sending 
any further traffic over it until it is repaired, so I would stop trusting
the RS at this point.

>reason.  their frelling bfds are not even fully compatible today.

Being that we are already in 'hack' territory
(http://seclists.org/nanog/2015/Feb/259)
This one IMHO requires the least protocol changes (we just register a new
diagnostic flag
from an existing IANA FCFS registry), no changes needed to BGP or the RS
implementation.

Saying this, it is useful for the RS operator to be aware that there is a
problem
(but then again, being that in almost all cases they run the exchange
itself
and , you would hope, have monitoring, I guess they would already be so)

>completely engineering-free discussions on the ietf list etc.

As much as I would love to be there, I can't, but I believe Jerome is
sending a Rep.

Dave.

---

From: Randy Bush <[email-masked]>
Date: Mon, 16 Mar 2015 12:39:38 +0100

as you know, i also have not trusted route servers.  arnold asked what
would allow me to extend a little trust.  draft-ymbk-idr-rs-bfd is an
attempt.

>> clue bat, please.  how does your scheme tell the rs to give me next
>> best paths instead of those over the failing link?
> sending any further traffic over it until it is repaired, so I would
> stop trusting the RS at this point.

we see many cases of small inter-peer failures where other links are
just fine.  it can be an interface, a router-to-switch cable, ...  this
is why we want to tell the rs to back off only the failing link.

> This one IMHO requires the least protocol changes (we just register a
> new diagnostic flag from an existing IANA FCFS registry), no changes
> needed to BGP or the RS implementation.

your draft requires changes to bfd.  ymbk is trying to avoid that.  a
hack to bird is something we can do ourselves, does not rely on the
vendors.

but this is a call for adoption.  imiho, the wg should adopt both drafts
so we can sort it in the wg.

randy

---

From: Jeffrey Haas <[email-masked]>
Date: Mon, 16 Mar 2015 13:04:41 -0400

[Responding from the top of the thread intentionally.]

On Sun, Mar 15, 2015 at 08:56:52PM +0000, Jerome Durand (jerduran) wrote:
> Dear all,
> 
> I think draft-ymbk-idr-rs-bfd-00.txt addresses real issues that need to be solved somehow and I?m happy to see work conducted on the topic.
> 

Things I like about your draft:
- Autodiscovery of the endpoint for BFD just by gleaning the nexthops.

Things that need a bit of clarification:
- What procedures do you do if the remote side doesn't support BFD? 
- What about if you see the remote nexthop but the remote nexthop doesn't
  see you?

Things I don't like:
- You're trying to use BFD diag codes without state changes as state for the
  session.  Wearing my BFD chair hat, I have some reservations about this.

The point raised in the thread about draft-ymbk is something I think is
worth addressing: Give the RS a chance to unhide paths if you can't use
them. 

All of this said, I think the core behavior of getting nexthop validation
via BFD is the key property that gets the best improvement.  I think this
gives us a basis to refine both proposals.

-- Jeff

---

From: Randy Bush <[email-masked]>
Date: Tue, 17 Mar 2015 11:35:55 +0100

> - Autodiscovery of the endpoint for BFD just by gleaning the nexthops.

in reality, often A receives some paths from B but B does not send any
paths to A.  so you will need a bit more for A to 'discover' the next
hop.

randy

---

From: David Freedman <[email-masked]>
Date: Tue, 17 Mar 2015 13:51:26 +0000

>paths to A.  so you will need a bit more for A to 'discover' the next

That's actually covered in the draft, when the BFD session transitions to
up (nod to Jeff here)
we set a diagnostic flag explaining that we did this because we saw some
routes (from the RS)
with next hop of the remote system. The session is established between
systems A and B when
either A has routes for B, or B has routes for A. The session stays until
both sides agree
they are not receiving routes destined for either party, at which point it
is pruned. 

Dave.

---

From: "Jerome Durand (jerduran)" <[email-masked]>
Date: Tue, 17 Mar 2015 22:11:30 +0000

Hi Jeffrey,

Thank you for taking the time to read the document and providing comments.

> - What procedures do you do if the remote side doesn't support BFD? 

Idea is that it is up to admin to decide what we wants to do with routes for any next-hop depending on its corresponding BFD session state. As an example one could decide to keep accepting routes if BFD never reaches an up state, but with a lower local-pref if desired. This point could be let to implementation. This is a good point we can try to document this in next versions.

> - What about if you see the remote nexthop but the remote nexthop doesn't

Remote next-hop would signal BFD session state in next control messages and everyone would agree that session state is now down. At this point policy can be applied on corresponding routes. We could detail how solution solves various types of problems that can be met on IXPs if this can help global understanding.

>  session.  Wearing my BFD chair hat, I have some reservations about this.

Well we do this for session ageing which we believe is crucial for scalability and manageability. We are trying to signal that BFD session is needed or not needed locally. This is different from BFD session state and that’s why we kept that at another level. In other words we say: « Session is UP but personnaly I don’t need a session with you anymore. Do you need to keep the session with me? » Idea of using diag codes is to do this signalling without changing BFD protocol definition.

Thanks,

Jerome

---

From: Thomas King <[email-masked]>
Date: Wed, 18 Mar 2015 18:54:22 +0000

Hi Thomas

> On 16 Mar 2015, at 09:56, Thomas Mangin <[email-masked]> wrote:
> 

I think this is not correct anymore:
http://net.t-labs.tu-berlin.de/~prichter/imc238-richterA.pdf

Nearly all customers at DE-CIX Frankfurt use the route servers.

I think many things happened in the past to make sure route servers run stable and reliable. With our proposal we are pushing hard in this direction. Especially, for large IXPs such as DE-CIX, AMS-IX, Linx or Netnod setting up bilateral peerings does not scale anymore (who wants to maintain 500+ BGP sessions?). Route servers are the only solution to keep the setup process manageable (from my point of view).

Best regards,
Thomas

---

From: "Neil J. McRae" <[email-masked]>
Date: Wed, 18 Mar 2015 19:37:51 +0000

> On 18 Mar 2015, at 18:54, Thomas King <[email-masked]> wrote:
> 
> Hi Thomas
> 

Thomas K - sorry but it is simple to manage, scales perfectly well and much more preferable. (And probably off topic here) 

Regards,
Neil