Subject: Re: [babel] Mirja Kühlewind's Discuss on draft-ietf-babel-rfc6126bis-12: (with DISCUSS and COMMENT)

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 5 Dec 2019 12:50:37 +0100

Hi Juliusz, hi all,

I took another look at the updated RFC and I have to say I’m still concerned about the default values (which is also an issue that was also raised by Alvaro) and the appendix B.

You know added another case:

"The following values are recommended in a network where reconvergence
   within 2 seconds after a mobility event is desired:

      Multicast Hello Interval: 0.5 seconds.

      Unicast Hello Interval: infinite (no Unicast Hellos are sent).

      Link cost, IHU Interval, Update Interval, IHU Hold Time, and Route
      Expiry Time: computed as in the first case above.

      Request Timeout: initially 0.5 seconds, doubled every time a
      request is resent, up to a maximum of three times.

      Urgent timeout: 0.2 seconds.

      Source GC time: 3 minutes.”

These value are much lower than in the first case and can easily leads to signification congestion, however, you don’t say anything appropriate about when those value might be used or should be avoided. What appendix B does is simply given some sample calculation about what the expected reconvergence time is based on the selected values but it does not give any guidance about which network conditions would bear which setup, nor does it talk about negative implication of selecting lower values.

I still think that it is important to provide default values as part of the normative specification. For me finding such default value (maybe with more discussions about in which network scenarios these vales makes sense and when it might be useful to consider different values) was part of the experiment with babel and if you are not able to address these questions, babel is not ready for PS. 

Alternatively you can check the values in RFC8085 again and see if these are acceptable values for most scenarios. Values in that RFC are lightly lower than the first example (e.g. 3 secs interval and 1 sec timeout) but quite a bit larger than the new example you introduced (without further guidance on usage of the values).

Especially in section 3.8.2.1. regarding the Request Timeout it is not sufficient to only point to the appendix. Please also note that there are actually two values here that need suitable default values, the time-out itself and the number of retries.

Thanks!
Mirja

> On 18. Oct 2019, at 18:46, Juliusz Chroboczek <[email-masked]> wrote:
> 
> Dear Mirja,
> 
>> 1) Let's move appendix B into the body of the document. I think these
>> are important information for an implementer and therefore it needs more
>> attention.
> 
> I am sorry, but I disagree.  The original document that I submitted for
> IETF review back in 2009 had the structure that you suggest, but the
> reviewer (Joel Halpern) convinced me to a adopt the following structure:
> 
>  - the body of the document describes the main algorithm, the parts that
>    are required to enforce the strong guarantees that the protocol makes
>    about lack of loops and forward progress;
>  - the appendices describe algorithms that have been shown to be useful
>    over some link layers but that might need to be tweaked when Babel is
>    run over new, exciting link layers.
> 
> Since Joel did a good job convincing me, it will take some work to
> convince me otherwise.
> 
>> 2) Maybe you can add something like: [...]  I’m sure you have a better
>> wording. My intention is simply to add another warning, to think
>> carefully instead of just pick some default.
> 
> I don't necessarily agree, but I've done as you request.
> 
>>>> Thanks. I was also hoping that you could make any recommendation on how
>>>> to implement that e.g. a fixed delay of a certain (default) value or
>>>> based on some other knowledge. [...]
> 
>>> You do realise it's a very high bar you're setting?
> 
> [...]
> 
>> I wasn’t looking for a tutorial for pacing but rather a reference to
>> add. E.g. just naming different approaches for how to implement pacing
>> and provide a reference would be a good thing I think. However, I will
>> not block on this one thing. Your choice.
> 
> I am not aware of any good reference on packet pacing that covers IEEE
> 802.11 multicast (an important usecase for Babel).  After checking some
> papers and scratching my head for a long time, I've just added another
> sentence to Section 3.1; I'm afraid I'm not competent to do anything more.
> 
>> I just wondering why you don’t call them default value rather then
>> examples. I think specifying default value normatively would be more
>> clear and is what we usually do in specs. However, I think in this case
>> it will not make a huge practical difference.
> 
> Great, let's agree that we're not in full agreement, then.
> 
> -- Juliusz
> 
> 
>

---

From: Mirja Kuehlewind <[email-masked]>
Date: Thu, 5 Dec 2019 12:53:19 +0100

Hi again,

And one more comment regarding pacing: Again I will not block on this issue anymore, however, as you have a babel implementation I would assume it would be easy to add more details on how pacing is done in your implementation in the draft (at least in the appendix).

Mirja

> On 5. Dec 2019, at 12:50, Mirja Kuehlewind <[email-masked]> wrote:
> 
> Hi Juliusz, hi all,
>

---

From: David Schinazi <[email-masked]>
Date: Tue, 10 Dec 2019 17:15:17 -0800

Hi Mirja,

Thank you for taking another look at the document.

I hope we can get this resolved quickly, as we have
already waited for 49 days for your last email.
I understand that you are busy but I did want to point
out that this is adding stress on the authors.

First, it appears that your review is still treating Babel like
a protocol running over UDP on the Internet. It is not.
Babel only operates point-to-point using IPv6 link-local
addresses, and the congestion properties of 802.11
are very different from the ones of an end-to-end
transport running over the wide area.

Second, the Babel working group prides itself with strict
adherence to the running code principle. Features or
recommendations do not make it into a Babel spec
unless they have been implemented and tested.

If we were discussing a transport protocol, I would
agree with your review. RFC 8085 is a great
document, but it wasn't written with link-local routing
protocols in mind.

As transport AD, it is your responsibility to ensure
new IETF protocols do not bring about another
congestion collapse, and I think that's critical.
I want to really thank you for the hours you spend
reviewing documents to ensure that goal. But I don't
think all of that guidance applies here.

Could I kindly ask you to take another look at your
review in this lens, and please let us know in what
way you think this draft has real-world issues?

We'd really like to make forward progress here, if
the only issue is default values, I'm not sure this is
worth causing delays on, as we haven't seen these
cause problems when implementing the protocol.

Thanks,
David

On Thu, Dec 5, 2019 at 3:53 AM Mirja Kuehlewind <[email-masked]> wrote:

> Hi again,
>
> And one more comment regarding pacing: Again I will not block on this
> issue anymore, however, as you have a babel implementation I would assume
> it would be easy to add more details on how pacing is done in your
> implementation in the draft (at least in the appendix).
>
> Mirja
>
>
>
> > I took another look at the updated RFC and I have to say I’m still
> concerned about the default values (which is also an issue that was also
> raised by Alvaro) and the appendix B.
> >
> > These value are much lower than in the first case and can easily leads
> to signification congestion, however, you don’t say anything appropriate
> about when those value might be used or should be avoided. What appendix B
> does is simply given some sample calculation about what the expected
> reconvergence time is based on the selected values but it does not give any
> guidance about which network conditions would bear which setup, nor does it
> talk about negative implication of selecting lower values.
> >
> > I still think that it is important to provide default values as part of
> the normative specification. For me finding such default value (maybe with
> more discussions about in which network scenarios these vales makes sense
> and when it might be useful to consider different values) was part of the
> experiment with babel and if you are not able to address these questions,
> babel is not ready for PS.
> >
> > Alternatively you can check the values in RFC8085 again and see if these
> are acceptable values for most scenarios. Values in that RFC are lightly
> lower than the first example (e.g. 3 secs interval and 1 sec timeout) but
> quite a bit larger than the new example you introduced (without further
> guidance on usage of the values).
> >
> > Especially in section 3.8.2.1. regarding the Request Timeout it is not
> sufficient to only point to the appendix. Please also note that there are
> actually two values here that need suitable default values, the time-out
> itself and the number of retries.
> >
> > Thanks!
> > Mirja
> >
> >

---

From: Mirja Kuehlewind <[email-masked]>
Date: Wed, 11 Dec 2019 15:55:05 +0100

Hi David,

Sorry for the delays (but there was also a large delay by the author in between). 

I don’t necessarily want to impose anything that is written in RFC8085 here, but what I’m saying it that the protocol in order to go to PS must given recommendations for default values or at a minimum discuss which values are appropriate in which situation (and what the consequences are otherwise). Due to the lack of these guidance I keep pointing at RFC8085 because that’s what the IETF agreed to be “safe” in the general case.

Currently the document proposes some values but it does not discuss in which scenario these values are appropriate to use (or when they are not appropriate to use anymore). Given people tend to simply implement default values, I would prefer to see a set of default values that is suitable for most scenarios and basically safe to use in all scenarios. If that is not possible it would also be okay to discuss the boundary of the values based on certain network conditions but usually that is more complicated. 

As you say, you have an implementation-driven approach. That is great and what we usually try to do in the IETF. For PS it is however usually expected that you have a running implementation as well as also some extend of operational experience (I guess that it what the experimental phase was for). If you have that experience it should be possible to e.g. to describe the network scenario that you used babel in and give the values that have been used in that scenario. What are those scenarios?

Mirja

> On 11. Dec 2019, at 02:15, David Schinazi <[email-masked]> wrote:
> 
> Hi Mirja,
>

---

From: Juliusz Chroboczek <[email-masked]>
Date: Tue, 17 Dec 2019 17:43:39 +0100

Dear Mirja,

> I keep pointing at RFC8085 because that’s what the IETF agreed to be
> “safe” in the general case.

I refer you to my mail of 9 August 2019, where I reply to almost exactly
the same statement.  RFC 8085 discusses what is considered recommended for
protocols that run over the Internet.  To witness:

   Internet paths can have widely varying characteristics, including
   transmission delays, available bandwidths, congestion levels,
   reordering probabilities, supported message sizes, or loss rates.
   Furthermore, the same Internet path can have very different
   conditions over time.

In Babel, like in OSPF, there is no "Internet path" involved -- there is
just an association between two direct neighbours (from the point of view
of layer 3).  It is therefore my opinion -- already stated in my mail of
August 9 --, that obeying the recommendations in RFC 8085 is not required
for Babel to become PS.

In the same mail of 9 August 2019, I make the claim that there is no
mechanism related to RFC 8085 in RFC 5340.  In other words, it would
appear that you are holding Babel to a much higher standard than OSPFv.

If that is the case, then I request that you state explicitly why you
think that Babel must be held to a much higher standard than OSPFv3.  If
that is not the case, then please clear your discuss.

> Currently the document proposes some values but it does not discuss in
> which scenario these values are appropriate to use (or when they are not
> appropriate to use anymore).

Appendix B gives values that are suggested for implementations.

> If you have that experience it should be possible to e.g. to describe
> the network scenario that you used babel in and give the values that
> have been used in that scenario. What are those scenarios?

The first set of parameters in Appendix B gives the values that are used
by recent versions of babeld (the reference implementation of Babel) and
that have been used in production for many years in a wide range of
networks, including wifi meshes, wired networks, and large community
networks consisting of a complex mixture of wireless and wired links.
These scenarios are described in the Applicability statement.

-- Juliusz

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 17 Dec 2019 18:13:43 +0100

Hi Juliusz,

Please see below.

> On 17. Dec 2019, at 17:43, Juliusz Chroboczek <[email-masked]> wrote:
> 
> Dear Mirja,
> 

You are citing this sentence out of context. I didn’t say that you have to stick to these values but (if you don’t want to do that) you have to provide reasonable values in the spec and discuss when it is safe to use low values.

> August 9 --, that obeying the recommendations in RFC 8085 is not required

Yes it’s  not an Internet path but you still want to design a protocol that is safe to deploy, as you say, in various scenarios and therefore you should build some safety measures to not overload the network.

> that is not the case, then please clear your discuss.

In OSPFv3 as far as I can see (and I am really not an expert here) all values that define intervals are in seconds which leads to a minimum value of 1 seconds. This is not the case for babel and therefore we need to be more carful here.

> Appendix B gives values that are suggested for implementations.
Yes but it doesn’t discuss for with _deployment_ scenarios these values are suitable.

> These scenarios are described in the Applicability statement.

First it would be helpful to then provide a pointer to that document. I assume you means draft-ietf-babel-applicability right? Then it would be nice to be more concrete. Did you deploy the same parameters in all of the discussed scenarios. Does it actually makes sense to use the same parameters in a "small unmanaged network" as well as in a "large overlay network"? And what does e.g. a small unmanaged network means? How many devices do you usually have in such a network to call it “small”? Is that network connected to the Internet or not?

---

From: Juliusz Chroboczek <[email-masked]>
Date: Tue, 17 Dec 2019 20:04:00 +0100

> In OSPFv3 as far as I can see (and I am really not an expert here) all
> values that define intervals are in seconds which leads to a minimum
> value of 1 seconds. This is not the case for babel and therefore we need
> to be more carful here.

After a change of topology, OSPF executes the procedure described in
Section 4.5.2 of RFC 5340 (Section 13.3 of RFC 2328).  At this point,
a potentially unbounded number of LSAs are flooded out every active
interface (point 5 in Section 13.3 of RFC 2328).  This flooding is not
regulated by any timer -- all that the protocol says is that the LSAs must
go out through a certain subset of the router's interfaces.

I could be wrong, but I am fairly confident that nothing in RFCs 5340 and
2328 specifies congestion control, flow control or packet pacing for this
flooding procedure.  My interpretation is that since OSPF is designed to
be deployed on local links, congestion caused by OSPF is not likely to
endanger the Global Internet, and therefore packet pacing (if required) is
left to the implementation.

I also don't recall any discussion of flow control in Moy's and Gredler's
books, but they have a lot of pages (especially Gredler's), so, again,
I could be wrong.

Where you are right, Mirja, is that packet pacing is a very difficult
problem in OSPF, where incorrectly delaying flooding could lead to
microloops; since Babel has a loop-avoidance mechanism built-in, it can
safely perform packet pacing should the need to do so be demonstrated in
a particular topology.

In summary, while I agree that packet pacing for Babel would be an
interesting research project, I maintain that by requiring packet pacing
as a condition for publication as PS, you are requiring an unreasonably
high standard, certainly a standard that neither OSPF nor IS-IS are held to.

> Yes but it doesn’t discuss for with _deployment_ scenarios these values

As far as I am aware, all current production deployments use the first set
of constants.  The one exception is an experimental testbed used for
mobility research, that uses the (more aggressive) second set of constants.

> Did you deploy the same parameters in all of the discussed scenarios.

Yes.

> Does it actually makes sense to use the same parameters in a "small
> unmanaged network" as well as in a "large overlay network"?

Yes.

> Is that network connected to the Internet or not?

Yes, but that's irrelevant -- the Babel traffic is link-local, and does
not go outside the local routing domain.

-- Juliusz