Subject: Re: Genart last call review of draft-ietf-detnet-architecture-08

From: János Farkas <[email-masked]>
Date: Fri, 19 Oct 2018 21:10:29 +0200

Hi Joel,

Thank you very much for your review!

Please find below in-line responses and how we plan to update the draft 
to resolve your comments.

Best regards,
Janos

On 9/22/2018 2:59 AM, Joel Halpern wrote:
> Reviewer: Joel Halpern
> Review result: Ready with Issues
>
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair.  Please treat these comments just
> like any other last call comments.
>
> For more information, please see the FAQ at
>
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>
> Document: draft-ietf-detnet-architecture-08
> Reviewer: Joel Halpern
> Review Date: 2018-09-20
> IETF LC End Date: 2018-10-03
> IESG Telechat date: Not scheduled for a telechat
>
> Summary: This document is ready for publication as a Proposed Standard
>
> I presume that the status was selected so as to avoid the need for downrefs
> when other Detnet documents normatively reference this one?
>
> Major issues: N/A
>
> Minor issues:
>      Section 3.1 states that worst case delay for priority queueing is
>      unbounded.  That does not match my understanding.  I know that DelayBound
>      DSCP behavior tightly (although, I think, not as tightly  as Detnet) limits
>      both the worst case delay and the delay variation.
Strict priority is not good enough for DetNet. A high priority packet 
may need to wait until the transmission of a lower priority packet is 
finished at an outbound port, which can cause too much uncertainties in 
the network.

>
>      It seems very odd that section 3.2.1.1 says that DetNet flows can not be
>      throttled when earlier text says that DetNet flows do have a maximum
>      bandwidth.  Buried in section 4.3.2 I find that what is meant by throttled
>      is not "enforcing a rate limit", but rather "sending congestion
>      notification to cause the source to slow down."  I think the wording about
>      "can not be throttled" both in 3.2.1.1 and 4.3.2 should be adjusted for
>      clarity.
The text will be updated to clarify.

Section 3.2.1.1:

OLD:
The primary means by which DetNet achieves its QoS assurances is to
reduce, or even completely eliminate, congestion within a node as a
cause of packet loss. Given that a DetNet flow cannot be throttled,
this can be achieved only by the provision of sufficient buffer
storage at each hop through the network to ensure that no packets are
dropped due to a lack of buffer storage.

NEW:
The primary means by which DetNet achieves its QoS assurances is to
reduce, or even completely eliminate, congestion within a node as a
cause of packet loss. This can be achieved only by the provision of
sufficient buffer storage at each hop through the network to ensure
that no packets are dropped due to a lack of buffer storage. Note that
a DetNet flow cannot be throttled, i.e., its transmission rate cannot be
reduced via explicit congestion notification.

Section 4.3.2:

OLD:
There is no provision in DetNet for throttling DetNet flows (reducing
end-to-end transmission rate via any explicit congestion
notification); the assumption is that a DetNet flow, to be useful,
must be delivered in its entirety.

NEW:
There is no provision in DetNet for throttling DetNet flows, i.e.,
the transmission rate cannot be reduced via explicit congestion
notification. The assumption is that a DetNet flow, to be useful,
must be delivered in its entirety.

>      3.2.1.1 also reads oddly in that it seems to recommend providing
>      significant buffering, when the need for and use of such buffers is a major
>      source of jitter.

The next paragraph in 3.2.1.1 explains that the buffers have to be 
adequately designed, which keeps under control the delay and jitter that 
can be caused by buffering:

"Ensuring adequate buffering requires, in turn, that the source, and
every DetNet node along the path to the destination (or nearly every
node, see Section 4.3.3) be careful to regulate its output to not
exceed the data rate for any DetNet flow, except for brief periods
when making up for interfering traffic. Any packet sent ahead of its
time potentially adds to the number of buffers required by the next
hop DetNet node and may thus exceed the resources allocated for a
particular DetNet flow."

>      In section 4.1.2, I realized that the Detnet Transit node terminology had
>      mildly confused me.  The text says "DetNet enabled nodes are interconnected
>      via transit nodes (e.g., LSRs) which support DetNet, but are not DetNet
>      service aware."  Reading this, and the definitions in section 2.1, it
>      appears that a Detnet Transit node is a node that is providing transport
>      behavior that detnet needs, but is not actually modified for detnet.
Based on last call comments: 
https://www.ietf.org/mail-archive/web/detnet/current/msg01791.html, the 
phrase "DetNet enabled nodes" is removed from the document and it has 
been made clear what type of DetNet node is meant:
The text is updated to:

    A "Deterministic Network" will be composed of DetNet enabled end
    systems, DetNet edge nodes, DetNet relay nodes and collectively
    deliver DetNet services.  DetNet relay and edge nodes are
    interconnected via DetNet transit nodes (e.g., LSRs) which support
    DetNet, but are not DetNet service aware.

>      Section 4.4.2 talks about per flow per node state.  It would be good to
>      have a forward reference to section 4.9 about scaling to larger networks.
Forward reference will be added.

> Nits/editorial comments:
>       It would be good if there were some explanation for the 75% maximum number
>       in section 3.3.1.  That there is some limit seems intuitive.  What value
>       that limit has is not so obvious.
75% is been removed based on last call comments: 
https://www.ietf.org/mail-archive/web/detnet/current/msg01791.html.

>      Section 4.7.3 introduces an example using Ethernet Mulicast destiantions as
>      part of labeling.  There is no earlier explanation of the use of an
>      Ethernet MAC Multicast destination, and the text does not seem to require
>      that the traffic be actually multicast.  Hence the reader is liable to be
>      confused by the reference to multicast.  I rate this only a nit as it seems
>      clear that this is an example whose details are presumably explained in
>      another document.    Still, it would help to clarify the example.
>
>
Multicast MAC address is removed based on last call comments: 
https://www.ietf.org/mail-archive/web/detnet/current/msg01791.html,

---

From: "Joel M. Halpern" <[email-masked]>
Date: Fri, 19 Oct 2018 15:17:34 -0400

Thank you Janos.  Two clarifications under retained text, with the rest 
elided.

Yours,
Joel

On 10/19/18 3:10 PM, János Farkas wrote:
...
> On 9/22/2018 2:59 AM, Joel Halpern wrote:
...
> finished at an outbound port, which can cause too much uncertainties in 

I understand that strict priority queueing is viewed as insufficient.  I 
wasn;t arguing about that.  I was arguing with the use of the word 
"unbounded".  As far as I can tell, with suitable priority queueing the 
worst case delay is bounded, simply not well enough bounded.

...
>     DetNet, but are not DetNet service aware.

Any chance you could simply say "transit nodes" instead of "DetNet 
transit nodes?  As far as I can tell, they are existing nodes that were 
designed and implemented (and even configured) potentially before DetNet 
was even defined?

...

---

From: János Farkas <[email-masked]>
Date: Fri, 19 Oct 2018 22:05:31 +0200

Joel,

Please see in-line.

On 10/19/2018 9:17 PM, Joel M. Halpern wrote:
> Thank you Janos.  Two clarifications under retained text, with the 
> rest elided.
>
> Yours,
> Joel
>
>> finished at an outbound port, which can cause too much uncertainties 
>> in the network.
>
> I understand that strict priority queueing is viewed as insufficient.  
> I wasn;t arguing about that.  I was arguing with the use of the word 
> "unbounded".  As far as I can tell, with suitable priority queueing 
> the worst case delay is bounded, simply not well enough bounded.

We can make the sentence clearer:

OLD:
In general, a trivial priority-based queuing scheme will give better
average latency to a data flow than DetNet, but of course, the worst-
case latency can be essentially unbounded.

NEW:
In general, a trivial priority-based queuing scheme will give better
average latency to a data flow than DetNet; however, it may be
that the worst-case latency cannot be reasonably bounded for DetNet.

>> the phrase "DetNet enabled nodes" is removed from the document and it 
>> has been made clear what type of DetNet node is meant:
>> The text is updated to:
>>
> transit nodes?  As far as I can tell, they are existing nodes that 
> were designed and implemented (and even configured) potentially before 
> DetNet was even defined?
>
> ...
It has been pointed out during last call 
(https://www.ietf.org/mail-archive/web/detnet/current/msg01791.html), to 
be specific on what node type is meant; and be consistent with 
terminology and definitions. DetNet transit node is a term defined by 
this document.
So, it seems better to have DetNet transit node.

Thank you!
Best regards,
Janos

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 22 Oct 2018 12:32:53 +0000

Hello János

I's change reasonably to strictly.

The whole point of bounded in detnet is that it is not a vague boundary but a hard, guaranteed one, regardless of any other activity in the network.

All the best,

Pascal

> -----Original Message-----
> From: János Farkas <[email-masked]>
> Sent: vendredi 19 octobre 2018 22:06
> To: Joel M. Halpern <[email-masked]>
> Cc: [email-masked]; [email-masked]; DetNet WG
> <[email-masked]>; [email-masked]
> Subject: Re: Genart last call review of draft-ietf-detnet-architecture-08
> 
> Joel,
> 
> Please see in-line.
> 
> 
> In general, a trivial priority-based queuing scheme will give better average
> latency to a data flow than DetNet, but of course, the worst- case latency can
> be essentially unbounded.
> 
> NEW:
> In general, a trivial priority-based queuing scheme will give better average
> latency to a data flow than DetNet; however, it may be that the worst-case
> latency cannot be reasonably bounded for DetNet.
> 
> 
> >
> > ...
> It has been pointed out during last call (https://www.ietf.org/mail-
> archive/web/detnet/current/msg01791.html), to be specific on what node
> type is meant; and be consistent with terminology and definitions. DetNet
> transit node is a term defined by this document.

---

From: János Farkas <[email-masked]>
Date: Mon, 22 Oct 2018 16:10:50 +0200

Hi Pascal,

Thank you for your comment!

There was another comment pointing out that the text I suggested is 
misleading.

Instead, I suggest the following update:

OLD:

NEW:
In general, a trivial priority-based queuing scheme will give better
average latency to a data flow than DetNet; however, it may not be
a suitable option for DetNet because of its worst-case latency.

I think it addresses both Joel's and Pascal's concern. I hope it is 
clear this way.

Thank you!

Best regards,
Janos