Subject: Opsdir last call review of draft-ietf-tsvwg-sctp-ndata-12

From: Stefan Winter <[email-masked]>
Date: Wed, 16 Aug 2017 02:24:35 -0700

Reviewer: Stefan Winter
Review result: Has Issues

One issue:

The chapter IANA considerations mentions four bits to be registered: E,B,U,I.
In the main body, only three of those are actually defined and used (End of
fragmented message, Beginning of fragmented message, Unordered/Ordered
message). Please add a definition of the I bit or remove the IANA registration
request for that bit.

And the rest are only musings not needing any action if the authors don't want
to action on them:

The introduction presents a good use case, quoting: "e.g., when the
transmission of an urgent
   message is blocked from transmission because the sender has started
   the transmission of another, possibly large, message".

The later queueing example in figure 1 has three SIDs being queued
simultanseously. It is not clear which of those "has already started" and which
are the important ones being delayed. For a better understanding of the reader,
it might be useful to revise the figure so that the head-of-line blocking
happens for SID 1 because transmission of  0 and 2 has already started (and
declare SID 1 to be the "important" message). The ASCII art would just have to
indent SID 1 content a bit (placing 0 and 2 earlier in time), and the resulting
serialisation would then put all the SID 1 messages at the very end, when 0 and
2 have been completely submitted (right?).

In 2.2.3, is there any particular reason why the "protocol violation" error
cause is only a MAY? As it enables debugging on the remote end, a SHOULD seems
useful.

---

From: Michael Tuexen <[email-masked]>
Date: Fri, 25 Aug 2017 16:32:31 +0200

> On 16. Aug 2017, at 11:24, Stefan Winter <[email-masked]> wrote:
> 
> Reviewer: Stefan Winter
Hi Stefan,

thanks for your review. See my comments in-line.

Best regards
Michael
> message). Please add a definition of the I bit or remove the IANA registration
The I bit is defined in RFC 7053 and that is the reason why we state
"DATA chunk defined in [RFC4960]and [RFC7053]". In addition to that it is
not mentioned (in contrast to the U, B and E bit), since the delaying of the
SACK is not specific to the I-DATA chunk.

I have added the following text to the body of Section 2.2:

<t>The handling of the I bit for the I-DATA chunk corresponds to the handling
of the I bit for the DATA chunk described in <xref target='RFC7053'/>.</t>

to improve the description of the I bit. See
https://github.com/sctplab/sctp-idata/commit/f16308ffe0e67ce5deb957bd40c25c5a0e691b82
Does that address your issue?
> simultanseously. It is not clear which of those "has already started" and which
There is actually one message queued for stream 0, three messages for stream 1,
and one message for stream 2.
> are the important ones being delayed. For a better understanding of the reader,
The figure is about in which sequence they are put on the wire. So no
transmission has startet. This examples also does not deal with the case
of one stream having a higher priority than another. That would be an
example using a priority scheduler. This examples illustrates the round
robin scheduler.
The point here is that a single scheduler (the round robin scheduler, for
example) behaves differently when user message interleaving is used or not.
That is an important point: You can even use the schedulers with regular
DATA chunk, i.e. with user message interleaving being negotiated.
> 2 have been completely submitted (right?).
One could think about adding another example based on the priority scheduler
and giving stream 1 a higher priority. But for illustrating the point of
schedulers behaving differently depending on the negotiation of user message
interleaving this seems not necessary.
> cause is only a MAY? As it enables debugging on the remote end, a SHOULD seems
This is for consistency with other SCTP specifications. There is always the balance
between providing more information to help diagnosing the problem and providing more
information to an attacker. This allows implementers to do what they prefer.
For interoperability it is not important if the error cause is included or not.

---

From: Stefan Winter <[email-masked]>
Date: Wed, 30 Aug 2017 12:55:27 +0200

Hi,

> Does that address your issue?

Yes, thank you.

> interleaving this seems not necessary.

That second example might indeed be useful.

It's simply a bit strange that in the introduction you speak about an
"urgent" message (something deserving a higher-priority sending) and
that another transmission has already started - but then the example has
none of those two.

In fact I wonder why you speak about urgent things in the introduction
at all. Neither the scheduler nor the interleaving are designed to help
urgent things to be sent earlier.

All they do is make the transmission fairer so that *all* chunks get a
more even serialised distribution on the wire.

So, rather than saying that the spec helps urgent things to be sent
earlier, it should maybe rather state that the spec helps every message
getting an equal, and equally distributed, share of the medium.

> For interoperability it is not important if the error cause is included or not.

That's fine, thanks.

Greetings,

Stefan

-- 
Stefan WINTER
Ingenieur de Recherche
Fondation RESTENA - Réseau Téléinformatique de l'Education Nationale et de la Recherche
2, avenue de l'Université
L-4365 Esch-sur-Alzette

Tel: +352 424409 1
Fax: +352 422473

PGP key updated to 4096 Bit RSA - I will encrypt all mails if the recipient's key is known to me

http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC0DE6A358A39DC66

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 30 Aug 2017 13:45:01 +0200

>> Does that address your issue?
There has also been added a description of all fields, since this issue came
up multiple times. See
https://xml2rfc.tools.ietf.org/cgi-bin/xml2rfc.cgi?input=&url=https%3A%2F%2Fraw.githubusercontent.com%2Fsctplab%2Fsctp-idata%2Fmaster%2Fdraft-ietf-tsvwg-sctp-ndata.xml&modeAsFormat=html%2Fascii&type=towindow&Submit=Submit
> that another transmission has already started - but then the example has
What about removing the word "urgent" from that sentence.
> more even serialised distribution on the wire.
That is not true in general. If you use the priority scheduler, one
stream gets preferred treatment. It also makes sure that the time
of a message spent in the stream queue is shorter. So these messages
get sent earlier.
In the case of the round robin scheduler, you are right that this is
about getting fairer treatment. If you take the WFQ scheduler, the
streams don't get the same treatment, but it depends on the weight
given to the stream.
> getting an equal, and equally distributed, share of the medium.
It is not meant that the spec does this. It was one example, what
a particular stream scheduler does. But there are multiple schedulers
defined in the document.

The RR scheduler example is not intended to illustrate what you can do
with all the schedulers, but that using message interleaving or not
has an impact on the scheduler. That is meant by stating:

This document also defines several stream schedulers for general SCTP associations.
The stream schedulers may behave differently depending on whether user message
interleaving has been negotiated for the association or not.

This is followed by illustrating this using the RR scheduler.

Best regards
Michael
> <0x8A39DC66.asc>

---

From: Benoit Claise <[email-masked]>
Date: Wed, 30 Aug 2017 18:51:21 +0200

Michael,

I stumbled on the same point as Stefan.

First of, I had to review RFC4960 for TSN/SSN. I was not sure whether 
TSN was per stream or not
It would have helped me to repeat this information in the intro.
Ok, to be candid, I deduced this information from figure 1 and 2.
>>> DATA chunk, i.e. with user message interleaving being negotiated.
This was my source of confusion.
With figure 1 and figure 2, you want to show the effect of Round Robin 
Scheduler with and without User Message Interleaving, while we were 
expecting the figure 2 to be the solution from this spec. Hence Stefan 
and I were looking for this "urgent message" in figure 2.

>> that another transmission has already started - but then the example has
Exactly.
Please add this extra example.
This would also clarify from the introduction that when you write ...

    The sending of such large messages over
    SCTP as specified in [RFC4960] can result in a form of sender side
    head of line blocking (e.g., when the transmission of an urgent
    message is blocked from transmission because the sender has started
    the transmission of another, possibly large, message).

.. the notion of transmission urgency is not within a stream but across 
streams, and that can be achieved with a priority scheduler.
The text might have to be rephrased to explain that this spec provides 
the ability to create stream scheduler with different relative stream 
treatments.
And also that urgent message would be placed in this high priority 
stream scheduler (reliable, I guess, but that's orthogonal) stream.
> This is followed by illustrating this using the RR scheduler.
Also, I don't see the value of the last sentence in this paragraph, 
especially with RFC2119 MAY.

    This section defines several stream schedulers.  The stream
    schedulers may behave differently depending on whether user message
    interleaving has been negotiated for the association or not.  An
    implementation MAY implement any subset of them.

Regards, Benoit

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 31 Aug 2017 14:00:05 +0200

> On 30. Aug 2017, at 18:51, Benoit Claise <[email-masked]> wrote:
> 
> Michael,
> 
Hi Benoit,

thanks for reviewing the document. See my comments in-line.

Best regards
Michael
> 
> First of, I had to review RFC4960 for TSN/SSN. I was not sure whether TSN was per stream or not

> With figure 1 and figure 2, you want to show the effect of Round Robin Scheduler with and without User Message Interleaving, while we were expecting the figure 2 to be the solution from this spec. Hence Stefan and I were looking for this "urgent message" in figure 2.
I don't know what you mean with "solution from this spec" means. This spec introduces
stream schedulers and a protocol extension to allow interleaving of user messages.
Using this to implement an "urgent" concept is only one possibility: usage of a
strict priority scheduler (SCTP_SS_PRIO). So this "urgent" is misleading. It seems
to ring the wrong bells. That is why I suggested to remove it...
> .. the notion of transmission urgency is not within a stream but across streams, and that can be achieved with a priority scheduler.
I would actually prefer to remove the word "urgent" from that sentence. This just rings the wrong bells.
> The text might have to be rephrased to explain that this spec provides the ability to create stream scheduler with different relative stream treatments.
What about:

OLD TEXT
   This document also defines several stream schedulers for general SCTP
   associations.  The stream schedulers may behave differently depending
   on whether user message interleaving has been negotiated for the
   association or not.

NEW TEXT
   This document also defines several stream schedulers for general SCTP
   associations allowing different relative stream treatments.
   The stream schedulers may behave differently depending
   on whether user message interleaving has been negotiated for the
   association or not.

Would that address this issue?
  
> And also that urgent message would be placed in this high priority stream scheduler (reliable, I guess, but that's orthogonal) stream.
I really would like to get rid of this. I don't think giving the strict
priority scheduler as specific role here is a good idea.

The RR scheduler was chosen as the simplest scheduler to illustrate
that scheduler can behave differently when using interleaving or not.
> Also, I don't see the value of the last sentence in this paragraph, especially with RFC2119 MAY.
> 
During the review the question was brought up which scheduler have to be implemented
by an implementation. This sentence was add to address it.
If you want, I can remove it.

---

From: Benoit Claise <[email-masked]>
Date: Thu, 31 Aug 2017 14:14:04 +0200

Hi Michael,
> I would actually prefer to remove the word "urgent" from that sentence. This just rings the wrong bells.
That's a missed opportunity in mind, but your call.
> Would that address this issue?
ok.
> If you want, I can remove it.
The point is that the MAY is that sentence doesn't mean anything.
Maybe you mean:

An implementation MUST implement at least one stream scheduler.

Regards, B.

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 31 Aug 2017 14:25:44 +0200

> On 31. Aug 2017, at 14:14, Benoit Claise <[email-masked]> wrote:
> 
> Hi Michael,
Hmm. Let me remove the word "urgent"...
>> Would that address this issue?
Fixed.
> An implementation MUST implement at least one stream scheduler.
That would require an implementation to implement at least one of
the ones being listed. You are right in the sense that an implementation
MUST implement at least one stream scheduler, but the original text
allows that one not to be specified in this document.

Are you saying that an implementation MUST implement one of list defined
by this document?

Best regards
Michael

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 31 Aug 2017 07:50:43 -0500

Just interjecting ...

On Thu, Aug 31, 2017 at 7:25 AM, Michael Tuexen <[email-masked]>
wrote:

> expecting the figure 2 to be the solution from this spec. Hence Stefan and
> I were looking for this "urgent message" in figure 2.
> >>> .. the notion of transmission urgency is not within a stream but
> across streams, and that can be achieved with a priority scheduler.

Yes, please! I wasn't wild about it, either.

> associations.

I'm pretty sure (and this came up in my AD review) the problem is that the
original specification DID define a stream scheduler, that's wasn't called
a stream scheduler, but SCTP implementations did schedule streams. And that
didn't matter until there was more than one way to schedule streams (as in
this document).

So you're working out better wording that says something like "in addition
to the way the original specification scheduled streams, we're defining
more of them". I think.

?

Spencer

---

From: Benoit Claise <[email-masked]>
Date: Thu, 31 Aug 2017 15:02:18 +0200

> Are you saying that an implementation MUST implement one of list defined
I'm saying that "An implementation MAY implement any subset of them." as 
a sentence doesn't make sense.
Are we going in circle here?

Regards, B.

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 31 Aug 2017 15:25:02 +0200

> On 31. Aug 2017, at 14:50, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> Just interjecting ... 
> 
> On Thu, Aug 31, 2017 at 7:25 AM, Michael Tuexen <[email-masked]> wrote:
> 
It's done:
https://github.com/sctplab/sctp-idata/commit/cf7b331512cd85d6195eb88fa96360673293323f
> I'm pretty sure (and this came up in my AD review) the problem is that the original specification DID define a stream scheduler, that's wasn't called a stream scheduler, but SCTP implementations did schedule streams. And that didn't matter until there was more than one way to schedule streams (as in this document).
The original specification did NOT specify a stream scheduler and left this completely implementation
specific.
Right now Linux is using (and only supporting a single scheduler) what we call in this document
SCTP_SS_FCFS. FreeBSD's default scheduler is what we call in this document SCTP_SS_RR and
Solaris uses one of these, forgot which one. Among these kernel implementations, only FreeBSD
supports multiple streams schedulers and its selection. Possibly other closed source SCTP
implementations might support other schedulers. Don't know.
> 
> So you're working out better wording that says something like "in addition to the way the original specification scheduled streams, we're defining more of them". I think.
I think we wanted to say something like:

Here is a list of schedulers. Your implementation might support some of them, but this is
left open. So you can't rely on it.

The list of schedulers we specified includes the ones being used as default by FreeBSD, Linux and
Solaris and was extended by the ones supported by FreeBSD and the one explicitly required by
WebRTC, which is SCTP_SS_WFQ.

What about this change:

OLD TEXT:
   An implementation MAY implement any subset of them.

NEW TEXT:
   An implementation MAY implement any subset of them. If the implementation
   is used for WebRTC Datachannels as specified in [I-D.ietf-rtcweb-data-channel]
   it MUST implement the Weighted Fair Queueing Scheduler defined in Section 3.6.

Best regards
Michael

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 31 Aug 2017 15:25:22 +0200

> On 31. Aug 2017, at 15:02, Benoit Claise <[email-masked]> wrote:
> 
> 
> I'm saying that "An implementation MAY implement any subset of them." as a sentence doesn't make sense.
What about:

What about this change:

OLD TEXT:

NEW TEXT:

Best regards
Michael

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 31 Aug 2017 08:46:12 -0500

Hi, Michael,

On Thu, Aug 31, 2017 at 8:25 AM, Michael Tuexen <[email-masked]>
wrote:

> > >>> With figure 1 and figure 2, you want to show the effect of Round
> Robin Scheduler with and without User Message Interleaving, while we were
> > >> I would actually prefer to remove the word "urgent" from that
> sentence. This just rings the wrong bells.
> > >>> The text might have to be rephrased to explain that this spec
> provides the ability to create stream scheduler with different relative
> stream treatments.
> > >> What about:
> > >>
> > >> OLD TEXT
> > I'm pretty sure (and this came up in my AD review) the problem is that
> the original specification DID define a stream scheduler, that's wasn't
> called a stream scheduler, but SCTP implementations did schedule streams.
> And that didn't matter until there was more than one way to schedule
> streams (as in this document).

Thanks for getting that right for me ...

Spencer

>  > So you're working out better wording that says something like "in
> addition to the way the original specification scheduled streams, we're
> defining more of them". I think.

>    it MUST implement the Weighted Fair Queueing Scheduler defined in

That works for me, but I'll let Benoit reply.

Spencer

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 31 Aug 2017 21:56:04 +0200

> On 31. Aug 2017, at 15:46, Spencer Dawkins at IETF <[email-masked]> wrote:
> 
> Hi, Michael,
> 
> On Thu, Aug 31, 2017 at 8:25 AM, Michael Tuexen <[email-masked]> wrote:
Benoit?

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 31 Aug 2017 16:01:37 -0500

Hi, Michael,

> > That works for me, but I'll let Benoit reply.

Since I'm not sure which timezone Benoit is in (except that I know it's
Yang Central Time), I'll respond by saying that I did talk with Benoit on
the call, after sending him this privately:

This is tripping over the same thing I was tripping over - SCTP standards
haven't had a stream scheduler, but SCTP implementations scheduled streams.
So Michael was thinking that you were asking about forcing existing
implementations to implement one of the new ones.

His response was to propose text basically saying "in addition to what
you're doing now, you MAY do one or more of the following".

I'm sorry I garbled my point so badly.

Benoit said on the call (if I understood him correctly) that my explanation
(filtered through your correction) worked for him. The point I was trying
to make earlier was that SCTP implementations scheduled streams somehow,
and that your document wasn't an effort to tell them to stop doing what
they were doing, but that implementers now had more standardized options
than they did before.

I think your proposed text says that well.

At this point, I think you're ready to submit a revised ID to reflect the
comments you've received. I'll review your changes, and that will give
Benoit a chance to interrupt if he needs to - he's quite responsive,
especially when he doesn't spend the evening on a telechat.

Thanks, and it's always a pleasure.

Spencer

---

From: Benoit Claise <[email-masked]>
Date: Fri, 1 Sep 2017 08:59:32 +0200

On 8/31/2017 11:01 PM, Spencer Dawkins at IETF wrote:
> Hi, Michael,
>
> On Thu, Aug 31, 2017 at 2:56 PM, Michael Tuexen <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>     > On 31. Aug 2017, at 15:46, Spencer Dawkins at IETF
>     <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >
>     > Hi, Michael,
>     >
>     > On Thu, Aug 31, 2017 at 8:25 AM, Michael Tuexen
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     > > On 31. Aug 2017, at 14:50, Spencer Dawkins at IETF
>     <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > >
>     > > Just interjecting ...
>     > >
>     > > On Thu, Aug 31, 2017 at 7:25 AM, Michael Tuexen
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     > >
>     > > > On 31. Aug 2017, at 14:14, Benoit Claise <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > > >
>     > > > Hi Michael,
>     > > >>> On 30. Aug 2017, at 18:51, Benoit Claise
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     > > >>>
>     > > >>> Michael,
>     > > >>>
>     > > >>> With figure 1 and figure 2, you want to show the effect of
>     Round Robin Scheduler with and without User Message Interleaving,
>     while we were expecting the figure 2 to be the solution from this
>     spec. Hence Stefan and I were looking for this "urgent message" in
>     figure 2.
>     > > >>> .. the notion of transmission urgency is not within a
>     stream but across streams, and that can be achieved with a
>     priority scheduler.
>     > > >> I would actually prefer to remove the word "urgent" from
>     that sentence. This just rings the wrong bells.
>     <https://github.com/sctplab/sctp-idata/commit/cf7b331512cd85d6195eb88fa96360673293323f>
>     > >
>     > > >>> The text might have to be rephrased to explain that this
>     spec provides the ability to create stream scheduler with
>     different relative stream treatments.
>     > > >> What about:
>     > > >>
>     > > >> OLD TEXT
>     > > >>> And also that urgent message would be placed in this high
>     priority stream scheduler (reliable, I guess, but that's
>     orthogonal) stream.
>     > > I'm pretty sure (and this came up in my AD review) the problem
>     is that the original specification DID define a stream scheduler,
>     that's wasn't called a stream scheduler, but SCTP implementations
>     did schedule streams. And that didn't matter until there was more
>     than one way to schedule streams (as in this document).
>     >  > So you're working out better wording that says something like
>     "in addition to the way the original specification scheduled
>     streams, we're defining more of them". I think.
> Since I'm not sure which timezone Benoit is in (except that I know 
> it's Yang Central Time), I'll respond by saying that I did talk with 
> Benoit on the call, after sending him this privately:
>
>
>     This is tripping over the same thing I was tripping over - SCTP
>     standards haven't had a stream scheduler, but SCTP implementations
>     scheduled streams. So Michael was thinking that you were asking
>     about forcing existing implementations to implement one of the new
>     ones.
>
>     His response was to propose text basically saying "in addition to
>     what you're doing now, you MAY do one or more of the following".
>
> Benoit said on the call (if I understood him correctly) that my 
> explanation (filtered through your correction) worked for him. The 
> point I was trying to make earlier was that SCTP implementations 
> scheduled streams somehow, and that your document wasn't an effort to 
> tell them to stop doing what they were doing, but that implementers 
> now had more standardized options than they did before.
>
> At this point, I think you're ready to submit a revised ID to reflect 
> the comments you've received. I'll review your changes, and that will 
> give Benoit a chance to interrupt if he needs to -
no interrupt needed. I'll trust the venerable Spencer.

Regards, Benoit
> he's quite responsive, especially when he doesn't spend the evening on 
> a telechat.
>