Subject: Re: [tsvwg] [OPS-DIR] Opsdir last call review of draft-ietf-tsvwg-sctp-ndata-12

From: Benoit Claise <[email-masked]>
Date: Fri, 1 Sep 2017 08:59:32 +0200

On 8/31/2017 11:01 PM, Spencer Dawkins at IETF wrote:
> Hi, Michael,
>
> On Thu, Aug 31, 2017 at 2:56 PM, Michael Tuexen <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>
>     > On 31. Aug 2017, at 15:46, Spencer Dawkins at IETF
>     <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     >
>     > Hi, Michael,
>     >
>     > On Thu, Aug 31, 2017 at 8:25 AM, Michael Tuexen
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     > > On 31. Aug 2017, at 14:50, Spencer Dawkins at IETF
>     <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > >
>     > > Just interjecting ...
>     > >
>     > > On Thu, Aug 31, 2017 at 7:25 AM, Michael Tuexen
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     > >
>     > > > On 31. Aug 2017, at 14:14, Benoit Claise <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > > >
>     > > > Hi Michael,
>     > > >>> On 30. Aug 2017, at 18:51, Benoit Claise
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     > > >>>
>     > > >>> Michael,
>     > > >>>
>     > > >>> I stumbled on the same point as Stefan.
>     > > >> Hi Benoit,
>     > > >>
>     > > >> thanks for reviewing the document. See my comments in-line.
>     > > >>
>     > > >> Best regards
>     > > >> Michael
>     > > >>> First of, I had to review RFC4960 for TSN/SSN. I was not
>     sure whether TSN was per stream or not
>     > > >>> It would have helped me to repeat this information in the
>     intro.
>     > > >>> Ok, to be candid, I deduced this information from figure 1
>     and 2.
>     > > >>>>>>> And the rest are only musings not needing any action
>     if the authors don't want
>     > > >>>>>>> to action on them:
>     > > >>>>>>>
>     > > >>>>>>> The introduction presents a good use case, quoting:
>     "e.g., when the
>     > > >>>>>>> transmission of an urgent
>     > > >>>>>>>  message is blocked from transmission because the
>     sender has started
>     > > >>>>>>>  the transmission of another, possibly large, message".
>     > > >>>>>>>
>     > > >>>>>>> The later queueing example in figure 1 has three SIDs
>     being queued
>     > > >>>>>>> simultanseously. It is not clear which of those "has
>     already started" and which
>     > > >>>>>> There is actually one message queued for stream 0,
>     three messages for stream 1,
>     > > >>>>>> and one message for stream 2.
>     > > >>>>>>> are the important ones being delayed. For a better
>     understanding of the reader,
>     > > >>>>>> The figure is about in which sequence they are put on
>     the wire. So no
>     > > >>>>>> transmission has startet. This examples also does not
>     deal with the case
>     > > >>>>>> of one stream having a higher priority than another.
>     That would be an
>     > > >>>>>> example using a priority scheduler. This examples
>     illustrates the round
>     > > >>>>>> robin scheduler.
>     > > >>>>>> The point here is that a single scheduler (the round
>     robin scheduler, for
>     > > >>>>>> example) behaves differently when user message
>     interleaving is used or not.
>     > > >>>>>> That is an important point: You can even use the
>     schedulers with regular
>     > > >>>>>> DATA chunk, i.e. with user message interleaving being
>     negotiated.
>     > > >>> This was my source of confusion.
>     > > >>> With figure 1 and figure 2, you want to show the effect of
>     Round Robin Scheduler with and without User Message Interleaving,
>     while we were expecting the figure 2 to be the solution from this
>     spec. Hence Stefan and I were looking for this "urgent message" in
>     figure 2.
>     > > >> I don't know what you mean with "solution from this spec"
>     means. This spec introduces
>     > > >> stream schedulers and a protocol extension to allow
>     interleaving of user messages.
>     > > >> Using this to implement an "urgent" concept is only one
>     possibility: usage of a
>     > > >> strict priority scheduler (SCTP_SS_PRIO). So this "urgent"
>     is misleading. It seems
>     > > >> to ring the wrong bells. That is why I suggested to remove
>     it...
>     > > >>>>>>> it might be useful to revise the figure so that the
>     head-of-line blocking
>     > > >>>>>>> happens for SID 1 because transmission of  0 and 2 has
>     already started (and
>     > > >>>>>>> declare SID 1 to be the "important" message). The
>     ASCII art would just have to
>     > > >>>>>>> indent SID 1 content a bit (placing 0 and 2 earlier in
>     time), and the resulting
>     > > >>>>>>> serialisation would then put all the SID 1 messages at
>     the very end, when 0 and
>     > > >>>>>>> 2 have been completely submitted (right?).
>     > > >>>>>> One could think about adding another example based on
>     the priority scheduler
>     > > >>>>>> and giving stream 1 a higher priority. But for
>     illustrating the point of
>     > > >>>>>> schedulers behaving differently depending on the
>     negotiation of user message
>     > > >>>>>> interleaving this seems not necessary.
>     > > >>>>> That second example might indeed be useful.
>     > > >>>>>
>     > > >>>>> It's simply a bit strange that in the introduction you
>     speak about an
>     > > >>>>> "urgent" message (something deserving a higher-priority
>     sending) and
>     > > >>>>> that another transmission has already started - but then
>     the example has
>     > > >>>>> none of those two.
>     > > >>> Exactly.
>     > > >>> Please add this extra example.
>     > > >>> This would also clarify from the introduction that when
>     you write ...
>     > > >>>
>     > > >>>   The sending of such large messages over
>     > > >>>   SCTP as specified in [RFC4960] can result in a form of
>     sender side
>     > > >>>   head of line blocking (e.g., when the transmission of an
>     urgent
>     > > >>>   message is blocked from transmission because the sender
>     has started
>     > > >>>   the transmission of another, possibly large, message).
>     > > >>>
>     > > >>> .. the notion of transmission urgency is not within a
>     stream but across streams, and that can be achieved with a
>     priority scheduler.
>     > > >> I would actually prefer to remove the word "urgent" from
>     that sentence. This just rings the wrong bells.
>     > > > That's a missed opportunity in mind, but your call.
>     > > Hmm. Let me remove the word "urgent"...
>     > >
>     > > Yes, please! I wasn't wild about it, either.
>     > It's done:
>     >
>     https://github.com/sctplab/sctp-idata/commit/cf7b331512cd85d6195eb88fa96360673293323f
>     <https://github.com/sctplab/sctp-idata/commit/cf7b331512cd85d6195eb88fa96360673293323f>
>     > >
>     > > >>> The text might have to be rephrased to explain that this
>     spec provides the ability to create stream scheduler with
>     different relative stream treatments.
>     > > >> What about:
>     > > >>
>     > > >> OLD TEXT
>     > > >>    This document also defines several stream schedulers for
>     general SCTP
>     > > >>    associations.  The stream schedulers may behave
>     differently depending
>     > > >>    on whether user message interleaving has been negotiated
>     for the
>     > > >>    association or not.
>     > > >>
>     > > >> NEW TEXT
>     > > >>    This document also defines several stream schedulers for
>     general SCTP
>     > > >>    associations allowing different relative stream treatments.
>     > > >>    The stream schedulers may behave differently depending
>     > > >>    on whether user message interleaving has been negotiated
>     for the
>     > > >>    association or not.
>     > > >>
>     > > >> Would that address this issue?
>     > > > ok.
>     > > Fixed.
>     > > >>
>     > > >>> And also that urgent message would be placed in this high
>     priority stream scheduler (reliable, I guess, but that's
>     orthogonal) stream.
>     > > >> I really would like to get rid of this. I don't think
>     giving the strict
>     > > >> priority scheduler as specific role here is a good idea.
>     > > >>
>     > > >> The RR scheduler was chosen as the simplest scheduler to
>     illustrate
>     > > >> that scheduler can behave differently when using
>     interleaving or not.
>     > > >>>> What about removing the word "urgent" from that sentence.
>     > > >>>>> In fact I wonder why you speak about urgent things in
>     the introduction
>     > > >>>>> at all. Neither the scheduler nor the interleaving are
>     designed to help
>     > > >>>>> urgent things to be sent earlier.
>     > > >>>>>
>     > > >>>>> All they do is make the transmission fairer so that
>     *all* chunks get a
>     > > >>>>> more even serialised distribution on the wire.
>     > > >>>> That is not true in general. If you use the priority
>     scheduler, one
>     > > >>>> stream gets preferred treatment. It also makes sure that
>     the time
>     > > >>>> of a message spent in the stream queue is shorter. So
>     these messages
>     > > >>>> get sent earlier.
>     > > >>>> In the case of the round robin scheduler, you are right
>     that this is
>     > > >>>> about getting fairer treatment. If you take the WFQ
>     scheduler, the
>     > > >>>> streams don't get the same treatment, but it depends on
>     the weight
>     > > >>>> given to the stream.
>     > > >>>>> So, rather than saying that the spec helps urgent things
>     to be sent
>     > > >>>>> earlier, it should maybe rather state that the spec
>     helps every message
>     > > >>>>> getting an equal, and equally distributed, share of the
>     medium.
>     > > >>>> It is not meant that the spec does this. It was one
>     example, what
>     > > >>>> a particular stream scheduler does. But there are
>     multiple schedulers
>     > > >>>> defined in the document.
>     > > >>>>
>     > > >>>> The RR scheduler example is not intended to illustrate
>     what you can do
>     > > >>>> with all the schedulers, but that using message
>     interleaving or not
>     > > >>>> has an impact on the scheduler. That is meant by stating:
>     > > >>>>
>     > > >>>> This document also defines several stream schedulers for
>     general SCTP associations.
>     > > >>>> The stream schedulers may behave differently depending on
>     whether user message
>     > > >>>> interleaving has been negotiated for the association or not.
>     > > >>>>
>     > > >>>> This is followed by illustrating this using the RR scheduler.
>     > > >>>>
>     > > >>>> Best regards
>     > > >>> Also, I don't see the value of the last sentence in this
>     paragraph, especially with RFC2119 MAY.
>     > > >>>
>     > > >>>   This section defines several stream schedulers.  The stream
>     > > >>>   schedulers may behave differently depending on whether
>     user message
>     > > >>>   interleaving has been negotiated for the association or
>     not.  An
>     > > >>>   implementation MAY implement any subset of them.
>     > > >> During the review the question was brought up which
>     scheduler have to be implemented
>     > > >> by an implementation. This sentence was add to address it.
>     > > >> If you want, I can remove it.
>     > > > The point is that the MAY is that sentence doesn't mean
>     anything.
>     > > > Maybe you mean:
>     > > >
>     > > > An implementation MUST implement at least one stream scheduler.
>     > > That would require an implementation to implement at least one of
>     > > the ones being listed. You are right in the sense that an
>     implementation
>     > > MUST implement at least one stream scheduler, but the original
>     text
>     > > allows that one not to be specified in this document.
>     > >
>     > > Are you saying that an implementation MUST implement one of
>     list defined
>     > > by this document?
>     > >
>     > > I'm pretty sure (and this came up in my AD review) the problem
>     is that the original specification DID define a stream scheduler,
>     that's wasn't called a stream scheduler, but SCTP implementations
>     did schedule streams. And that didn't matter until there was more
>     than one way to schedule streams (as in this document).
>     > The original specification did NOT specify a stream scheduler
>     and left this completely implementation
>     > specific.
>     > Right now Linux is using (and only supporting a single
>     scheduler) what we call in this document
>     > SCTP_SS_FCFS. FreeBSD's default scheduler is what we call in
>     this document SCTP_SS_RR and
>     > Solaris uses one of these, forgot which one. Among these kernel
>     implementations, only FreeBSD
>     > supports multiple streams schedulers and its selection. Possibly
>     other closed source SCTP
>     > implementations might support other schedulers. Don't know.
>     >
>     > Thanks for getting that right for me ...
>     >
>     > Spencer
>     >
>     >  > So you're working out better wording that says something like
>     "in addition to the way the original specification scheduled
>     streams, we're defining more of them". I think.
>     > I think we wanted to say something like:
>     >
>     > Here is a list of schedulers. Your implementation might support
>     some of them, but this is
>     > left open. So you can't rely on it.
>     >
>     > The list of schedulers we specified includes the ones being used
>     as default by FreeBSD, Linux and
>     > Solaris and was extended by the ones supported by FreeBSD and
>     the one explicitly required by
>     > WebRTC, which is SCTP_SS_WFQ.
>     >
>     > What about this change:
>     >
>     > OLD TEXT:
>     >    An implementation MAY implement any subset of them.
>     >
>     > NEW TEXT:
>     >    An implementation MAY implement any subset of them. If the
>     implementation
>     >    is used for WebRTC Datachannels as specified in
>     [I-D.ietf-rtcweb-data-channel]
>     >    it MUST implement the Weighted Fair Queueing Scheduler
>     defined in Section 3.6.
>     >
>     > That works for me, but I'll let Benoit reply.
>     Benoit?
>
>
> Since I'm not sure which timezone Benoit is in (except that I know 
> it's Yang Central Time), I'll respond by saying that I did talk with 
> Benoit on the call, after sending him this privately:
>
>
>     This is tripping over the same thing I was tripping over - SCTP
>     standards haven't had a stream scheduler, but SCTP implementations
>     scheduled streams. So Michael was thinking that you were asking
>     about forcing existing implementations to implement one of the new
>     ones.
>
>     His response was to propose text basically saying "in addition to
>     what you're doing now, you MAY do one or more of the following".
>
>     I'm sorry I garbled my point so badly.
>
> Benoit said on the call (if I understood him correctly) that my 
> explanation (filtered through your correction) worked for him. The 
> point I was trying to make earlier was that SCTP implementations 
> scheduled streams somehow, and that your document wasn't an effort to 
> tell them to stop doing what they were doing, but that implementers 
> now had more standardized options than they did before.
>
> I think your proposed text says that well.
>
> At this point, I think you're ready to submit a revised ID to reflect 
> the comments you've received. I'll review your changes, and that will 
> give Benoit a chance to interrupt if he needs to -
no interrupt needed. I'll trust the venerable Spencer.

Regards, Benoit
> he's quite responsive, especially when he doesn't spend the evening on 
> a telechat.
>
> Thanks, and it's always a pleasure.
>
> Spencer
>
>     >
>     > Spencer
>     >
>     > Best regards
>     > Michael
>     > >
>     > > ?
>     > >
>     > > Spencer
>     >
>     >
>
>