Subject: [tsvwg] Bundling on a per message basis in SCTP and SCTP API

From: "Maxim Proshin [GMAIL]" <[email-masked]>
Date: Wed, 4 Jul 2012 11:01:10 +0400

Hi,

I would like to get some clarifications and maybe suggestions from tsvwg
experts on the following issue:

Bundling is one of the SCTP protocol operations which can
be enabled/disabled in run-time by the user according to RFC 4960.

RFC 4960, ch.1.5.5 describes that:

   The SCTP user has the option to request bundling of more than one
   user message into a single SCTP packet.  The chunk bundling function
   of SCTP is responsible for assembly of the complete SCTP packet and
   its disassembly at the receiving end.

Moreover RFC 4960, ch.10.1 (ULP-SCTP) describes how the user can handle
the bundling status on a per message basis:

   E) Send

    Format: SEND(association id, buffer address, byte count [,context]
            [,stream id] [,life time] [,destination transport address]
            [,unordered flag] [,no-bundle flag] [,payload protocol-id] )
    -> result

   This is the main method to send user data via SCTP.

...

 Optional attributes:

...

   o  no-bundle flag - instructs SCTP not to bundle this user data with
      other outbound DATA chunks.  SCTP MAY still bundle even when this
      flag is present, when faced with network congestion.

...

It means that RFC 4960 introduces a possibility for the user to disable
bundling *per message*.

At the same time SCTP API standardized in RFC 6458 does not have such
possibility. Particularly it is impossible to handle the bundling status on
a per message basis. There is only SCTP_NODELAY option which is used *on a
per association basis*:

*8.1.5**.  SCTP_NODELAY*

   This option turns on/off any Nagle-like algorithm.  This means that
   packets are generally sent as soon as possible, and no unnecessary
   delays are introduced, at the cost of more packets in the network.
   In particular, not using any Nagle-like algorithm might reduce the
   bundling of small user messages in cases where this would require an
   additional delay.

   Turning this option on disables any Nagle-like algorithm.
   This option expects an integer boolean flag, where a non-zero value
   turns on the option, and a zero value turns off the option.

Someone may say that SCTP_NODELAY covers the option from RFC 4960. But it
is not exactly true and using SCTP_NODELAY results in additional logic on
the user layer (and even performance/throughput
decrease) because the user should enable bundling every time after
disabling it for a particular message and should control it. Thus,
SCTP_NODELAY does not suit if the user just wants to disable bundling per
message.

What was the reason to not introducing the same option in RFC 6458? *Was it
a deliberate decision or a simple oversight? Have you had any discussions
regarding it?***

Do you have any ideas how the 'no-bundle flag' option from RFC 4960 can be
reused in SCTP API?

Thanks in advance!

-- 
BR, Maxim

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 4 Jul 2012 10:38:46 +0200

On Jul 4, 2012, at 9:01 AM, Maxim Proshin [GMAIL] wrote:

> Hi,
> 
>  
> I would like to get some clarifications and maybe suggestions from tsvwg experts on the following issue:
> 
>  
> Bundling is one of the SCTP protocol operations which can be enabled/disabled in run-time by the user according to RFC 4960.
> 
> Moreover RFC 4960, ch.10.1 (ULP-SCTP) describes how the user can handle the bundling status on a per message basis:
> 
>  
>    E) Send
> 
> It means that RFC 4960 introduces a possibility for the user to disable bundling per message.
Please note that the the beginning of section 10 of RFC 4960 says:
The following functional description of ULP interface primitives is shown for illustrative
purposes.  Different SCTP implementations may have different ULP interfaces.
>  
> At the same time SCTP API standardized in RFC 6458 does not have such possibility. Particularly it is impossible to handle the bundling status on a per message basis. There is only SCTP_NODELAY option which is used on a per association basis:
> 
>  
> 8.1.5.  SCTP_NODELAY
> 
> Someone may say that SCTP_NODELAY covers the option from RFC 4960. But it is not exactly true and using SCTP_NODELAY results in additional logic on the user layer (and even performance/throughput decrease) because the user should enable bundling every time after disabling it for a particular message and should control it. Thus, SCTP_NODELAY does not suit if the user just wants to disable bundling per message.
The socket API as specified in RFC 6458 is just different from the illustrative API described in RFC 4960.
> 
>  
> What was the reason to not introducing the same option in RFC 6458? Was it a deliberate decision or a simple oversight? Have you had any discussions regarding it?
I think no-one really needed it on a per message base...
> 
> Do you have any ideas how the 'no-bundle flag' option from RFC 4960 can be reused in SCTP API?
Disable Nagle, if you need.

Best regards
Michael

---

From: "Maxim Proshin [GMAIL]" <[email-masked]>
Date: Wed, 4 Jul 2012 13:28:58 +0400

Hi Michael,

Thanks!

>The socket API as specified in RFC 6458 is just different from the
illustrative API described in RFC 4960.

Yes, it is true. But RFC 6458 has been finalized much later than RFC 4960
and it is strange to see that not all proposals are covered in SCTP API.

>I think no-one really needed it on a per message base...

Have you discussed it in IETF or tsvwg community? It can be useful for
insatnce when one type of messages (low rate) should be sent without
bundling, i.e. right after the application provided it to SCTP, and the
other type of messages (high rate) should be bundled as usually.

>Disable Nagle, if you need.

Then bundling will be disabled for all messages (in a normal situation
without congestion) and it will not help in the example described by me
above.

---
BR, Maxim

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 4 Jul 2012 13:34:56 +0200

On Jul 4, 2012, at 11:28 AM, Maxim Proshin [GMAIL] wrote:

> Yes, it is true. But RFC 6458 has been finalized much later than RFC 4960 and it is strange to see that not all proposals are covered in SCTP API.
> 
> Have you discussed it in IETF or tsvwg community? It can be useful for insatnce when one type of messages (low rate) should be sent without bundling, i.e. right after the application provided it to SCTP, and the other type of messages (high rate) should be bundled as usually.
RFC 6458 was being developed within TSVWG for almost 10 years... I'm not sure your scenario would
be well covered by the don't bundle bit... It sounds more like you want some kind of priority
assigned to the message, some stream scheduler reflecting this and then not delaying the
high priority messages due to the Nagle algorithm...

Best regards
Michael
> Then bundling will be disabled for all messages (in a normal situation without congestion) and it will not help in the example described by me above.
> 
> ---
> BR, Maxim

---

From: Chris Benson <[email-masked]>
Date: Thu, 5 Jul 2012 10:41:49 -0700 (PDT)

Maxim,

I think the user only cares about delay or no delay - this 
is not quite the same as bundled or not bundled.

If we try to draw a line between the internal mechanisms of 
SCTP (such as bundling), and its externally viewable beahviour 
(such as potential transmission delays), then your stated 
scenario looks slightly different:

(1) There is a "slow" source of data - it is OK for its 
messages to be delayed slightly (somewhat greater bandwidth).

(2) There is a "critical" source of data (using the same 
association). Any "avoidable" delay is not acceptable.

In this example, I think the user is interested in no delay
of messages of type (2). They can still be bundled with
PREVIOUSLY held messages of type (1) and that would be OK.

"Send type (2) immediately - but perhaps in a bundle" is
the significant difference. I think that's all a user wants.
The user cares about DELAY or NO_DELAY rather than details
of how that happens inside SCTP.

There may be acceptable intermingling: bundling a new type (2) 
"critical" message with an old waiting type (1) "slow" message.  
With this view, it is perhaps more obvious that the "user
perceived feature" does affect the whole association and not 
just individual messages. A "critical" message can affect
(reduce) the delay time span of a "slow" message in order 
to still have the benefits of bundling.

A "critical" message of type (2) can still be bundled,
it just shouldn't be delayed. It should trigger turning 
off the transmit delay FOR THE WHOLE ASSOCIATION, including
any further delay of pending "slow" messgaes of type (1).

If I were to manage the use of this combination within the 
confines of RFC 6458 (one implemented User Layer API - not
the definitive only way), then as the SCTP-user, I would 
maintain my own boolean flag of the current association's
NO_DELAY state and do this:

  send_slow_data
  {
    if (NO_DELAY is ON)
        turn NO_DELAY off;
    send_the_data;
  }

  send_critical_data
  {
    if (NO_DELAY is OFF)
        turn NO_DELAY on;
    send_the_data;
  }

where of course send_the_date means deliver it to SCTP for 
transmission, which will potentially delay when NO_DELAY is off.
I can imagine that some "critical" data might be bundled with
previously submitted "slow" data, but not delayed to make a
later bundle.

I think you get what the SCTP-user wants even within RFC 6458.

But that's just my view of my universe!  ;-)

With thanks from Chris Benson.

On Wed, 4 Jul 2012, Maxim Proshin [GMAIL] wrote:

>>  Date: Wed, 4 Jul 2012 11:01:10 +0400
>>  From: "Maxim Proshin [GMAIL]" <[email-masked]>
>>  To:  <[email-masked]>
>>  Subject: [tsvwg] Bundling on a per message basis in SCTP and SCTP API
>>  
>>  Hi,
>>  
>>  
>>

---

From: Randy Stewart <[email-masked]>
Date: Fri, 6 Jul 2012 06:30:50 -0400

Maxim:

For over 10 years the socket API was discussed in tsvwg. In that 10 years I don't
recall one request to be able to disable bundling on a per message basis. If there
would have been, I would have gladly added it. I think most likely, as Michael suggests, 
that turning on/off Nagle (like you do in TCP) was seen as good enough.

One other thing to note about the RFC, I think if you read it closely you might
find that the stack may bundle whenever it wants disregarding such requests. This
is because bundling is used to optimize transport, but usually with a delay in sending
something (aka Nagle). However there are times when the stack, without delay, has the
opportunity to bundle chunks together (think of a small fast retransmit), in such a case
the stack can bundle no matter what the user requested, this *is* in the standards 
section if I recall correctly, where as (as Michael pointed out) the API was
a "mythical" API (we used to have that wording in it since we wanted no implementation
to feel they had to do exactly what was said), which was put in as a non-standards track
portion to give the reader a "feel" for what the API would look like. Even RFC 6458 (though
implemented by the three major stacks Solaris, Linux and BSD) is an informational RFC, not
a standards track… since these types of things do not really effect inter-operability (portability
yes, inter-operability no).

I am sure that if some stack maker really wanted or had a need for this behavior they would
supply the function. Even though, as I state above, I think the stack may still bundle
in some instances.

R

> Then bundling will be disabled for all messages (in a normal situation without congestion) and it will not help in the example described by me above.

-----
Randall Stewart
[email-masked]