Subject: Re: HTTP over SCTP without chunked encoding

From: Henrik Nordstrom <[email-masked]>
Date: Thu, 25 Jun 2009 03:12:54 +0200

Sorry for the late response.

ons 2009-04-08 klockan 16:38 +1000 skrev Mark Nottingham:
> I'm far from an expert in SCTP, but at first glance, this is  
> potentially attractive, but slightly more intrusive in the HTTP stack,  
> because it's introducing a new way to frame messages, rather than  
> reusing exising code to do so.

Both yes and no.

In terms of specification chunked encoding is a transport feature,
independent of messaging semantics. But some implementations mixes the
two a bit (quite natually).

I am of the opinion that both modes should be specified from start, with
receivers supporting both.  It's fairly trivial (with a little overhead)
to rewrite a SCTP structured stream into chunked encoding if needed even
without parsing so the implementation cost in supporting this at the
receiver is minimal.

Mapping chunked to SCTP PPID requires a little more than what was
described earlier.

A chunked HTTP message consists of the following parts

* Initial header
* Some data chunks
* last-chunk
* trailer
* END [CRLF]

Allowing this to be mapped cleanly to SCTP messages would provide a
better SCTP transport for http with

* less overhead, by reusing the length & ppid fields already present in
SCTP.
* if the same mapping is used for Content-Length messages as well then
the parsing problems outlined in the response (and request) splitting
attacks can be avoided

This requires a bit more than the two mentioned PPID definitions
mentioned earlier. In SCTP discussions I proposed a mapping of in total
4 PPID values I think, or to be exact 2 values + 1 bit signalling END. 

  HTTP Header  (also reused for the trailer portion)
  HTTP Data payload

Which would compose a HTTP message into the following SCTP messages

chunked response with trailer:

  1. HTTP Header
  2. HTTP Data payload*N
  3. HTTP Header + END

In addition to this there needs to be some way to indicate per-stream
error conditions, in both directions. At least until SCTP supports such
notifications per-stream rather than on an association as a whole.

On a related note it's worth mentioning that SSL imposes it's own
interesting issues in this.. the above scheme only works for plain HTTP,
not when wrapped in SSL. For SSL wrapped requests I see no option but to
use chunked encoding within SSL as done today.

> Now, whether this trade-off is worth it or not is an open question, of  
> course. I think most people are focused on the aspects of SCTP that  
> relieve the problems experienced with pipelining, and while having a  
> more efficient framing mechanism is interesting, it's not low-hanging  
> fruit.

Actually the two is quite related.. at least in my eyes working at a
proxy level were you have to deal with a high level of concurrency.

For a single-user-agent it's not as complicated however. There the
proposed simple SCTP trabsoirt provides a reasonably close resemblance
to requests pipelined over a single TCP connections. But trying to map
multiple pipelined TCP connections to a single SCTP association do not
work out well in the proposed simple form, at least not yet.

Regards
Henrik

---

From: [email-masked]
Date: Thu, 25 Jun 2009 16:53:13 -0400 (EDT)

Henrik -

Can you elaborate on your comment about trying to map multiple pipelined
TCP connections to a single SCTP association not working out well? 
Thanks.

- Jon

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 26 Jun 2009 02:18:52 +0200

> TCP connections to a single SCTP association not working out well? 

Sorry, should have said multiple TCP connections vs a single SCTP
association. It's not related to pipelining but the differences in
transport level (in HTTP terms) signaling between multiple TCP
connections and a single SCTP association.

The main problem there is selective stream cancellation by the client.

If all pending requests is to be canceled together then this is not an
big issue, just tear down the SCTP association and be done with it.  But
if you need to more selectively cancel requests (i.e. a big download you
accidently started) without breaking the other concurrent pending
requests then things get a little trickier..

In the TCP case (non-pipelined) the client just closes that TCP
connection which terminates that requests with no impact on the other
requests.

Regards
Henrik

---

From: Fred Baker <[email-masked]>
Date: Thu, 25 Jun 2009 17:37:37 -0700

I would first ask how often this is an issue - I certainly stop http  
downloads from time to time, but it's not a primary behavior.

It also seems to me that the problem is self-correcting. If I have N  
potential streams and have opened M file downloads, of which I want to  
stop one, I can redirect the errant stream to /dev/null until the  
other downloads complete. At that point, it is either done or not  
done. If it completed in the meantime, I lost 1/M of my available rate  
during the event but nothing permanent happened. If it is not  
complete, I can kill the session and open a new one. And btw, since I  
will change the cookie, I could even reopen it using the same port  
number.

---

From: Henrik Nordstrom <[email-masked]>
Date: Fri, 26 Jun 2009 09:29:31 +0200

> downloads from time to time, but it's not a primary behavior.

When applied to a proxy environment (which is my field) it's very
common, as you then have a very high level of concurrency from very many
users. Partially aborted requests is a fairly high percentage of all
requests.

> done.

Yes, at a significant bandwidth waste. Not useful for me.

>  If it completed in the meantime, I lost 1/M of my available rate  
> during the event but nothing permanent happened.

Which if this was a proxy may be many orders of magnitude more bandwidth
than the download used while it was being forwarded to a client..

Regards
Henrik