Subject: [hybi] Framing Management Rationale

From: "Bob Gezelter" <[email-masked]>
Date: Wed, 17 Aug 2011 11:36:21 -0700

The other day, I registered my support for retaining the "Frame Too
Large" error and including a mechanism to declare maximum and minimum
frame sizes in each direction for WebSocket protocol connections. This
posting is my promised rationale on that support.

There have been several recent comments made in favor of very large
frame sizes. In particular, there was one comment suggesting that a
single huge frame would suffice for the entirety of a telephone
conversation. The stated goal was assuring performance by minimizing
overhead. This was accompanied by a reference to "sendfile." Presumably,
the "sendfile" referenced is the LINUX sendfile() (described in the
LINUX man pages at
http://www.kernel.org/doc/man-pages/online/pages/man2/sendfile.2.html)
as:

 "...Because this copying is done within the kernel, sendfile() is more
efficient than the combination of read(2) and write(2), which would
require transferring data to and from user space."

This description is not directly translatable into a network context.
sendfile() does not eliminate all overhead of the disk file operation,
the man page only describes eliminating the outermost layer: copying
data to/from the user-mode process. In the network space, this is
particularly misleading. A WebSocket protocol API equivalent of
sendfile() would also have to do significant processing even while
sending an entire, arbitrary file as a single frame. The frame, and
indeed message overhead would not be significant in this case. With all
due respect to the writer, such usage is not consistent with good use of
the WebSocket protocol, the underlying transport protocol (presently
TCP), or the physical realities of network communications. It does,
however, underscore the importance of carefully reviewing protocol
proposals in full detail and depth. 

Additionally, disk storage is not real-time telephony. A real-time
application (e.g., telephony) has two requirements: fidelity and minimal
latency. These goals are not compatible with large frame sizes. On the
contrary, relatively small frame and message sizes are best choices.

Of course, one can always circumvent framing by processing data as it
comes in, by parsing the payload incrementally, and ignoring all of the
features of protocol-based framing with ones locally developed framing.
This is effectively a bare TCP socket without protocol. Using the
WebSocket protocol to implement such an approach is eliminating the
WebSocket protocol from the operation in all but name, leaving what is
in effect a bare TCP connection with the extra overhead of the
WebSockets API (while not infinite, extended duration frames obliterate
the utility of PING/PONG and any future control extensions). Also, the
ability of the underlying network stack (e.g., TCP) to accept continuous
data in the face of network connectivity events is limited and will
further cause problems.
 
If stack traversal overhead is a major performance problem, the solution
lies in modifications to the API implementation to permit queuing of
multiple records with a single API call (or upcall to the networking
stack), a technique known as "blocking", which has been well-known in
computing since the 1950’s.

Indisputably, very small frame sizes sacrifice some efficiency; payload
is small relative to overhead. Increasing frame size provides space for
larger payloads;  dramatically increasing efficiency. However, the
increase in efficiency lessens in importance exponentially as the frame
size increases: 12 bytes of overhead/frame is 10% of a 128-byte frame,
it is less than 0.1% of a 12,800 (<2**14) byte frame. "Bandwidth
Efficiency" is simply not a compelling argument for frame sizes in
excess of 2**16. (The "Jumbo" frames on high-speed CSMA/CD networks
[e.g., gigabit Ethernet] are an entirely different issue. They are used
not for efficiency at high speeds, but rather are required to maintain
the integrity of the CSMA/CD arbitration scheme over the physical
network.)

A simple review of the realities of transmitting data clearly shows that
any application where a 0.01% loss of performance is unacceptable is an
application that is simply waiting to fail. Basic queuing theory is
clear:  a 0.01% safety margin is far from adequate to allow for any
non-uniform behavior in the underlying transport layers. 

WebSocket protocol is not a standalone protocol. Rather, the WebSocket
protocol is effectively a higher-level transport protocol, presently
layered on a TCP/IP base. Both protocols must be considered when
assessing decisions such as frame sizes. 

The vision of using a "sendfile"-like mega-frame to enable ultra-low
overhead is a chimera for many different reasons. Simply put, there is
tremendous gain from the initial steps to increase frame size, however,
in the end, the efficiency gained for each doubling of frame size is
half of the gain of the preceding step.

The related question is that of frame size limits, and how they should
be handled by the WebSocket protocol. The operative questions are: What
are the use cases of the WebSocket protocol? How do those use cases
relate to decisions taken in the design of other protocols (e.g., HTTP,
TCP)?

WebSocket protocol connections differ from HTTP connections, in that the
majority of HTTP connections are limited in duration. By contrast, the
WebSocket protocol is intended to provide a low-overhead facility for
communications in support of interactive web applications. It is
reasonable to presume that WebSocket protocol connections will have
lifetimes far longer than HTTP connections. 

The WebSocket protocol also enters a world different than that which
gave rise to TCP. The growing importance of mobile and un-tethered
devices raises the question of how protocol design changes affect
performance and usability in a network context when the underlying
network changes during the life of a connection. 

While TCP provides a stream-interface seemingly unconnected with the
underlying inherent packetization of IP, the stream interface is an
intellectual illusion of convenience. Performance and network efficiency
are inexorably linked to how the TCP data stream fits within underlying
IP packets. This phenomenon is even more familiar to those who have used
message-oriented protocols (e.g., DECnet’s NSP). Simply put, if a
message fits within a single underlying packet, it will be far more
efficient than a message that requires two packets. While this may
appear irrelevant when messages span many packets, it remains
significant in the aggregate. It is particularly significant when the
individual messages exchanged approximate the payload sizes of the
underlying packets, when protocol overhead is included. In those cases,
the removing a single byte, or a handful of bytes from a message can
nearly double throughput.

Frame-to-frame size variations are a also have significant potential to
create severe performance and resource problems. Buffer allocation is
one of the most costly operations in an operating system’s repertoire.
Constant allocations/deallocation of varying size memory blocks often
leads to significant overhead processing and memory pool fragmentation.
It is true that the buffers in question reside in user memory space, and
are managed by the JavaScript run-time environment (at least when the
integral JavaScript WebSockets API is in use). However, there are still
issues. Garbage collection is one of the more expensive operations, and
memory pool fragmentation leads to increased virtual size, which over
time leads to resource shortages. Continuously varying frame sizes is
not likely to be an efficiency improvement in most cases, the reduction
in frame overhead not withstanding. 

It is for this reason that I favor a dynamic adjustment mechanism for
frame size within the WebSocket protocol. Particularly with mobile
devices, which may wander repeatedly between underlying networks (e.g.,
GSM and WiFi), the ability to adjust flow control configuration without
need to disrupt ongoing connections is an important capability.

Such an approach provides a facility that can also be used to adjust for
asymmetrical bandwidth or performance between the different dataflow
directions.

I will draft a proposal for such a scheme and put it on the mailing
list. Obviously, I will be happy to discuss the above.

---

From: John Tamplin <[email-masked]>
Date: Wed, 17 Aug 2011 15:11:46 -0400

On Wed, Aug 17, 2011 at 2:36 PM, Bob Gezelter <[email-masked]> wrote:

> The stated goal was assuring performance by minimizing

It isn't just the overhead, but that the server can "fire and forget" the
response.  For example (ignoring error handling):

void sendFileAsMessage(int sock, char* fname) {
  int fd = open(fname, O_RDONLY);
  size_t len = getFileLength(fd);
  writeWSHeader(sock, len, WS_BINARY);
  sendfile(sock, fd, NULL, len);
  close(fd);
}

and then you are done with that file and don't need to do anything further
to have it sent.  Obviously, masking breaks this on the client->server
direction, but it seems likely that this would only be used on the server
anyway.

Note I was not the one advocating this use case as the reason for supporting
2^63-byte frames - just explaining the rationale as it was given.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 18 Aug 2011 11:39:26 +1000

Bob,

thanks for your analysis - a couple of points

> features of protocol-based framing with ones locally developed framing.

Is this really an allowable mode of operation?

Currently we have nothing in the specification that says when an
intermediary is required to forward data.  I do not  see any
compulsion for an intermediary to forward bytes as they are received
so any application that depended on the timely arrival of bytes within
a websocket frame would be as fragile as current comet solutions that
rely on streamed HTTP responses.

If fact there is also no requirement for an intermediary to forward
data on frame boundaries, as we allow intermediaries to aggregate
frames, thus we implicitly allow them to delay sending one frame until
the arrival of following frame(s).

I think there is an implicit assumption, that perhaps we should make
explicit, that intermediaries should forward as soon as possible after
the receipt of a message boundary.   With such an explicit
requirement, it would then be clear that applications that want
streaming telephony style communication should be using lots of little
messages rather than lots of frames or data blocks within frames.

> Continuously varying frame sizes is not likely to be an efficiency improvement in most
> cases, the reduction in frame overhead not withstanding.

In addition, a mismatch in buffer sizes between sender and receiver
can result in significant inefficiencies.

A sender with a larger buffer (and thus fragment size) may force a
receiver with a smaller default buffer size to continually reallocate
buffers.  A sender with a small buff (and thus fragment size) may talk
to a receiver with a large default buffer size, where much of that
buffer is wasted because large frames are never received.

Yes I know that implementations may be written to treat frames as
streams and avoid the need to hold an entire frame in a single buffer.
  But regardless of that possibility, there will be implementations
that will prefer to defer frame handling (including operations like
unmasking, aggregation, extensions, utf-8 conversion) until an entire
frame is received.

My own implementation can handle any sized frames via a fixed sized
receive buffer.  But I don't know how large to make that receive
buffer - if I guess 8k, but senders only ever send 2k frames, then I'm
wasting 6k per connection.   If I guess 2k, but senders use 8k frames,
then I have the inefficiencies of doing 4x the "frame" handling that I
need to. An application can set a max message size, but that is is
unicode characters and may not reflect a max frame size well,
specially if the sender is limiting their frames sizes.

I think the ability to declare a max frame size is useful, even in the
absence of a frame-too-large error.  For my implementation, it would
at least allow significant memory savings by right sizing the buffers.
    The specification explicitly allows senders to limit frame size by
their own buffer size.  If we allow such limits to exist, I fail to
see the benefits of keeping those limits secret.

I do understand Andy's concern that max frame size might be wink to
poor receiver implementations - so how about a third option:  An
optional header to declare min/max frame size, but remove the
frame-too-large error so as to indicate that implementations must be
able to handle any frame size up to their max message size limit.

---

From: "Andy Green (林安廸)" <[email-masked]>
Date: Thu, 18 Aug 2011 09:20:35 +0100

> posting is my promised rationale on that support.

Phew this is an enormous email Bob.  Much of this email is talking about 
other stuff like sendfile.

I would have thought if there's a good reason for websockets adding 
frame mtu you could sum it up in three short paragraphs.

> http://www.kernel.org/doc/man-pages/online/pages/man2/sendfile.2.html)

I agree sendfile and large frames don't hold any water for me either. 
Sendfile hides a lot of stuff under one simple API call, well, so does 
sending a websocket message in Javascript in one call.  When you add in 
masking processing, introduction of huge latencies and that many times 
the payload is not coming from a file, it sounds like we should ignore it.

> comes in, by parsing the payload incrementally, and ignoring all of the

Before you can call that circumvention rather than "processing the frame 
payload optimally at frame layer" you need to explain what you think the 
framing is additionally "for" if not just a segmentation scheme allowing 
other frames to be interleaved.

> is small relative to overhead. Increasing frame size provides space for

Right it's about the ratio of payload that's delivered inside the frame 
to the number of framing bytes.  It's a game of diminishing returns 
though; the incremental efficiency for having a 65K frame size instead 
of 64K frame size is really very small.

> the integrity of the CSMA/CD arbitration scheme over the physical

Again this is a whole other subject that explaining the need for frame 
mtu, but again, I agree with you 2^63 frames are either worthless or 
evil depending on how you look at it.

> application that is simply waiting to fail. Basic queuing theory is

OK so I learn from that not to sweat, eg, 64K frame size or 65K.

(More stuff about sendfile snipped)

> The related question is that of frame size limits, and how they should

Alright!  It's here!

> relate to decisions taken in the design of other protocols (e.g., HTTP,

The operative question is what does introducing maximum frame size 
concept buy us when we look at real-world implementations of the frame 
processing code that currently don't need it.

Even Greg's telling thismorning that he is using the robust packet 
buffer spill to message processing scheme you referred to as 
"circumvention" and I believe that is optimal.

> lifetimes far longer than HTTP connections.

Okie.

> network changes during the life of a connection.

That really confused me, I think you're talking about the network 
performance for that link varying over time.  It sounded like you meant 
automatic network handoff, like 3G -> wlan and you're suddenly on a 
different network.

> efficient than a message that requires two packets. While this may

Sounds reasonable at IP layer.  Of course, if your message can fit in 
your actual mtu at that layer is up to the protocol and the payload.

> nearly double throughput.

Hum well, it can eliminate ACK RTT at least.  If the messages was tiny, 
like one byte, then yes not having to do the extra framing will be 
significant advantage.

So... this is an argument for introduction of frame mtu... how?

> Constant allocations/deallocation of varying size memory blocks often

Er... what?  The OS does not know a websocket frame from its elbow.

All it sees is a bunch of packets coming.

> It is true that the buffers in question reside in user memory space, and

There are no buffers dependent on frame size necessary in userland or 
anywhere else.  I'll sum this up and explain it again in a moment for you.

> in frame overhead not withstanding.

Yeah garbage collection is expensive but this is nothing to do with the 
issue due to your wrong idea above.

> need to disrupt ongoing connections is an important capability.

OK.  The mail is over and I can describe what seems to be the mistake at 
the end in the bit that's relevant.

If you read Greg's reply he tells you:

''...My own implementation can handle any sized frames via a fixed sized
receive buffer. ...'' [1]

The point is he has written the critical frame processing code at hand 
as have I.  What he's explaining to you is that there is no per-frame 
malloc or whatever you had in mind for the frame processing which is the 
basis of your suddenly jumping to a frame mtu being necessary due to 
"memory fragmentation" and "garbage collection".  No allocations are 
happening in frame layer at all.

Instead at the frame processing layer, there's a fixed size buffer 
allocated either on the stack frame or static if it's done 
singlethreaded.  That buffer size is unrelated to the size of websocket 
frames expected to appear.  So, you could set it to 64K or 4K or what 
you think is good according to how many packets might be waiting in the 
OS to fill it; in singlethreaded method like libwebsockets there's only 
one of them in the whole library instance so the size is not critical 
resourcewise.

You read() packet content into that buffer, parse out the headers 
bytewise so you don't care if they straddle a buffer boundary, and pass 
the remainder of the buffer or the frame payload in there up to the 
message handling layer, via a pointer in my case.  The message handler 
says, "oh, nice, xyz bytes more message came, I shall take care of 
that".  And if there is more left in the fixed buffer, or more packets 
waiting without blocking, then we go around again passing that payload 
up to the message layer.

That is why I say frame size, framing as a whole has no meaning outside 
of stripping it off and immediately passing the payload up as message 
payload.  We should have language to actively discourage assumption that 
received frame sizes or start and end have anything to do with what was 
sent to avoid fragility from intermediaries doing their legit thing.

Nowhere in there does negotiation of max frame size buy us anything. 
Even if you want framing to react dynamically to throughput conditions, 
with control frame latency in mind, that's something the sender and 
intermediary senders can monitor and do themselves without any 
negotiation with the receiver.

-Andy

[1] Greg goes on to wonder what size that buffer should be, but so long 
as it's much larger than the mtu of the underlying transport (not frame 
mtu) and can dump out much of the packets the OS has pending in kernel 
memory already for us, the size should not be critical for performance.

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 18 Aug 2011 01:55:30 -0700

> intermediary is required to forward data.  I do not  see any

I totally agree: there should be something in the spec.

> messages rather than lots of frames or data blocks within frames.

So streaming an audio file will mean you cannot have "audio file" =
"message", have the data sent in small frames, and the audio
playback start immediately, since interm. might buffer the whole file?
So I need to invent my own fragmentation and put those fragments into
little WS messages?
Then why do we have message fragmentation in WS at all?

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 19 Aug 2011 08:53:25 +1000

> Then why do we have message fragmentation in WS at all?

We  have fragments so an implementation may flush a buffer before it
knows the full size of a message.
We also have fragments so that future MUX implementations can avoid
channel occupancy.

We don't have fragments to communicate some kind of semantic boundary
within a message.

cheers

---

From: Tobias Oberstein <[email-masked]>
Date: Thu, 18 Aug 2011 16:09:28 -0700

> We don't have fragments to communicate some kind of semantic boundary

Yes to all three, but how does that relate to the example?

> the receipt of a message boundary.   With such an explicit

No, they should forward _frames_ as soon as possible. Ideally, they should
forward even data within frames as soon as possible. Moreover they should
not "abuse" their right to coalesce frames to introduce unnecessary/unwanted
delays.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 19 Aug 2011 09:36:56 +1000

> Yes to all three, but how does that relate to the example?

Frames should not be exposed to an application layer, so any audio
streaming application will not be able to send data in WS fragments.
 However an audio application can send messages.    So an audio
application could get an audio stream and break it into 125 bytes
binary messages and pass each of those to it's WS API.  Each would be
sent as a message (probably as a single frame).  The other end would
receive each of these messages and add the binary content to it's
audio play buffer.

Framing overhead will be 2 bytes per 125 bytes of data.    Alternately
the app could use larger byte messages and attempt to optimise for
TCP/IP MTU , with an overhead of 4 bytes per ~1500 bytes of data.

> not "abuse" their right to coalesce frames to introduce unnecessary/unwanted

But that is contradictory.  How can an intermediary coalesce fragments
without delaying the first fragment.

Also, I see no reason for an intermediary to forward on data.   If it
is receiving a stream of 1B TCP/IP packets, then there is a lot to be
gained from shielding the application servers from such inefficient
traffic.  Batching data in a latency vs throughput tradeoff can be a
very good thing to do for many applications.

There can also be a case for forwarding bytes ASAP, but I don't think
that any WS application should be written to rely on such path
optimisations.

cheers

---

From: Tobias Oberstein <[email-masked]>
Date: Fri, 19 Aug 2011 03:35:56 -0700

>>>> So I need to invent my own fragmentation and put those fragments 
>>>> into little WS messages?

>Frames should not be exposed to an application layer, so any audio streaming application will not be able to send data in WS fragments.

Frames should not be exposed, but that does not imply that an application cannot stream.

InputStream is = new InputStream("myaudio.mp3");
ws.sendBinaryMessage(is);

The latter returns immediately, the underlying WebSockets implementation just pulls
the input stream (in arbitrary chops), generates frames and pushes data out on TCP.

It's a question of API, not protocol.

>> frames, thus we implicitly allow them to delay sending one frame 
>> until the arrival of following frame(s).
>>
>> explicit, that intermediaries should forward as soon as possible 
>> after the receipt of a message boundary.   With such an explicit
>
> No, they should forward _frames_ as soon as possible. Ideally, they 
> should forward even data within frames as soon as possible. Moreover 
> they should not "abuse" their right to coalesce frames to introduce 
> unnecessary/unwanted delays.

> But that is contradictory.  How can an intermediary coalesce fragments without delaying the first fragment.

Yes, that's why I've said "should" and "not abuse".

What is the value of allowing intermediaries to coalesce frames?

I can see the benfit some might have to fragment, but coalesce .. what's the use case for intermediaries?

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 19 Aug 2011 21:28:41 +1000

> It's a question of API, not protocol.

Except that we currently don't know if the protocol is required to
expedite frame delivery.
An implementation might read the entire input stream to create a
single frame message containing the entire mp3.

> I can see the benfit some might have to fragment, but coalesce .. what's the use case for intermediaries?

Consider an intermediary that is muxing many different connections
(from clients) onto a smaller number of connections to an application
server.  This is often done in HTTP now to reduce the number on
connections on the application server.      The Mux will fragment
large inbound and outbound messages to small frames in order to avoid
channel occupancy.  However there is no reason that the outbound
messages should be sent in small frames and there may be some benefits
in aggregating back to larger frames, specially if there is some other
per frame extension involved.

Conversely, inbound messages might arrive in many tiny frames due to
some other transport/client limitation, an intermediary might
aggregate those into an entire message before forwarding to the
application server - so the server does not need to deal with any IO
on a channel until the entire message has arrived at the gateway and
only has the LAN to cross.

regards

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 19 Aug 2011 20:46:47 +0100

John Tamplin wrote:
>    On Wed, Aug 17, 2011 at 2:36 PM, Bob Gezelter <[1][email-masked]>
>    wrote:
> 
>      [2]http://www.kernel.org/doc/man-pages/online/pages/man2/sendfile.2.
>      html)

Most server OSes have an equivalent function, sendfile() is just the
Linux name for it.  It's not a Linux feature, it wasn't even first.

>      Â "...Because this copying is done within the kernel, sendfile() is
>      more efficient than the combination of read(2) and write(2), which would
>      This description is not directly translatable into a network
>      context.  sendfile() does not eliminate all overhead of the disk file
>      operation,

The man page is misleading.

sendfile() does eliminate most of the copying done in software.

But that's not really the point, because copying isn't slow on modern systems.

The other performance characteristic is that it *shares memory* among
large numbers of sockets.

It applies to dynamically produced messages too -- not just files.

An application which generates transient data in memory and will
broadcast the same large data fragments to many WS clients, can use
sendfile() on a shared memory file, or its relative vmsplice(), to use
the same memory buffers in the queued of large numbers of outgoing sockets.

And these are just what current OSes provide.

>      data to/from the user-mode process.

Yes, but the man page doesn't cover all the salient characteristics.

>      In the network space, this is
>      sending an entire, arbitrary file as a single frame.

>From the server to client, because there's no masking in that
direction, it can share the memory.  (Or has the WS spec changed on
this again?)

At the level of (server) WebSocket APIs, you don't necessarily need
any more API for the implementation to be using these techniques
underneath its normal sending and buffering procedures.

Yes it does significant processing, but less than you might think.  In
particular, it does not have to process every TCP packet -- that is
offloaded to the network card these days.

>      The frame, and indeed message overhead would not be significant
>      in this case

That's precisely the point.

>      With all due respect to the writer, such usage is
>      not consistent with good use of the WebSocket protocol, the
>      underlying transport protocol (presently TCP), or the physical
>      realities of network communications.

With respect, good use of WebSocket protocol is unknown at this point.

Sendfile-like mechanisms work well with TCP, largely because
server-level network hardware provides acceleration for it.

>      Note I was not the one advocating this use case as the reason
>      for supporting 2^63-byte frames - just explaining the rationale
>      as it was given.

sendfile() is *not* the reason for 2^63 frames.

Please don't perpetuate that myth.  That accolade belongs to "no
rarely exercised code branch due to arbitrary but occasionally
reached limit".

For sendfile, 2^24 would be plenty.

The sendfile rationale is that 2^16 limit is a bit small if there's no
good reason for it (being, after all, the size of a single TCP packet
over jumbo frames ethernet.)

Whether sendfile, or indeed larger than 2^16 frames, are useful will
be found by implementation experience, and changes in the net over the
coming years.  The point was to not rule out a plausible
implementation strategy, especially where large numbers of clients
connect to few servers, in the absence of a good reason.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 19 Aug 2011 21:11:57 +0100

> TCP/IP MTU , with an overhead of 4 bytes per ~1500 bytes of data.

WS design means streaming applications which don't need WS message
boundaries must create artificial boundaries just for this purpose and
do their own fragmentation layer in the app, and an intermediary is
forbidden from coalescing those, even though some applications would
be happy to have them coalesced.

I.e. those apps have to invent their own app-level fragmentation
protocol on top of WS.

I'm thinking of anything that looks like byte-stream-over-WebSocket,
which if you think about it, includes MP3-over-WebSocket.

> > not "abuse" their right to coalesce frames to introduce unnecessary/unwanted

It's worse than that: They can't safely forward partial frame data at
all, as the remainder of the frame to be forwarded may not arrive in a
useful time, and then the intermediary is stuck, if it needs to send
a control (shutdown, error, MUX...).

For control frames to work properly, an intermediary MUST either delay
whole frames, or refragment to what it has available on sending.

Like TCP and HTTP chunked encoding do.

> without delaying the first fragment.

Easily.  I believe the smart way is coalescing when the previous
outgoing fragment hasn't left the system, be it contention, receiver
flow control, or just not yet got around to scheduling the previous
transmit yet.  In other words, look at the outgoing queue.

An intermediary like that would get rid of _all_ fragment boundaries
as soon as it receives the data, and it would insert new fragment
boundaries when it sends data to represent the amount available in the
send buffer at that moment.

> very good thing to do for many applications.

Yes.

But that argument also applies when you get a fragment boundary.  The
received fragment boundary gives no useful information about the
efficiency of the next hop.  And since it also has no behaviour the
app can depend on, there is no reason to do anything with it other
than discarding it, and regenerate fragments when sending according to
real-time send buffer size.

Fragment boundaries should be treated like HTTP chunks or TCP
segments.  Something to be discarded when you receive the data, even
if you're an intermediary.  They only exist so the remote sender has a
way to inject something other than more bytes of the same message.

> that any WS application should be written to rely on such path

I agree, which is why I asked for the forwarding behaviour to be
flagged explicitly in each frame, so things like
byte-stream-over-Websocket don't have to invent their _own_
intermediary-unfriendly fragmentation layer on top of WS messages.

All the best,
-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 19 Aug 2011 21:38:03 +0100

> specially if the sender is limiting their frames sizes.

That doesn't make sense.  If sender has 8k of data for you, it will
send 4 x 2k frames or 1 x 8k frame, depending on "max frame size".

Either way, your efficient buffer size is 8k bytes, which you then
parse into frames, and pass around for any special processing.

If you are receiving from the socket 2k bytes at a time when there's
8k available (whether in multiple fragments or one), not only is that
inefficient, and it doesn't help you anyway because you will still 
have fragment boundaries unaligned with the your buffer.

> For my implementation, it would at least allow significant memory
> savings by right sizing the buffers.

I don't see how it helps, because the "right size" buffers are
whatever will receive the incoming socket bytes in the fewest
socket-receive operations with minimal wasted space -- which depends
on the byte rate of the sender (relative to receive socket-read rate),
which is not affected by the fragment size.

You may well take the received byte stream (received as blocks of
bytes from the socket) and split it into WS fragments, maybe copying
them into separate fragment buffers if you are doing fancy processing,
or to keep the code simple.

But when you do that parse & split, you know *exactly* the "right size"
to allocate for each fragment.

(There are strategies for efficiently limiting unused buffer space at
the blocks-received-from-socket level, but that's another topic.)

> absence of a frame-too-large error.

I agree, it is a sensible option, but not for the "right size buffer"
reason.  It's for the "keep it reasonable for my application doesn't
like working on big buffers" reason.

Very similar to TCP MSS, and invisible to applications in the same
way.

Except WS "max frame size" should be hop-by-hop (unlike TCP MSS) --
there is no point in it being end-to-end, as fragments themselves are
hop-by-hop.

> see the benefits of keeping those limits secret.

But do make it hop-by-hop.  It would be silly if the final receiver
limited the frame size for hops other than the last one; there is no
efficiency gain from that.

> able to handle any frame size up to their max message size limit.

Who's min/max frame size goes in the header?
Intermediary 1, intermediary 2, or the end receiver?

All the best,
-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Fri, 19 Aug 2011 21:54:52 +0100

"Andy Green (林安廸)" wrote:
> The point is he has written the critical frame processing code at
> hand as have I.  What he's explaining to you is that there is no
> per-frame malloc or whatever you had in mind for the frame
> processing which is the basis of your suddenly jumping to a frame
> mtu being necessary due to "memory fragmentation" and "garbage
> collection".  No allocations are happening in frame layer at all.
> 
> singlethreaded.  That buffer size is unrelated to the size of
> websocket frames expected to appear.  So, you could set it to 64K or
> 4K or what you think is good according to how many packets might be
> waiting in the OS to fill it; in singlethreaded method like
> libwebsockets there's only one of them in the whole library instance
> so the size is not critical resourcewise.
> 
> bytewise so you don't care if they straddle a buffer boundary, and
> pass the remainder of the buffer or the frame payload in there up to
> the message handling layer, via a pointer in my case.  The message
> handler says, "oh, nice, xyz bytes more message came, I shall take
> care of that".  And if there is more left in the fixed buffer, or
> more packets waiting without blocking, then we go around again
> passing that payload up to the message layer.

Well put.

> That is why I say frame size, framing as a whole has no meaning
> outside of stripping it off and immediately passing the payload up
> as message payload.  We should have language to actively discourage
> assumption that received frame sizes or start and end have anything
> to do with what was sent to avoid fragility from intermediaries
> doing their legit thing.

I agree completely - any "meaning" of the frame boundary should be
actively encouraged, and examples should show frame boundaries being
stripped at the point they are received, so nothing else sees them.

That includes intermediaries receiving frames to be forwarded.  They
should strip away received boundaries straight away, and introduce
_their own_ frame boundaries on the sending side.

Frames are fundamentally a hop-by-hop concept -- or they should be.

Messages are the end-to-end concept.

This is exactly the same as HTTP's chunked encoding (hop-by-hop) and
HTTP messages (end-to-end).

(Don't confuse this with TCP's segments that live end-to-end and are
hidden at the final receiver.  That's the wrong example.)

> Even if you want framing to react dynamically to throughput
> conditions, with control frame latency in mind, that's something the
> sender and intermediary senders can monitor and do themselves
> without any negotiation with the receiver.

I agree completely.  Each sender's job is to decide where to insert frame
boundaries.  This includes intermediary senders.

Consequently, frames cannot have any application-dependent meaning --
including any aspect of dependable timing.

Really that leaves only two extremes for forwarding guarantees:
Eagerly forward each byte subject to local buffering optimisations
(like HTTP CONNECT proxies used for HTTPS must do), or permit delaying
up to a message boundary.

There are heuristics, policies and "best practice" options in the
middle.  They might be what happens (given it is not specified, and
cannot be entirely specified in the real world of TCP.)  But whatever,
they should not care about fragment boundaries; those should be
stripped as soon as received.

(In my humble opinion of course :-)

All the best,
-- Jamie

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Sat, 20 Aug 2011 14:01:41 +0200

> direction, it can share the memory.  (Or has the WS spec changed on

Are we assuming plain WebSocket? if the connection uses TLS/SSL
(wss://) then all this discussion is useless, am I right?

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: Jamie Lokier <[email-masked]>
Date: Sat, 20 Aug 2011 14:44:14 +0100

> (wss://) then all this discussion is useless, am I right?

Yes, with an exception when TLS/SSL is added/removed at an intermediary
(e.g. offloaded to front-ends in a server farm).

The intermediaries also have to forward every byte on TLS hops, and
can't refragment, so the other part of this discussion about
buffer/timing/fragments does not apply either.  Intermediaries will
still delay (if only because TCP does and internal schedule decisions)
but purely heuristically.

However, this does not rule out MUXing multiple TLS streams in principle!
Although the MUX designs we have seen, and the natural TLS binding for
WebSocket, cannot do that.

-- Jamie

---

From: Greg Wilkins <[email-masked]>
Date: Sun, 21 Aug 2011 20:44:32 +1000

> Intermediary 1, intermediary 2, or the end receiver?

I totally agree that frame size should be hop by hop, so any min/max
frame size received would only be applicable for the adjacent hop.

cheers