Subject: Re: [hybi] On Fragmenting of Frames

From: Bob Gezelter <[email-masked]>
Date: Fri, 15 Apr 2011 05:40:55 -0500

Nowhere in the recent discussion of Frame Length have I seen a reference to
the realities of extended length (> 64KB) frames. Instead, I have seen
references to requiring extended frame sizes to support "sending extended
messages with no overhead". With all due respect for hyperbole, this concern
is not supported by the realities of the network. [In the interests of
smooth reading, I am omitting the citations from the following couple of
paragraphs.]

The WebSocket protocol is emphatically TCP-based (however, my concerns also
hold true for WebSocket protocol implementations over other network
protocols, e.g., SCTP). Even "Jumbo Frames" limit the MTU to less than 10K
bytes (octets). Of necessity, a WebSocket frame of more than slightly less
than IP MTU (upon which TCP is reliant) will involve the chunking of the
message into a number of packets, and the reassembly of those packets at the
destination into a single, sequenced stream. This is not negotiable, it is a
fundamental part of TCP, which is in turn dependent on IP.

Even if we were to ignore the overhead and processing of IP and TCP, the
realities of the network make it unwise to use extended frame sizes at the
WebSocket protocol level. Extended frame sizes quickly create large
potentials for network and application problems. Consider the case of a
1Gbps network. 1Gbps translates to a theoretical (emphasis: THEORETICAL)
maximum bandwidth of 125 MB/sec. Sending gigabytes of traffic in one frame
means that the time until the next frame is measured in seconds. This is,
with all due respect, not a realistic presumption, even on a LAN. If one
includes wide area networks, it is even farther from real-world feasibility.

Protocol overhead drops quickly with packet length to a point. Once frame
overhead is well less than 1%, it becomes statistically insignificant. It
is worth noting that Packet Switching was chosen for the ARPAnet because it
solved a number of problems, one of which was latency. Message switching
long predated packet switching, and was well-known to have issues. Hence,
packet switching.

The latency of PING and PONG are directly affected. If an extended length
frame is transmitting, PING and PONG will be delayed. For that matter,
since no event will be signaled until end of frame, seconds could pass
before ANY event is noticed. This is a poor idea, even when transmission
errors are handled at lower levels. I can easily see timeouts being
declared whose root cause is nothing more than a very large frame. [We
have been here before; before ubiquitous broadband, it was common when
running error-correcting modems over direct dialed circuits. It was
difficult to justify to users, and difficult to track down after the
fact. It was not something that is worth repeating.]

In summary, I would be all for limiting the frame size to 16 bits on grounds
of latency and efficiency. Larger frame sizes are not a demonstrable
performance gain. If an application is dependent on the performance
difference between a 64Kbyte frame and a 1Mbyte frame, it in all likelihood
is far too close to the edge of what can be achieved reliably. We do no one
a service by encouraging unreliable design.

- Bob
+--------------------------------------------------------------------------+
| Robert "Bob" Gezelter                       E-Mail:  [email-masked]  |
| Robert Gezelter Software Consultant                                      |
| 35-20 167th Street, Suite 215               Fax:       (on Request)      |
| Flushing, New York  11358-1731                                           |
| United States of America                    http://www.rlgsc.com         |
+--------------------------------------------------------------------------+

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 15 Apr 2011 15:01:37 +0200

2011/4/15 Bob Gezelter <[email-masked]>:
> protocols, e.g., SCTP)

This is a good point. Modern Internet communication protocols don't
restrict themself to a specific transport protocol (as TCP). Instead,
their respective RFC's talk about "reliable transport protocols".

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: "Arman Djusupov" <[email-masked]>
Date: Fri, 15 Apr 2011 16:02:46 +0300

> The latency of PING and PONG are directly affected. If an extended 
> length frame is transmitting, PING and PONG will be delayed. For that 
> matter, since no event will be signaled until end of frame, seconds 
> could pass before ANY event is noticed. This is a poor idea, even when 
> transmission errors are handled at lower levels. I can easily see 
> timeouts being declared whose root cause is nothing more than a very 
> large frame. [We have been here before; before ubiquitous broadband, 
> it was common when running error-correcting modems over direct dialed 
> circuits. It was difficult to justify to users, and difficult to track 
> down after the fact. It was not something that is worth repeating.]

Sending a 64KB frame over a slow GPRS network can delay the PING/PONG for as
much time as sending a 1GB frame over a local network. We can't assume that
by setting the maximum frame length to 64KB we are ensuring the reliable
delivery of control frames, because this is too dependent on the network
condition. Under some circumstances even a few bytes long frame cannot be
delivered because the remote side doesn't read from the connection due to
high load or channel throttling rules.

At the end of the day it's the sender that controls the outgoing frame
sizes. If the sending side wants to improve the delivery of its control
frames it can always choose to send frames of smaller size or even implement
an algorithm that would adjust the frame size depending on network
conditions and other requirements.

> In summary, I would be all for limiting the frame size to 16 bits on 
> grounds of latency and efficiency. Larger frame sizes are not a 
> demonstrable performance gain. If an application is dependent on the 
> performance difference between a 64Kbyte frame and a 1Mbyte frame, it 
> in all likelihood is far too close to the edge of what can be achieved 
> reliably. We do no one a service by encouraging unreliable design.

I can agree that 64KB is OK as a frame size for most of use cases today, but
this might be too small for the faster networks of tomorrow, or too big for
some compact devices still in use today. In my opinion the frame length
field should allow a negotiable/configurable frame size within reasonable
boundaries, rather than be capped to 64KB.

With best regards,
Arman

---

From: Bob Gezelter <[email-masked]>
Date: Fri, 15 Apr 2011 08:10:09 -0500

Inaki,

Agreed. with a caveat.

Functional requirements of underlying layers, as opposed to specification of
a particular underlying layer is not a "modern" phenomenon. See RFC 772,
Suzanne Sluizer and Jon Postel (1980, September) “Mail Transport Protocol”.

The WebSocket protocol specification reads far clearer with the TCP'isms
relegated to a separate dependent RFC, a point I noted in my blog posting
the other week [see http://rlgsc.com/r/20110323.html].

- Bob

---

From: Bob Gezelter <[email-masked]>
Date: Fri, 15 Apr 2011 08:18:51 -0500

Arman,

Agreed. I meant the 64K as a "Straw Man". Actually, IMHO, something more
along the size of a Jumbo Frame is more appropriate (e.g., 8-10KBytes).

When protocol overhead drops below a small fraction of bandwidth (for
the sake of argument, 1% by volume) successive doublings of frame size
become all but irrelevant (32 bytes of overhead/10K is already 0.32%
overhead); further increases do not appreciably reduce overhead.

In most cases this is not an buffering problem, as the underlying transport
guarantees a sequenced stream of error-free bytes. Rather, the problem is a
question of latency.

Admittedly, it is all but impossible to determine the speed/available
bandwidth of the limiting path of a connection, and the limiting path might
change for any number of reasons (e.g., railcar derailing and damaging a
cable).

For these reasons, I advocate the use of fairly short frames. In any event,
it would take a major change in physical constants before I could see
justifying 1Mbyte+ frame sizes as providing a significant gain.

- Bob

---

From: Andy Green <[email-masked]>
Date: Fri, 15 Apr 2011 14:22:49 +0100

On 04/15/2011 02:02 PM, Somebody in the thread at some point said:

Hi -

> high load or channel throttling rules.

You're right, but it's actually easier to solve with 64KByte frame 
limit.  PING and PONG timeout number is also undefined so it can just 
tend to cover the worst case "good enough link" experience with 64KBytes 
blocking it.

You can't take that approach if the ceiling is left at 2^63.

> boundaries, rather than be capped to 64KB.

Worst that can happen is you need to send lots of 64KByte frames, it 
doesn't sound so bad.  If your code keeps sending while POLLOUT is 
coming you have plenty of frames prepared in the pipe and each frame is 
multiple packets on the wire, so its ability to fill available bandwidth 
shouldn't be hampered.

-Andy

---

From: Iñaki Baz Castillo <[email-masked]>
Date: Fri, 15 Apr 2011 15:28:17 +0200

> Suzanne Sluizer and Jon Postel (1980, September) “Mail Transport Protocol”.

I should replace "modern" with "other". Thanks for pointing it out.

> the other week [see http://rlgsc.com/r/20110323.html].

Awesoming article. I hope the specification will be improved/fixed
taking into account the issues you describe.

-- 
Iñaki Baz Castillo
<[email-masked]>

---

From: John Tamplin <[email-masked]>
Date: Fri, 15 Apr 2011 09:43:24 -0400

> smooth reading, I am omitting the citations from the following couple of

This whole discussion is addressing the wrong point -- those arguing for
incredibly long frames were not doing so for raising the percentage of
payload bytes on the wire, but rather for efficiency of the sender.
 Basically, they wanted to be able to "fire and forget" sending a huge file
(with sendfile and similar constructs), with the expectation that as
networks get faster larger files will be feasible.  Rather than assuming
some smaller maximum file size is "good enough", it seemed inexpensive to
allow 8-byte lengths and be sure it was enough.

If you haven't read the archives surrounding the discussion of framing
choices, you cannot imagine how contentious it was, so I would be loathe to
revisit this decision.  In the worst case, we have wasted the initial length
byte=127 case if this is never used, or 1/7th of a bit.  That doesn't seem
worth arguing over.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: "Arman Djusupov" <[email-masked]>
Date: Fri, 15 Apr 2011 18:32:10 +0300

> In the worst case, we have wasted the initial length byte=127 case if 

> this is never used, or 1/7th of a bit.  That doesn't seem worth 

> arguing over.

My primary hope was that we could negotiate the max message size. IMO this is important; as I explained in an earlier message, if the receiving side has a size quota that is not known to the sending side, the sending side might end up sending a large amount of data before the receiving side realizes that its size quota has been reached and denies receiving any more data.

With best regards,

Arman

---

From: Willy Tarreau <[email-masked]>
Date: Fri, 15 Apr 2011 17:46:17 +0200

> My primary hope was that we could negotiate the max message size. IMO this is important; as I explained in an earlier message, if the receiving side has a size quota that is not known to the sendin…

Well, this behaviour already works well with HTTP and does not cause
any particular problem to browsers. Moreover, having a mandatory message
size voids the possibility to send compressed stream without buffering
them all at once first.

Look how it is done with HTTP. Want to retrieve a static object ?
Content-length is used (= message size here). Want to use a dynamic
object or a compressed object ? Chunked encoding is used (= frame size
here).

So it's important not to make message size something mandatory, otherwise
some useful features will not be possible anymore.

Regards,
Willy

---

From: Bob Gezelter <[email-masked]>
Date: Fri, 15 Apr 2011 11:11:29 -0500

Willy,

Last time I checked the draft specification (this morning), message size was
distinct from frame size, to wit, messages are composed of one or more
frames.

Thus, even setting the most conservative limit on frame size (< a jumbo
frame; about 9kb) says nothing about how long a message can be.

The above is independent of my well-known belief that multiplexing is needed
in the base WebSocket protocol.

- Bob

---

From: Jamie Lokier <[email-masked]>
Date: Sat, 16 Apr 2011 19:00:19 +0100

> You can't take that approach if the ceiling is left at 2^63.

2^63 was chosen precisely because it doesn't impose an arbitrary
limit, as any limit that functionality _depends_ on an arbitrary
threshold that is good for one network will inevitably be a very poor
choice for another one.

This means any regulation of frame queuing/blocking latencies or
buffer limitations must be addressed by other means - explicitly - and
scaled with the network's characteristics.

The 1Gbit/s ethernet example is a poor choice, because that is not
even fast by current standards.  Current high-end laptops have
10Gbit/s connections, 100Gbit/s ethernet is already standardised and
has available hardware, terabit ethernet is the next goal, and it
seems likely that within 10 years, 10 terabit wired connections will
be available on high-end consumer hardware if there is a commercial
demand.

> multiple packets on the wire, so its ability to fill available bandwidth 

You have focused on byte overhead (for which your argument makes
sense), and have neglected the example of sendfile which can use
hardware-assisted TCP and shared memory for duplicate transmissions
*only* when the WS frames are *significantly larger* than TCP packets.
That's because TCP hardware assistence tends not to be able to splice
together shared preexisting data with program generated headers at
arbitrary offsets in TCP packets.  The memory sharing is the main
benefit for scaling up, not the avoidance of memory copies.

Sendfile is not he point - that's just a specific current
implementation detail, for which there are probably no current
applications.  

The point is the fact that arbitrary cut-offs do have unwanted effects
that are unnecessary, and there is no obviously good reason for those
cut-offs in this instance.

WS is aimed at the next 10 years, during which it is virtually
guaranteed that large messages will get used if they work reliably,
and much faster networks than we have now will be used by a
significant subset of users.

It is common to send replacement HTML for parts of pages inside
messages, and that can easily run into megabytes - these days, let
alone in 10 years when such a thing is likely to be so cheap there is
no point doing anything else.  So large messages are expected, and
then the question would be why did we require them split *arbitrarily*
into little tiny 64kb fragments handled by the CPU on a terabit
network which has hardware-assisted TCP/SCTP/future at both ends?

I agree that splitting into small fragments is good for a variety of
reasons, including latency, receiver capability and ability to inject
control messages.  But the splitting size is not something we should
casually fix for the lifetime of WS, as network speeds and application
uses already vary over an enormous range.

-- Jamie

---

From: Jamie Lokier <[email-masked]>
Date: Sat, 16 Apr 2011 19:05:54 +0100

Arman Djusupov wrote:
>    My primary hope was that we could negotiate the max message size. IMO
>    this is important; as I explained in an earlier message, if the
>    receiving side has a size quota that is not known to the sending side,
>    the sending side might end up sending a large amount of data before
>    the receiving side realizes that its size quota has been reached and
>    denies receiving any more data.

That's a flow control problem or a message size problem, not a
fragment size problem.

If the problem is you want to be able to send a control frame (such as
"close" or "abort") and the receiver would block receiving, the
solution is positive token flow control (which does limit the fragment
size - but as a side effect).

Setting the fragment size alone *doesn't* solve that problem.

If the problem is the receiver has a limit on how big a message it can
accept at all, you want to limit the message size, not the fragment
size, and it's clear that there is no good *fixed* message size limit
for the broad range of applications and networks.

-- Jamie

---

From: Andy Green <[email-masked]>
Date: Sat, 16 Apr 2011 19:55:04 +0100

On 04/16/2011 07:00 PM, Somebody in the thread at some point said:

Hi -

> sense), and have neglected the example of sendfile which can use
...
> Sendfile is not the point - that's just a specific current

I 'neglected' something that's 'not the point' and has 'no current 
applications', it sounds like I am doing alright.

> network which has hardware-assisted TCP/SCTP/future at both ends?

To follow your argument, hardware assist may be be able to handle 
something as simple as frame chopping for websockets "in ten years" too, 
what you're talking about is hardware-assisted chopping of frames into 
packets already.  In that case it's neutral about frame size limit.

> uses already vary over an enormous range.

Yes there is pressure to keep the frames small in reality, expecially in 
x-google-mux case where they can bind 8192 subchannels each blocked 
until other senders' frames are sent.

I think we can agree if you are subchannel #8191 and have some kind of 
latency requirement you will not be happy all the other subchannels are 
sending 2^63 byte frames (each taking 5 months to transmit on a 10Tbit/s 
link, so he gets his turn 3 Millenia later... bit laggy).

I take the point the pressure to keep frames 'small' is relative to the 
channel capacity and MTU and that tends to increase though.  How about 
2^32 that's blocking 10Tbit/s link for 3ms.

-Andy

---

From: Jamie Lokier <[email-masked]>
Date: Sun, 17 Apr 2011 00:41:26 +0100

> applications', it sounds like I am doing alright.

The point is arbitrary small thresholds aren't helpful when there
is no clear benefit or reason.  Sendfile is an example of why - it is
not the point itself.  Also I was wrong when I said no current
applications.  I thought of a couple of realistic ones since writing
that, which I won't go into here.

The case in favour of 64k frames seems weak to me.  It will seem
archaic in the coming years, like the 640k RAM limit in MSDOS and the
quaint 2G file size limit we used to have.

> packets already.  In that case it's neutral about frame size limit.

Sure it can be done; that's not the point.

The point is a limit does have implications beyond counting the header
bytes proportional overhead.  And so a limit should be justified.
Nothing more than that, really.

> until other senders' frames are sent.

Just 8192?  Seems a bit short-sighted.  With some communication
patterns (generally pub-sub with very little activity but confirmed
presence needed), a million subchannels works efficiently, even on a
single small PC over a home network connection.  And with much less
memory than using lots of sockets.

It is more relevant when you think that good muxing should make
WebSocket objects in Javascript so cheap to create that they could be
created casually and remove the need for application scripts to
address, format and dispatch messages internally.  But I'm digressing.

> link, so he gets his turn 3 Millenia later... bit laggy).

That is nonsense: If you are a multiplexer, you choose sensible frame
sizes for the situation - and "sensible" is a rather dynamic quantity.
If you are not, you don't have that problem and any frame size is ok,
- maybe even 2^63, if you just want to get WS out of the way and
transmit an infinite byte stream.

Frame sizes don't stay the same across mux/demux boundaries.
Splitting & merging is permitted, which is the whole point of frames.

A fixed 64k or 10k frame size limit does not address latency
adequately.  If you have 8192 subchannels with latency requirements,
you need QoS reservations and/or bandwidth-latency estimation, and the
muxer needs to choose its frame size from that.

And ideal muxer would probably vary the frame size dynamically,
reducing as the amount of predicted unacknowledged data increases to
bound the expected latency of the next small message.  It has some
similarities with process scheduling, which dynamically adjusts
timeslices and learns to distinguish batch activity from transient
activity.

The point is that is very much something that should be left to
implementations.  We don't have enough experience to know what values
to use, let alone what adaptive heuristics to use.

> 2^32 that's blocking 10Tbit/s link for 3ms.

That'd be fine with me.

I argued for 2^24, but it was easy to miss :-)

I think the reason it came out as 2^63 in the end was so that there'd
be no untested boundary case or simply ignored wraparound to become a
submarine bug in widely deployed code when it encounters handles a long
frame in future.

-- Jamie

---

From: Andy Green <[email-masked]>
Date: Sun, 17 Apr 2011 08:49:09 +0100

On 04/17/2011 12:41 AM, Somebody in the thread at some point said:

Hi -

> Just 8192?  Seems a bit short-sighted.  With some communication

8192 in one mux layer.  n of those can be mux subchannels at the next 
layer up and so on.

> address, format and dispatch messages internally.  But I'm digressing.

Yeah I already pointed that out a while back, it's a good reason to 
expect that we see mux connections pretty much as the norm in the 
browser case.

> That is nonsense: If you are a multiplexer, you choose sensible frame

The example uses the current status of x-google-mux AIUI, it doesn't 
define chopping frames, just prepends them with mux block headers.

> timeslices and learns to distinguish batch activity from transient

The current approach AIUI is that frame fragmentation by intermediary of 
mux is possible, but only if the extensions are all understood by the 
guy who wants to fragment them.

When you combine the high incidence of mux connection with frame 
fragmentation by mux though, you can see arguments for monster ws frame 
size are all moot.

> I argued for 2^24, but it was easy to miss :-)

Well fine then.

> submarine bug in widely deployed code when it encounters handles a long

It'd be better to encourage or define a test suite.  I wrote a fragment 
fuzzer along with some simple protocols, Greg has tried to define a set 
of tests formally based on libwebsockets test protocols.  They not only 
confirm reliable operation inside the individual implementation but have 
a huge advantage they can confirm implementations had the same 
understanding of the spec by running one guy's client against another 
guy's server; Pat McManus went around testing interoperability with 
deflate-stream and found errors that were easy to fix once known.

Allowing crazy numbers in the protocol instead that we know can't be 
supported or fully tested and saying it's bulletproof sounds less good.

-Andy

---

From: John Tamplin <[email-masked]>
Date: Sun, 17 Apr 2011 11:17:10 -0400

On Sun, Apr 17, 2011 at 3:49 AM, Andy Green <[email-masked]> wrote:
> 8192 in one mux layer.  n of those can be mux subchannels at the next layer up and so on.

That is just the current proposal to get experience with it -- we
could always define some variable-length encoding (in fact it was
suggested to do that with low channel numbers to conserve space) and
allow them to get arbitrarily large, though we would probably have to
support negotiation of a maximum channel number in that case.

> The example uses the current status of x-google-mux AIUI, it doesn't define chopping frames,
> just prepends them with mux block headers.

Actually the "Fairness" section talks about refragmenting as
necessary.  You will have to do that based on the available send quota
for a channel anyway.

> The current approach AIUI is that frame fragmentation by intermediary of mux is possible,
> but only if the extensions are all understood by the guy who wants to fragment them.

That is what is in the base WebSockets protocol -- an intermediary may
not change the frame at all if it does not understand all the
extensions which were negotiated.

---

From: Andy Green <[email-masked]>
Date: Sun, 17 Apr 2011 17:26:46 +0100

On 04/17/2011 04:17 PM, Somebody in the thread at some point said:

Hi -

> On Sun, Apr 17, 2011 at 3:49 AM, Andy Green<[email-masked]>  wrote:

I guess unless the quota is managed very nicely there is a magic number 
of active subchannels where the quota traffic itself and the latency 
characteristics of delivering the quota vs the payloads causes 
performance to melt down.

> necessary.  You will have to do that based on the available send quota

Well I missed it.  You could take the approach you just delay sending it 
until there's enough quota to send it all too.  Although as far as I 
read there's no relationship between quota numbers and max legal frame 
size defined.

-Andy

---

From: Greg Wilkins <[email-masked]>
Date: Mon, 18 Apr 2011 08:41:07 +1000

On 16 April 2011 01:46, Willy Tarreau <[email-masked]> wrote:
> any particular problem to browsers.

But it does cause problems on servers.

HTTP servers typically have undeclared limits on the size of URL
accepted (that limits parameters sent with a GET), and limits on the
size of form encoded content (that limits the size of parameters sent
with a POST).

We have response codes 413 and 414 to cope with these circumstances,
and maybe the equivalent is sufficient for websocket - but I hardly
see having the ability to declare a maximal frame and/or message size
as bad thing - at worse it might be under utilised.

cheers

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 18 Apr 2011 07:06:42 +0200

> size of form encoded content (that limits the size of parameters sent

Indeed, though that's slightly different.

> as bad thing - at worse it might be under utilised.

Having the ability to specify the *max* sizes is a good thing and I'd
also like to have it. What I was saying is that we should not have to
announce the message size, only the frame size.

Regards,
Willy

---

From: Bob Gezelter <[email-masked]>
Date: Mon, 18 Apr 2011 06:14:51 -0500

This will have to be short, as I have some deadlines to meet today.

I have noted a number of comments concerning my comments about frame sizes.
Some of the comments seemed to mis-construe my suggestion as a suggestion
of limits on message size. I do not like extremely long messages for other
reasons, but I see no reason to preclude them from the specification.

Long frame sizes, on the other hand, have implications on how the WebSocket
protocol rides on its transport mechanism, which is a relevant concern. In
particular, I disagree with the suggestion that frame sizing should be
driven by some of the concerns expressed, to wit: sendfile and hardware
assisted TCP.

sendfile is indeed described as a "... Because this copying is done within
the kernel. ..." However, I believe that this characterization is not
to be taken literally. A large file (arguendo, hundreds of megabytes), may
be at best mapped into virtual space, but each group of disk blocks will
have to be brought into memory from the (possibly non-continuous) areas of
the disk, and then written back out. This is not an overhead-free operation.
Instead, the sendfile description seems to reap significant benefit from
avoiding multiple in-memory copies that would otherwise be necessary.

Hardware assisted TCP is similarly not all of the limitation that it seems.
All devices providing hardware assist for TCP almost certainly implement
scatter/gather mapping of some form. Therefore, pre-pending protocol
information onto the front of WebSocket protocol frames does not prevent
TCP hardware assist. It would be more correct to say that the efficiency
of the WebSocket protocol in such situations will depend heavily on the
choices of those implementing the middleware layer that actually speaks
the protocol. If the middleware layer (e.g., WebSockets) is not well
optimized, then we will pay the price.

Multiplexing is a whole other topic for another posting.

- Bob

---

From: "Arman Djusupov" <[email-masked]>
Date: Tue, 19 Apr 2011 10:30:55 +0300

Hello Greg,

> We have response codes 413 and 414 to cope with these circumstances, and
> maybe the equivalent is sufficient for websocket - but I hardly see having
the
> ability to declare a maximal frame and/or message size as bad thing - at
> worse it might be under utilised.

I don't think 413 type of error codes can be used efficiently since many
simple client implementations do not read from the connection while sending
a POST request. As a result, even if the server tries to respond with 413,
the response is not getting read until the client has sent the complete
request message. So what happens is that the server either writes out the
error response and drops the connection, or writes out the error response
but keeps reading and discarding the remaining incoming message to recover
the HTTP connection. 

With best regards,
Arman