Subject: [hybi] Masking only Payload/Extension Data

From: Brian <[email-masked]>
Date: Tue, 8 Mar 2011 23:29:01 -0800

In hope of getting consensus on the idea that only the payload and
extension data should be masked and not the framing itself, I took a
pass at adjusting sections 4.1 and 4.2 accordingly.  It didn't take
much, just a few minor tweaks.

What do you think?  Any chance we could reach a rough consensus on
masking only the extension/payload data?

4.1.  Overview

   In the WebSocket protocol, data is transmitted using a sequence of
   frames.  The payload portion of frames sent from the client to the
   server is always masked to avoid confusing network intermediaries,
   such as intercepting proxies.  The payload portion of frames sent
   from the server to the client are not masked.

   The base framing protocol defines a frame type with an opcode, a
   payload length, and designated locations for extension and
   application data, which together define the _payload_ data.  Certain
   bits and opcodes are reserved for future expansion of the protocol.
   As such, In the absence of extensions negotiated during the opening
   handshake (Section 5), all reserved bits MUST be 0 and reserved
   opcode values MUST NOT be used.

   A data frame MAY be transmitted by either the client or the server at
   any time after handshake completion and before that endpoint has sent
   a close message (Section 4.5.1).

4.2.  Client-to-Server Masking

   The client MUST mask the payload of all frames sent to the server,
   including all extension data.

   The masking-key is contained completely within the payload.

   The masking-key is a 32-bit value chosen at random by the client.
   The masking-key MUST be derived from a strong source of entropy, and
   the masking-key for a given frame MUST NOT make it simple for a
   server to predict the masking-key for the payload in a subsequent
   frame.

   Each masked payload consists of a 32-bit masking-key followed by
   masked-data:

     masked-payload  = masking-key masked-data
     masking-key     = 4full-octet
     masked-data     = *full-octet
     full-octet      = %x00-FF

   The masked-data is the clear-text payload "encrypted" using a simple
   XOR cipher as follows.

   Octet i of the masked-data is the XOR of octet i of the clear text
   frame with octet i modulo 4 of the masking-key:
     j              = i MOD 4
     masked-octet-i = clear-text-octet-i XOR octet-j-of-masking-key

   When preparing a masked-payload, the client MUST pick a fresh masking-
   key uniformly at random from the set of allowed 32-bit values.  The
   unpredictability of the masking-nonce is essential to prevent the
   author of malicious application data from selecting the bytes that
   appear on the wire.

Brian McKelvey

---

From: Andy Green <[email-masked]>
Date: Wed, 09 Mar 2011 07:47:10 +0000

> masking only the extension/payload data?

+1 from me.

It also solves Toni Ruottu's slightly exotic case where he might want to 
mask his server frames, since he can do that as a legit extension 
negotiated at handshake-time, stick his mask as first extension data, 
then and the framing and content structure is identical to masked client 
frames.

I also remind folks that to set the clear length field in Javascript, 
you have to get the browser to accept to create an object of the 
requisite length.  That extends past 1GB for 4 bytes of ASCII and 256GB 
for 5 bytes of ASCII starting with a letter.  Since the next thing is 
defined to be the mask in extension data, 3 or 4 bytes of control is all 
you can realistically hope for.  If you can knock over an intermediary 
with 4 bytes, then it will fall over anyway once every 4G frames that go 
through it and beyond caring about.

-Andy

---

From: [email-masked]
Date: Wed, 9 Mar 2011 00:16:43 -0800

> masking only the extension/payload data?

+1 from me, if it helps.

Also, I agree all of pros/cons decribed in Wiki page someone 
summarized(thanks!):
http://trac.tools.ietf.org/wg/hybi/trac/wiki/MaskingProposals 

- Yutaka

> _______________________________________________
> hybi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hybi

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Wed, 9 Mar 2011 00:27:36 -0800

Can someone please explain to me why this matters? Given that we went with a
32-bit mask included in the frame, and it's just xor, I'm really not
sympathetic to arguments about "it's hard to unmask" etc. The only thing
that I've heard that resonates at all is that you have to know which
direction the connection is going, e.g. which is the client and which is the
server, that said it seems like any proxy doing anything interesting would
probably already know this?

-Ian

---

From: Andy Green <[email-masked]>
Date: Wed, 09 Mar 2011 08:45:24 +0000

On 03/09/2011 08:27 AM, Somebody in the thread at some point said:

Hi -

> Can someone please explain to me why this matters? Given that we went
> with a 32-bit mask included in the frame, and it's just xor, I'm really

It's not driven by complaints about the masking complexity.

> not sympathetic to arguments about "it's hard to unmask" etc. The only
> thing that I've heard that resonates at all is that you have to know
> which direction the connection is going, e.g. which is the client and
> which is the server, that said it seems like any proxy doing anything
> interesting would probably already know this?

Because it's a streamed protocol, one problem it creates is a snooper 
trying to perform frame sync on stuff coming out of a client can find a 
pathological situation as it is, it's literally a randomized stream with 
no structure at all that may not have any framing relationship to packet 
boundaries.  It's four random bytes then everything xor'd with random 
without let-up, there's nothing to grab hold of once it has started.

If the framing is left standing proud of the mask action, there is at 
least always framing structure in the stream to sync on.

Also although I am grateful for the choice you made of simply 4-byte 
mask compared to AES and other castles in the air, actually sticking a 
fixed 4-bytes on the front in one direction is kind of a hack protocol-wise.

If the framing is definitely the boss in this protocol, and not 
subordinate to the masking, then it will be possible to negotiate 
different masking schemes at handshake-time if that becomes desirable 
without changing intermediaries that don't care about payload but must 
watch the streams for control packets or whatever: the intermediaries 
can just obey framing rules even with no masking negotiated or ROT13 or 
whatever negotiated that they don't understand or care about, but now 
won't break on.

As mentioned one specific case of wanting to do the above is Toni 
Ruottu's where the browser emits server frames.  He can solve his 
quandry with just an extension then, changing masking rules without 
upsetting the framing rules and so allowing things that don't understand 
to not break.

-Andy

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Mar 2011 22:03:37 +1100

I'm +1 on the proposal.

2011/3/9 Ian Fette (イアンフェッティ) <[email-masked]>:
> Can someone please explain to me why this matters?

modified from the earlier thread on this:

Masked Frames PROS:
 + all bytes other than the key are masked.  This prevents an attacker
from using frame fields like the length to send "clear text"

Masked Frame CONS:
 - Masking is not independent of framing - poor layered protocol design.
ie this is the same reason that gzip in HTTP does not change the framing
of HTTP messages.
 - all WS handling that needs to be aware of frame boundaries will
need to implement masking.  This will mean that more infrastructure
will have masking hard coded into it, thus it will soon become very
difficult, if not impossible to change framing as it will be baked
into WS aware intermediaries.
 - Intermediaries that do not care about content will still need to
unmask and remask the frames.
 - the parser/generator code for WS becomes more complex and
asymmetric because it must implement masking for some WS streams but
not for others.
 - unmasking cannot be done as single optimised pass over the data,
but must be done on-the-fly in the parsing code.
 - the options for masking algorithms is limited.  For example session
based keys could not be used in a MUX environment as the session key
would need to be known to access the extension data to know the stream
ID, so the session ID could be known so you can't know the session
key.   While session based masking may not be desirable for other
reasons, this limitation indicates a fundamental limitation of the
masked frame approach.

Mask in Frame PROS:
 + Masking is independent of framing.   Good layered protocol design.
 + Intermediaries that do not process the payload will not need to
implement masking. This saves CPU, memory and also will make it easier
to change and/or remove masking in future.
 + WS parsing code is simple and symmetric.
 + session based masking algorithms can be used if desired
 + Can utilise the extension data space to carry the key - thus
testing the implementation of the parsing of extension data length and
preventing implementations assuming this will be zero.

Mask in Frame CONS:
 - The frame op-code, flags and length are sent in the clear.  Of
these, an attacker has reasonable control over the length field, so
could conceivably send 8 characters including "GET\n" as the length
(although this would be very difficult to do via the current
javascript API.  There are also simple heuristic defences for this,
such as fragmenting any frame that has a length that is all ascii and
has a CR)

In summary,   mask in frame is far more flexible, low cost and
demonstrable better design, at cost of exposing a very very unlikely
subcase of an already very very unlikely vulnerability to an as of yet
undiscovered threat, of which an exploitable variation is already in
the wild using non ws clients.

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Wed, 9 Mar 2011 03:33:59 -0800

Ok, I really don't care so I will just go along here. That said, it doesn't
seem like the above will let you figure out if you're looking at a masked
frame or an unmasked frame. Does that matter?

-Ian

2011/3/9 Greg Wilkins <[email-masked]>

---

From: Greg Wilkins <[email-masked]>
Date: Wed, 9 Mar 2011 22:49:21 +1100

> frame or an unmasked frame. Does that matter?

Not to me.   Even though masking IS NOT an extension, I think of it
just like one.  You need to have seen the handshake or some other
context of the frame to know if the extension applies or not.

cheers

---

From: Bruce Atherton <[email-masked]>
Date: Wed, 09 Mar 2011 09:27:33 -0800

On 09/03/2011 3:33 AM, Ian Fette (イアンフェッティ) wrote:
> Ok, I really don't care so I will just go along here. That said, it 
> doesn't seem like the above will let you figure out if you're looking 
> at a masked frame or an unmasked frame. Does that matter?
>
>

It does to me and (I think) Andy, but we were the lone voices willing to 
use a reserved bit for this. Without more support I am ready to admit 
defeat on this point.

---

From: Andy Green <[email-masked]>
Date: Wed, 09 Mar 2011 17:59:22 +0000

> It does to me and (I think) Andy, but we were the lone voices willing to

Yes you're right I also think it'd be good to burn a bit for masked. 
We're already sending those reserved bits.

Actually, I know it's a bit off-topic and not for -07, but when I was 
doing extensions stuff for libwebsocket I was curious and having to make 
guesses about the kind of approach mux extension will take.  I don't 
want to argue about it now, it'd just be interesting to take the 
temperature of people (or person, John) who thought ahead on it already.

Eg, you get a byte in extended area telling the channel and coding 
channel bringup and shutdown events?

> use a reserved bit for this. Without more support I am ready to admit

Right I heard people carefully reserving the extended bits but if we 
reserve them all at 0 forever in all contexts we will be wasting them 
alright.

-Andy

---

From: [email-masked]
Date: Wed, 9 Mar 2011 10:41:25 -0800

> use a reserved bit for this. Without more support I am ready to admit 

Let me join in the voices for +1 on the use of reserved bit.

I agree with your points of debate in the following messages:
 - http://www.ietf.org/mail-archive/web/hybi/current/msg06631.html
 - http://www.ietf.org/mail-archive/web/hybi/current/msg06633.html

Reasons:

 o Sniffers can choose whether to go ahead and parse masked payload 
   based on the bit without having to capture from the start.
   (It greatly helps us in debugging or trouble shooting)
 o Ability to turn off masking for debugging/trouble shooting.
   (this should not be manipulable by user code, of course, just to be 
clear)
 o It is worthier allocating a bit for this than reserving it
   for something we don't even know especially when we still have 
   at least a few bits left.
 o Lacking the bit seems to be the source of sense of asymmetry -
   as in the frame not being self-descriptive for what's in the 
   payload (not to be confused by 'understanding' the payload),
   seems to be a bad protocol design choice.

- Yutaka

---

From: John Tamplin <[email-masked]>
Date: Wed, 9 Mar 2011 14:04:41 -0500

>    (It greatly helps us in debugging or trouble shooting)

I agree this is useful, but I disagree that this is sufficient to warrant
the cost.

>    (this should not be manipulable by user code, of course, just to be

This is orthogonal to whether to use a bit in the frame, and from previous
discussions there is little support for being able to disable it.

>    at least a few bits left.

The problem is there are only a few reserved bits, and we don't know how
many it would be useful to have.  One thing that was discussed earlier that
would require allocation of a reserved bit would be per-frame compression,
so you can avoid compressing non-compressible data.  Another possibility is
that we generally allocate more opcodes for extensions, in which case we
would like to be able to allocate some of the reserved bits to the opcode
field.  We won't know we used too many until we need more than we have, at
which point it becomes expensive (50% additional overhead for small frames)
to get more.  When we came up with this framing, this was the compromise
between cost and leaving options open for the future that everybody could
live with.

Think about it from an efficiency point of view -- you are suggesting
allocating a bit in every frame that is going to be the same for every frame
in the same direction.  How can that possibly be good stewardship of limited
resources, rather than specifying it once in the header (or not at all if it
is known to be set or clear based on which side is initiating the
connection)?

>    seems to be a bad protocol design choice.

The frame is already not self-descriptive -- if compression is negotiated,
you have to know that (plus the current compression state) in order to
interpret the frames.  Likewise, other extensions will likely modify
interpretation of the payload.

I still think it is not worth allocating a reserved bit to indicate masking
is present, but if most people want it I won't object too strenuously.

-- 
John A. Tamplin
Software Engineer (GWT), Google

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 10 Mar 2011 07:33:41 +1100

I'm +1 for using a reserved bit to indicate masking, but not if that
clouds any consensus for in frame masking.

Besides, the way the protocol is developing, we are allocated data
space for control codes, for masks etc. then most extensions will be
free to put bits there.

For me, the only reason to use reserve bits instead of payload bits,
is that it is much much simpler for the framing layer to interpret
those bits.  ie you couldn't have a masking bit inside a masked
payload.

So I think masking is a perfect exemplar for the use of a reserve bit.
 If it can't get over the line, then I doubt any usage can.

cheers

---

From: John Tamplin <[email-masked]>
Date: Wed, 9 Mar 2011 15:40:06 -0500

>  If it can't get over the line, then I doubt any usage can.

Assuming the browser vendors stick to the requirement that client-side
masking is mandatory, then every single MASKED bit will be set on the
client->server side, and either all will be set or all will be cleared on
the server side of the connection.  The receiver knows without a doubt the
state of masking for the next frame it will receive (and should drop the
connection if that expectation isn't met).  That doesn't seem like a useful
candidate to have something that doesn't change in every single frame, when
the sole benefit of it is for debugging.

Compare that to allocating a bit to show a frame is compressed -- we
certainly know that some messages will be compressible (text), while others
won't (images, audio, pre-compressed data, etc).  The receiver has no way of
knowing ahead of time which sort of message will next be sent, so it makes
sense to allocate a bit to say whether this frame was compressed or not.

---

From: [email-masked]
Date: Wed, 9 Mar 2011 12:41:21 -0800

[email-masked] wrote on 03/09/2011 11:04:41 AM:

> I agree this is useful, but I disagree that this is sufficient to 
> warrant the cost.
>  
clear)
> 
> This is orthogonal to whether to use a bit in the frame, and from 
> previous discussions there is little support for being able to disable 
it.

Right, I meant, to sniffers, the bit will tell whether the payload is 
masked
in the midst of trasferring data while turning on/off on the fly for
debugging purpose.

> The problem is there are only a few reserved bits, and we don't know
> how many it would be useful to have.  One thing that was discussed 
> earlier that would require allocation of a reserved bit would be 
> per-frame compression, so you can avoid compressing non-compressible
> data.  Another possibility is that we generally allocate more 
> opcodes for extensions, in which case we would like to be able to 
> allocate some of the reserved bits to the opcode field.  We won't 
> know we used too many until we need more than we have, at which 
> point it becomes expensive (50% additional overhead for small 
> frames) to get more.  When we came up with this framing, this was 
> the compromise between cost and leaving options open for the future 
> that everybody could live with.

Obviously you have much clearer picture of possible usage of the reserved
bits than I. I should respect your view/concern on this.

>  
> Think about it from an efficiency point of view -- you are 
> suggesting allocating a bit in every frame that is going to be the 
> same for every frame in the same direction.  How can that possibly 
> be good stewardship of limited resources, rather than specifying it 
> once in the header (or not at all if it is known to be set or clear 
> based on which side is initiating the connection)?

We are already sending these reserved bits on the wire for nothing as
of today, which is where I am coming from. (I understand, when we
need to extend reserved bits field in the future, that's when the use 
of bit may be perceived as 'extra-bandwidth'.)

> The frame is already not self-descriptive -- if compression is 
> negotiated, you have to know that (plus the current compression 
> state) in order to interpret the frames.  Likewise, other extensions
> will likely modify interpretation of the payload.

The compression is an extension and is optional, where making is a MUST
in normal operations. We can always turn off compression, if needed during 

trouble shooting, then I am hoping to be able to dissect packets on
the wire with unmasked data on the screen. Since WebSocket is long
lived, it could be difficult to capture entire data with sniffer
with finite amount of memory/disk space, and I am aware Wireshark really 
slows down when it captures large files for a long time. If we could 
selectively capture/dissect data, that would be very very helpful.

> I still think it is not worth allocating a reserved bit to indicate 
> masking is present, but if most people want it I won't object too 
strenuously.

I have been spending large portion of my time for debugging/trouble 
shooting, I may be too concerned about debuggability. I understand your
points of arguments, though let me keep my +1 on the use of reserved
bit for now. Thanks a lot for your comments.

- Yutaka

---

From: "Pat McManus @Mozilla" <[email-masked]>
Date: Wed, 09 Mar 2011 16:08:59 -0500

> So I think masking is a perfect exemplar for the use of a reserve bit.

It is completely redundant information for any websocket implementation.

as for wireshark - it gives one more input into a pile of heuristics
that it already has to use given the lack of a binding between
websockets and tcp frames. Frankly, I don't see what is so hard about
guessing whether or not masking is in play without the bit - especially
with the asymmetric framing. 

And no matter what you do, anything that has thrown away the TCP state
will be flat out guessing anyhow. The trivial case is a websocket
payload which is also valid websocket frame with the payload arranged at
the start of the TCP packet payload.

it's just one wasted bit. I don't really care that much. But I think -06
has it this all defined right.

I think a much more persuasive argument for including a direction bit is
binding some form of non reliable websockets over UDP or DCCP in the
future. 

-- 
http://www.getfirefox.com/

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 9 Mar 2011 22:42:12 +0100

> the start of the TCP packet payload.

As someone playing all the day with network captures, I totally agree with
you on this point. There is no relation between individually captured
network packets and streamed contents if you don't have them from the
start. Some framed protocols such as HTTP which have massive amounts
of headers have their framings more easy to spot, but we managed to
make something very cheap network-wise with WS and I suspect that the
framing will be indiscernable from binary contents.

However, I agree with most of Greg's pros/cons, especially with the
one about the protocol design. Having a protocol whose upper layer
changes the lower layer framing is really hard to extend, and having
two versions of each decoding function in intermediaries is needlessly
expensive to maintain. I'd be much more happy with a protocol where
something in the framing or handshake indicates that the payload is
masked and where the frames are transported in native form.

I also like the analogy with HTTP BTW. It is possible (and common) to
chunk-encode the transfer of gzipped-encoded contents. When doing so,
the HTTP headers are clear-text, the transfer encoding is clear, and
the gzipped contents are inserted in chunks. Fortunately we don't
gzip the chunk size nor the header, nor do we chunk-encode the headers,
otherwise it would become a terrible mess.

So +1 from me for mask in the payload with an indication either in the
handshake or using a reserved bit, but for considerations of protocol
design and longterm extensivity, not just for basic asumptions about
what a sniffer could decode in randomly captured packets.

Regards,
Willy

---

From: Joel Martin <[email-masked]>
Date: Thu, 10 Mar 2011 09:28:44 +0300

+1 for masking only payload/extension data.

No opinion on whether to use an opcode bit or not for this.

Joel Martin (kanaka)

---

From: Adam Barth <[email-masked]>
Date: Thu, 10 Mar 2011 00:39:08 -0800

-1 for masking only payload/extension data.  It's less secure, and
that's not acceptable.

Adam

---

From: Andy Green <[email-masked]>
Date: Thu, 10 Mar 2011 08:48:42 +0000

> -1 for masking only payload/extension data.  It's less secure, and

Since this is the second time you asserted it's "not wise" and "less 
secure" and now "not acceptable" with no explanation, do you think you 
could take us through your reasoning?

-Andy

---

From: Adam Barth <[email-masked]>
Date: Thu, 10 Mar 2011 01:08:12 -0800

On Thu, Mar 10, 2011 at 12:48 AM, Andy Green <[email-masked]> wrote:
> Since this is the second time you asserted it's "not wise" and "less secure"
> and now "not acceptable" with no explanation, do you think you could take us
> through your reasoning?

I was tired of repeating my explanation the last time we had this thread:

http://www.ietf.org/mail-archive/web/hybi/current/msg06630.html

As far as I can tell, no one has provided any additional information.
We can have these threads until the cows come home, but if no one is
adding any information, they're a waste of time.

Adam

---

From: Andy Green <[email-masked]>
Date: Thu, 10 Mar 2011 09:32:41 +0000

On 03/10/2011 09:08 AM, Somebody in the thread at some point said:
> On Thu, Mar 10, 2011 at 12:48 AM, Andy Green<[email-masked]>  wrote:

You have linked to the email where you simply tell us masking inside the 
framing is "not wise", you didn't participate in the rest of the thread, 
or add to or challenge Greg's list of pluses and minuses.

> adding any information, they're a waste of time.

You have not added any information to this thread either.

I think if you're going to set off bombs like "not wise", "less secure" 
and "not acceptable" you need to at least qualify them with what exactly 
it's less secure against and why, who can't accept it under what 
criteria.  Maybe what you think it is less secure against doesn't matter 
when it is analysed.  Things that you or I individually "can't accept" 
still may be good for the protocol.  Otherwise, it sounds pretty 
indistinguishable from handwaving.

How it feels to me is your one-sentence sniping is a threat to reignite 
a security taint on the protocol if you don't get listened to.  I would 
like to improve my opinion of what you're trying to achieve by 
interjecting, but when asked for an explanation so I can try to 
understand if the issue is real or not you are "too tired" again.

-Andy

---

From: Brian <[email-masked]>
Date: Thu, 10 Mar 2011 02:25:28 -0800

By my count, we have six voices in favor so far, including myself:
Andy Green
Ytaka Takeda
Greg Wilkins
Willy Tarreau
Joel Martin
Brian McKelvey

One on record as not having a strong opinion one way or the other:
Ian Fette

And one opposed:
Adam Barth

Adam's curt and patronizing reply is once again unenlightening as to
any viable reason why we need to mask the frame header as well as the
payload, beyond just his gut feeling on the matter.  I'd like to
re-assert that since the only field under a potential hacker's control
is the length (and potentially rsv bits via undefined extensions in
the future) that the practical possibility of the frame header being
used for an attack that actually accomplishes anything is so remote
it's utterly laughable.  Adam, if you really want to block this
proposal, please respond with an actual analysis and a potential
exploit case that's compelling enough to reverse the momentum built
behind this proposal.

Brian

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 10 Mar 2011 11:39:14 +0100

On Thu, Mar 10, 2011 at 02:25:28AM -0800, Brian wrote:
> it's utterly laughable.

I'd like to add that it's important to remember that this controllable
length is not even at the beginning of the frame, and that there are
uncontrollable bytes before.

Regards,
Willy

---

From: Joel Martin <[email-masked]>
Date: Thu, 10 Mar 2011 14:05:52 +0300

I've been following this discussion pretty closely and I'm willing to change
my mind if Adam (or somebody else) can clearly lay out a threat model that
isn't easily addressed by minor tweaking. I really am interested in what
real threat may exist and not just being stubborn. Please enlighten me.

As somebody else has proposed, the allowable range of values in the length
field can always be reduced in the future by fragmenting to avoid _specific_
known vulnerable intermediaries (in the implementations without even
changing the protocol). I.e. if "GET\n" is a problem, then client
implementations can just never send that value.

I suspect most would agree that masking of everything is better than
stalling the protocol any longer, but if Adam is the only objector and the
browser folks are okay with masking only payload then perhaps we should move
forward with it.

Joel Martin (kanaka)

---

From: Adam Barth <[email-masked]>
Date: Thu, 10 Mar 2011 03:30:45 -0800

Thanks for having an open mind Joel.  Let's start with some basic assumptions:

1) Attacks only get better over time.  Over the (hopefully) many years
folks will be using this protocol, the attacks we know about today
will continue to exist, and security researchers will continue to
learn more techniques and improve their attacks.  That means we should
design protocols with a margin of safety rather than pushing up
against the razor edge of what we can reliably exploit today.
Admitted, this aspect of security is somewhat of an art, and at some
point we'll need to defer to the judgement of security experts as to
how much of a margin we need for safety.

2) If an attacker can choose a moderately long sequence of bytes of
the wire, we lose.  More specifically, we can construct attacks today
that abuse certain configurations of proxies using a moderate length
of chosen bytes.  Because of (1), we should expect the length of these
sequences and the amount of damage attacker can cause with them to
increase over time.  The story would be different if we had some solid
reasoning about lower bounds on length or upper bounds on damage, but
we don't.

For simplicity, let's consider the base framing protocol:

http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-06#section-4.3

Let's assume we're only going to mask the extension data and the
application data.  That leds approximately 10 bytes at the beginning
of the frame that's unmasked.  Are these bytes under the control of
the attacker?  In large part, they are.  For example, the attacker is
relatively free to select the payload length.  The attacker is also
likely to have a large amount of control over the opcode (recall that
the attacker can drive the protocol state machine into relatively
arbitrary states).  As for the various reserved bits, we don't know
very much about how these bits will be used, but if they're likely to
be at least partially under the influence of the attacker.

Worse, consider the case where the attacker sends a sequence of very
short (e.g., zero or one byte) messages.  Now the attacker can control
(or can strongly influence) an appreciable fraction of the bytes on
the wire.  Additionally, the attacker gets feedback (i.e., by seeing
the bytes that arrive at the server) and can dynamically adjust
subsequent messages based on this feedback.

So, does the above add up to an attack I can demonstrate today?  No.
Is there reasonable evidence that attackers will be able to stich
together the above weaknesses to construct an attack in the future?
Yes.  Given what we know today, my view is that not masking the entire
frame is an unacceptable level of risk.

Adam

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 10 Mar 2011 03:52:40 -0800

The attacker is relatively free to select the length of the message, but
that may or may not translate into the length of a given frame. For
instance, a browser may decide to fragment a large message. Looking at the
base framing, the attacker can also choose between a text and binary frame.
In practice, I would be surprised if the attacker had control over more than
seventeen bits coming from a browser. We (the browser at its discretion)
could also choose to e.g. fragment a message if it saw a length equal to
"GET\n" or "POST" or any number of such keywords, though I admit that it is
no sure fire defense against potential attacks.

At the end of the day, I think you are right in saying that it does open up
additional attack surface and potential. The question is how plausible do we
believe such an attack to be, what is the potential damage, and what do we
gain in exchange? The handshake in theory means that we're looking at
attacks against ill-formed proxies as the primary attack target/vector. We
can argue about whether these are likely to grow in number or shrink in
number, but at the end of the day it is a relatively small number of
endpoints that are already exploitable via other means as you demonstrated.
I am willing to live with masking, but at some point I do think we need to
look at it as a cost benefit analysis, and not be willing to afford infinite
cost to the "proxies may get exploited" line of reasoning. I don't mean to
dismiss the issue by any means, but merely to say that there is a limit to
how much we should accomodate these things.

-Ian (hopefully that's a sufficiently cryptic response to maintain my
previously stated neutral stance)

---

From: Ian Fette (イアンフェッティ) <[email-masked]>
Date: Thu, 10 Mar 2011 04:02:09 -0800

That said, looking at the wiki again, I am not really sure I understand the
benefits that intermediaries would get from this. Either way it looks like
they need to be able to pick out frame boundaries, and to do that reliably I
think they have to read frame header 1, see how long that frame is, and
basically skip that many bytes to get to frame 2. I don't understand how
they could do it without parsing the frame header for frame 1. The masking
doesn't make it that difficult to get to the frame header, you have to xor
on the order of 10 bytes. A lot of the complexity savings seem predicated on
the thought that we may choose a different, stream-based masking algorithm,
which I thought we already decided against because of some of those exact
issues?

Again, I'm not against it, but I don't really see enough benefit to say I'm
for it either.

-Ian

2011/3/10 Ian Fette (イアンフェッティ) <[email-masked]>

---

From: Andy Green <[email-masked]>
Date: Thu, 10 Mar 2011 12:12:50 +0000

On 03/10/2011 11:30 AM, Somebody in the thread at some point said:

Hi -

> how much of a margin we need for safety.

The only things we should be deferring to are correct explanations that 
hold water for the other people on the list, and verifiable claims that 
can be reproduced.

Frankly because someone says they are a "security expert" doesn't exempt 
them from explaining their reasoning just because they feel they are 
awesome, and their reasoning should make sense.

It's just that if they call themselves a 'security expert', we are 
justified in expecting a very high level of reasoning and correctness in 
what they are telling, not that we sit in a circle around them accepting 
pearls of wisdom without doubts.

> of chosen bytes.  Because of (1), we should expect the length of these

Can you define what these proxies are and put a number on 'moderate'? 
These are claims of specific successful attacks you are referring to now 
so you should be able to be very specific indeed about the length of 
controlled data that was needed.  Why are you using a word like 
"moderate" when you have an actual number?

> relatively free to select the payload length.  The attacker is also

Although there can be up to 8 bytes reserved for length, to control that 
from Javascript land you need to allocate an object of the length you 
wanted, and you don't know the fragmentation policy of the browser 
either.  So if you do allocate the 1GByte object needed to get a 4-byte 
ASCII string in the length, you don't know it won't go out as a bunch of 
smaller guys using CONTINUATION and even with the 2-byte length field 
you now don't control at all -- in the interests of latency reduction 
that is highly likely when you tried to arrange for a 1GB message.

> be at least partially under the influence of the attacker.

Well, this is FUD.  But, given the subject, we have to accept a little 
FUD is part of discussing 'unknown attacks'.  However -->

> subsequent messages based on this feedback.

At this point, we must remind ourselves that all of this is ONLY in the 
context of an established websocket connection between two fixed 
endpoints, one of which definitely speaks client websockets and the 
other of which definitely speaks server websockets.

So the only kind of attacks that are possible from the client is to hack 
the server websockets implementation, which hopefully the server author 
took care to try to avoid being possible, or to try to affect a broken 
intermediary via websockets.

Alongside the power of the future to magically make attacks more 
possible, in fact over time people will also harden and uplevel their 
intermediaries too, and they will become websocket-aware in a deeper 
way.  Of course, that'll happen quicker if someone identifies the 
claimed vulnerable ones.  However in spite you don't seem to have any 
doubts they exist, I never saw you claim to know which proxies are 
vulnerable, just that there is some indirect evidence that they may exist.

As Joel pointed out it's also open to us to uplevel the protocol.

> frame is an unacceptable level of risk.

I am still waiting to be able to write a tool that can test proxies for 
and demonstrate the attack you were writing about 'yesterday'.

-Andy

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Mar 2011 00:49:52 +1100

On 10 March 2011 22:05, Joel Martin <[email-masked]> wrote:
> stalling the protocol any longer,

I don't think a bad design better than no design.

The reasons that I believe the tiny increase in the attack surface is
an acceptable risk are:

 + there are no known exploits or attacks based on this.
 + controlling the length field is difficult
 + browsers can mitigate by fragmentation
 + the protocol can mitigate by sending hello frames (or ping/pongs)
after handshake

In summary, there are no known (or even imagined) attacks, and if
there were, we have mitigations that are possible that can be applied
by browsers, within the current specification.

cheers

---

From: "Pat McManus @Mozilla" <[email-masked]>
Date: Thu, 10 Mar 2011 10:04:58 -0500

On Thu, 2011-03-10 at 14:05 +0300, Joel Martin wrote:

> stalling the protocol any longer, but if Adam is the only objector and
> the browser folks are okay with masking only payload then perhaps we
> should move forward with it.

I object too. I think not masking the header gives up a minor security
benefit for an insubstantial benefit. None of the arguments about the
benefits of the change have convinced me they add up to very much.

http://www.ietf.org/mail-archive/web/hybi/current/msg06689.html
http://www.ietf.org/mail-archive/web/hybi/current/msg06705.html
http://www.ietf.org/mail-archive/web/hybi/current/msg06718.html

---

From: Andy Green <[email-masked]>
Date: Thu, 10 Mar 2011 15:36:29 +0000

On 03/10/2011 03:04 PM, Somebody in the thread at some point said:

Hi -

> benefit for an insubstantial benefit. None of the arguments about the

It is much cleaner protocol-wise to have consistent framing and mask 
inside the extension area and allows server traffic masking to be 
selected the same way.

Once that's done, if 4-byte recirculating xor with per-frame random key 
isn't enough, and we have been told it is not "enough" before if you 
recall, then it'll be easy to upgrade the protocol to use AES masking or 
whatever without breaking the framing.

> benefits of the change have convinced me they add up to very much.

To be fair these are mainly defending the idea of masking as a whole, by 
explaining that it is cheap -->

> http://www.ietf.org/mail-archive/web/hybi/current/msg06718.html

That's not really the issue, the issue is do we really impact these 
security concerns by letting what is often just two bytes but may extend 
to ten structured bytes with opcode + length in go out unmaksed.

What in fact does your Mozilla implementation do about breaking large 
messages into CONTINUATION?

As a sort of side note, since you have been supporting protecting these 
alleged vulnerable proxies, do you have any effort or interest in any 
effort to actively detect and identify them, maybe a fuzzing tool for 
content?  It's curious there's so much handwringing about these alleged 
vulnerable intermediaries but actually I didn't see anyone give a toss 
about finding them until now.

-Andy

---

From: John Tamplin <[email-masked]>
Date: Thu, 10 Mar 2011 11:22:47 -0500

On Thu, Mar 10, 2011 at 10:36 AM, Andy Green <[email-masked]> wrote:

> It is much cleaner protocol-wise to have consistent framing and mask inside
> the extension area and allows server traffic masking to be selected the same
> way.
>

While I agree it is awkward to not be able to have a totally separate layer
where the masking is removed (since you can't know the length of the masked
frames without looking inside the masking), I don't think anyone is going to
implement it that way.  Instead, what will happen is that a flag will be
passed in saying to expect a mask, and it is just as easy to read that mask
before the opcode/length as it is afterwards (whether doing it with blocking
reads or a state machine for processing the next available bytes).  So, I
don't really see how it makes much difference from an implementation point
of view.

> isn't enough, and we have been told it is not "enough" before if you recall,
> then it'll be easy to upgrade the protocol to use AES masking or whatever
> without breaking the framing.

I don't see a difference here either -- if in the future a more
sophisticated masking is required, an extension would be negotiated that
will change it.  Browsers could choose to drop connections if the server
didn't accept this extension after some transition period, and then the new
masking algorithm would be applied to what was going on the wire.  If you
wanted to support both, it is easy enough for that boolean flag to become an
enum specifying the type of masking to be applied.

Regarding the negative impact of masking outside the framing, remember it is
only transparent intermediaries we are worried about (which includes packet
sniffers) since they can only observe the handshake but can't participate in
it.  Thus, if an extension is negotiated they don't understand, they can't
do anything with the stream.  I don't see there is much that can be done
about it -- for example, imagine the deflate-stream extension in -06 was not
part of the standard but a later extension.  No intermediary would even be
able to find frame boundaries if they didn't understand the extension.

As for using wireshark/etc to debug it, a simple heuristic like looking at
the port should get 95% of it right, and the user can always force
interpretation as WebSocket-client or WebSocket-server if they don't capture
the handshake.  Other intermediaries that must operate independently (such
as sniffers enforcing policy) are going to have to see the handshake to do
their job, and they have to already distinguish WebSocket traffic from
others in order to do their job, and in the process they will know which
side they are looking at.

So, my position is that I am unconvinced of the benefits of moving the mask
after the opcode/length, but I also think the additional risk exposed by
doing so is minimal, so I don't feel to strongly about it.

---

From: Patrick McManus <[email-masked]>
Date: Thu, 10 Mar 2011 11:29:43 -0500

> It is much cleaner protocol-wise to have consistent framing and mask 

The point of my message this morning was to make my opinion clear as it
seemingly had been missed or misunderstood. You don't need to make the
same arguments again - I'm not trying to argue mine again - simply show
that they were argued.

> messages into CONTINUATION?

for a variety of reasons right now it does not generate fragments.
Though I am considering changing that based on operational experience.

---

From: Bruce Atherton <[email-masked]>
Date: Thu, 10 Mar 2011 10:18:39 -0800

Are we voting on this now? If so, add my +1 for the proposal not to mask 
the framing.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 10 Mar 2011 19:32:52 +0100

On 10.03.2011 19:18, Bruce Atherton wrote:
> ...

No, we don't.

(see <http://tools.ietf.org/html/rfc4677#section-5.2>)

Best regards, Julian

PS: that being said, I support the proposal not to mask the framing data.

---

From: David Endicott <[email-masked]>
Date: Thu, 10 Mar 2011 13:33:17 -0500

Count me in as +1 in favour of no masking.

My reasoning being:
1. An application layer using a Websocket API cannot affect the
headers, only payload.
2. The Websocket connection is an *established* TCP connection that
has already navigated any intermediaries and satisfied their
connection requirements.
3. If these intermediaries choose to examine the data being passed
through this established connection, then it is their problem if they
expose themselves.
4. This alleged intermediary vulnerability would be available via any
mal-formed stream, if the intermediary is doing content examination.
5. Masking does not prevent man-in-the-middle attacks as the masking
key is included with the frame.
6. Agents that open a bare TCP connection and emulate Websocket allow
the attacker to craft custom frames or handshakes.  They can of course
generate whatever end-result masking they need.

So, I conclude that masking does not protect against (a)
man-in-the-middle or (b) a malicious application layer, or (c) attack
application.    Further, since the initial HTTP Upgrade handshake
negotiates connection establishment via any intermediary (satisfies
proxies, etc.), we must feel free to transmit any data content without
worrying we would disturb any hops between endpoints.   That is
fundamental to the definition of transparent intermediary.

On Thu, Mar 10, 2011 at 1:18 PM, Bruce Atherton <[email-masked]> wrote:
> Are we voting on this now? If so, add my +1 for the proposal not to mask the
> framing.
>

---

From: John Tamplin <[email-masked]>
Date: Thu, 10 Mar 2011 13:52:41 -0500

> Count me in as +1 in favour of no masking.

We declared consensus long ago that we would require masking on
client->server.  Your points below are nothing that wasn't brought up before
that decision, so I don't see why we should revisit it.

This discussion is about whether to have the mask before or after the
opcode/length fields in the frame.

---

From: Bruce Atherton <[email-masked]>
Date: Thu, 10 Mar 2011 10:58:44 -0800

> (see <http://tools.ietf.org/html/rfc4677#section-5.2>)

I didn't mean a vote to determine which way to go on the proposal, I 
meant another straw poll to indicate whether there is rough consensus. 
If we are adding up numbers of people arguing one way or another, that 
feels like a straw poll to me.

It seems like a good time for one anyway. There seem to be three schools 
of thought: those who feel there are costs to masking the framing that 
should be avoided given the minimal benefits; those who feel there are 
no costs to masking the framing and so no reason to lose the minimal 
benefits; and those who believe that the benefits are not minimal and 
should trump any of the costs.

I don't see any of those positions changing any time soon, no matter how 
much further discussion there is. The only question now is whether rough 
consensus has been reached one way or another.

OTOH, perhaps the chairs have already determined that there is a rough 
consensus and a poll is not needed.

---

From: David Endicott <[email-masked]>
Date: Thu, 10 Mar 2011 14:08:40 -0500

Sorry, I misunderstood the context of the conversation.

It would seem to me that it must go after the length field, or else
how does the server know how much of the following stream to unmask.
It would need to unmask enough of the header to determine the entire
frame size and then continue.  That seems an unnecessary burden and
complicates server reception processing.

+1 for masking (as silly as it is) after the header.

---

From: John Tamplin <[email-masked]>
Date: Thu, 10 Mar 2011 14:13:24 -0500

> complicates server reception processing.

I'm not sure I understand your objection.  If you are writing a blocking
implementation, in the unmasked case you are going to read 2 bytes, and then
look at the second byte to decide how many more bytes are in the header.
 That lets you determine the length of the frame and read the rest of it.
 In the masked case, (as implemented in -06) you will read 6 bytes, then
start applying the mask to the bytes read as you process them.  In the
non-blocking case, you will have a state machine keeping track of where you
are, and as you receive a byte after you have processed the mask you unmask
it and process it.  I don't see that it adds much complexity.

---

From: David Endicott <[email-masked]>
Date: Thu, 10 Mar 2011 14:29:52 -0500

Fair point.  I agree it is not an undue burden given that we already
have to deal with variable sized length fields anyway.

Upon reflection, I change my vote to: don't care.    I cannot see an
obvious benefit or drawback to either implementation.

---

From: David Endicott <[email-masked]>
Date: Thu, 10 Mar 2011 14:36:43 -0500

Afterthought:  an unmasked header would allow Websocket aware
intermediaries to manipulate frames without unmasking them.
Websocket aware load balancers and distributed frameworks come to
mind.

---

From: John Tamplin <[email-masked]>
Date: Thu, 10 Mar 2011 14:49:29 -0500

> Websocket aware load balancers and distributed frameworks come to

Right, that has been the main objection (by my estimation anyway) to masking
the header.  However, unmasking 2-10 bytes doesn't seem like a large
burden.

---

From: David Endicott <[email-masked]>
Date: Thu, 10 Mar 2011 14:59:58 -0500

Certainly not for a computer.  (But I'm getting older and it's
becoming difficult to xor in my head these days.)

But any extra work is extra work, and if the agent is manipulating
frames without processing them then its likely performance is a
concern.  Not that I really think xor'ing a few bytes is going to cost
enough clock cycles to matter.

My vote is thus: ambivalent; with a logically weak prejudice for
unmasked headers.

---

From: Brian <[email-masked]>
Date: Thu, 10 Mar 2011 12:56:53 -0800

Just a quick updated tally of support/opposition:

By my count, we now have eight voices in favor so far, including myself:
Andy Green
Ytaka Takeda
Greg Wilkins
Willy Tarreau
Joel Martin
Brian McKelvey
Bruce Atherton
Julian Reschke

Three on record as not having a strong opinion one way or the other:
Ian Fette
John Endicott (Generally in favor, but not strongly so)
John Tamplin (Generally opposed, but hasn't mentioned a strong
opposition that I've seen)

And two opposed:
Adam Barth
Patrick McManus

If I've missed or misrepresented anyone, please let me know.

I agree with Bruce in that I feel that pretty much everything that can
be said on the topic at hand has been said.  Sal, what is the next
step for the WG toward a declaration of consensus one way or the
other?  Do we need a hum or formal straw poll, or will this thread
suffice as such?  Is further discussion warranted?

Brian

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Mar 2011 08:22:21 +1100

On 11 March 2011 03:22, John Tamplin <[email-masked]> wrote:
> implement it that way.

If masking is removed from the framing, Jetty will certainly be
implementing masking as a separate layer. It would be applied
internally as an extension.

> will change it.

But that will not be possible because intermediaries and tools will
have been written that must understand the masking, so they can
understand the framing.
We know that it is impossible to suddenly change all intermediaries or
even to negotiate different protocols through them.    Once you make
masking part of the framing, it will be there forever.

The only way to achieve flexibility with masking is to allow elements
that don't care about masking (ie most intermediaries) to not have to
implement masking.

> for example, imagine the deflate-stream extension in -06 was not
> part of the standard but a later extension.

For the same reasons,  I think deflate-stream is a bad idea.     If it
was a good idea, then somebody would have done it to HTTP a long time
ago to reduce the verbosity of HTTP headers.

cheers

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Mar 2011 08:26:54 +1100

I would like to note a procedural issue that has occurred many times
in this working group.

Things like masked framing have been added to the specification
without consensus (there was consensus on masking, but objections had
been raised regarding the masking before it was added).   We then have
to argue to have the feature removed and if consensus is not achieve
then that feature remains.    This in effect means that the status quo
is to add a non- consensus feature.

Surely this should have been done the other way around.  A broad
consensus should have been achieved in the WG before such a
significant change the the agreed framing was put into the draft.

regards

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Mar 2011 08:28:08 +1100

On 11 March 2011 08:26, Greg Wilkins <[email-masked]> wrote:
> been raised regarding the masking before it was added).

oops - I mean to say:

there was consensus on masking, but objections had been raised
regarding masking the framing before it was added.

cheers

---

From: John Tamplin <[email-masked]>
Date: Thu, 10 Mar 2011 16:33:50 -0500

> regarding masking the framing before it was added.

I don't think there was consensus either way, but if a new draft was going
to be written it had to say something and be something implementable -- it
couldn't just say "WebSockets will implement masking, but we won't describe
the details yet" as no implementations would be interoperable.

I don't think the fact that the way it is in the draft requires a higher bar
to change it than if it weren't in the draft.

---

From: Greg Wilkins <[email-masked]>
Date: Fri, 11 Mar 2011 08:56:53 +1100

> the details yet" as no implementations would be interoperable.

Indeed it couldn't and it had to say something concrete.   But I think
we needed to spend a bit more time discussion alternatives that would
meet the consensus before we put such a significant change into the
framing.  But I think there was some alternative fatigue at that
point, so it is somewhat understandable... just unfortunate (and a
repeated anti-pattern).

> to change it than if it weren't in the draft.

Unfortunately the bar is higher.  If we don't reach consensus to
remove masking from the framing now, then framing would have been
significantly changed without any consensus.     I think the chairs
need to consider that when making the call on this issue - as it is
not unanimous.

cheers

---

From: John Tamplin <[email-masked]>
Date: Thu, 10 Mar 2011 16:59:43 -0500

> need to consider that when making the call on this issue - as it is

If we don't reach consensus on how masking should be done, then we don't
have a final spec yet.  It doesn't matter which alternative that lacks
sufficient support is mentioned in the draft, it still isn't done at that
point.

---

From: Salvatore Loreto <[email-masked]>
Date: Fri, 11 Mar 2011 14:58:44 +0200

On 3/10/11 10:56 PM, Brian wrote:
>   Sal, what is the next

Hi there,

I think this is a meaningful thread,
that absolutely needs further discussions before a declaration of 
consensus one way or the other!

we have Monday March 14th as deadline for draft to be considered at 
IETF80 meeting in Prague ( http://www.ietf.org/meeting/80/index.html ).
Ian Fette (who btw is in Japan) is working on a 07 version of the draft, 
not sure (due to the situation in Japan) if he will be able to meet the 
deadline;
in the case he will, 07 won't include any changes related to the masking 
issue.

To be clear and avoid misunderstanding: 07 will be just one more step 
but not the final candidate version...
things can (will) still change on the issues we know do not have reached 
consensus (e.g. whether to mask the framing or not),
but changes can also come from possible feedback from implementation and 
operational experience.

as next steps I would suggest:

-it would be worth to discuss the issue during our 2hours f2f meeting in 
Prague.
if any of the people in favor (or opposing) of 'Masking only Payload' is 
planning to attend the meeting, I will be happy to give him a slot
to present/discuss the issue. (please contact me if you want the slot)
Note1: there is the possibility to attend the meeting remotely (via 
audio streaming and jabber; or we can setup some collaboration tool for it),
but to have a good discussion we need at least one champion being 
physically present in the meeting
Note2: remember http://tools.ietf.org/html/rfc4677#section-5.2

-keep discussing in the mailing list and if we won't reach a consensus 
Joe and I will decide what to do (e.g. if we need an hum/straw poll etc.)

cheers
/Sal

-- 
Salvatore Loreto
www.sloreto.com