Subject: I-D Action: draft-ietf-httpbis-encryption-encoding-03.txt

From: [email-masked]
Date: Sun, 09 Oct 2016 22:01:22 -0700

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Hypertext Transfer Protocol of the IETF.

        Title           : Encrypted Content-Encoding for HTTP
        Author          : Martin Thomson
	Filename        : draft-ietf-httpbis-encryption-encoding-03.txt
	Pages           : 19
	Date            : 2016-10-09

Abstract:
   This memo introduces a content coding for HTTP that allows message
   payloads to be encrypted.

Note to Readers

   Discussion of this draft takes place on the HTTP working group
   mailing list ([email-masked]), which is archived at
   https://lists.w3.org/Archives/Public/ietf-http-wg/ .

   Working Group information can be found at http://httpwg.github.io/ ;
   source code and issues list for this draft can be found at
   https://github.com/httpwg/http-extensions/labels/encryption .

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-httpbis-encryption-encoding/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-httpbis-encryption-encoding-03

A diff from the previous version is available at:
https://www.ietf.org/rfcdiff?url2=draft-ietf-httpbis-encryption-encoding-03

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

---

From: Martin Thomson <[email-masked]>
Date: Mon, 10 Oct 2016 16:08:27 +1100

> https://tools.ietf.org/html/draft-ietf-httpbis-encryption-encoding-03

This version enacts the plan I described at the last meeting (and also
in email: https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/0368.html).

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 10 Oct 2016 17:03:08 +0000

--------
In message <[email-masked]>

>in email: https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/0368.html).

Two comments.

As far as I can see, you are now introducing a new HTTP header syntax,

I would propose that you collapse the two headers to one, and stay
inside the "common structure" I proposed[1], so we won't need another
parser.

For instance change:

   HTTP/1.1 200 OK
   Content-Type: text/html
   Content-Encoding: gzip, aesgcm
   Transfer-Encoding: chunked
   Encryption: keyid="[email-masked]";
               salt="m2hJ_NttRtFyUiMRPwfpHA"

to:

   HTTP/1.1 200 OK
   Content-Type: text/html
   Content-Encoding: gzip, aesgcm
   Transfer-Encoding: chunked
   Encryption: aesgcm;keyid="[email-masked]";
               salt="m2hJ_NttRtFyUiMRPwfpHA"

Poul-Henning

[1]
    https://datatracker.ietf.org/doc/draft-kamp-httpbis-structure

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Martin Thomson <[email-masked]>
Date: Tue, 11 Oct 2016 09:42:42 +1100

> As far as I can see, you are now introducing a new HTTP header syntax,

Not as far as I can see.  If we accept the premise that the common
structure is "list of (thing plus parameters)" then this fits that as
long as "thing" is permitted to be null.  This is consistent with
Forwarded-For and Cache-Control, for instance.  (I didn't make this
comment when reviewing your draft, because I don't think we're at that
point just yet.)

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 11 Oct 2016 16:42:15 +1100

Hi everyone,

As Martin indicated before, this is a fairly big change, so let's run a second WGLC on this document, ending 2016-10-25.

Please send feedback to this mailing list and/or the issues list.

Regards,

> On 10 Oct. 2016, at 4:01 pm, [email-masked] wrote:
> 
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 11 Oct 2016 16:50:08 +1100

[ Re-sending with WGLC in Subject, to make sure people see this. Apologies for the noise. ]

Hi everyone,

Regards,

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 11 Oct 2016 07:30:47 +0000

--------
In message <CABkgnnU3vOBBGm=[email-masked]>
, Martin Thomson writes:
>structure is "list of (thing plus parameters)" 

It isn't, its "list of named dictionaries" and your format lacks the name.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 13 Oct 2016 11:54:29 +0200

Hi there,

below are my first comments, mainly focusing on the non-crypto parts.

(I might try to implement at least the receiving part, in which case 
I'll have certainly more comments...)

Best regards, Julian

2.  The "aesgcm" HTTP Content Coding

    (...)

    The "aesgcm" content coding uses a fixed record size.  The resulting
    encoding is either a single record, or a series of fixed-size
    records.  The final record, or a lone record, MUST be shorter than
    the fixed record size.

That seems incorrect. If it's "either a single record or a series of 
fix-sized records", how then can then last record be shorter?

3.  The Encryption HTTP Header Field

    If the payload is encrypted more than once (as reflected by having
    multiple content codings that imply encryption), each application of
    the content coding is reflected in a separate Encryption header field
    value in the order in which they were applied.

I believe we need to clarify the precise interaction between new content 
codings and this header field. I'll assume any new content coding needs 
to opt-in to use this field value as well, so it's clear how to remove 
entries when unwrapping codings.

In particular: what if the parameters are optional? (and, nitpicking: is 
this mechanism really only applicable to encryption codings?)

    Encryption header field values with multiple instances of the same
    parameter name are invalid.

...of the same parameter name in a single encryption_params production...

(which reminds me that you can't have '_' in ABNF production names)

4.  Crypto-Key Header Field

    Crypto-Key header field values with multiple instances of the same
    parameter name are invalid.

(see above)

5.1.  Encryption of a Response

    Here, a successful HTTP GET response has been encrypted using input
    keying material that is identified by a URI.

by a URI? (leftover from earlier versions?)

    Note that the media type has been changed to "application/octet-
    stream" to avoid exposing information about the content.

Maybe mention the alternative of not sending it at all.

6.1.  Key and Nonce Reuse

    (...)

    If a content encryption key is reused - that is, if input keying
    material and salt are reused - this could expose the plaintext and
    the authentication key, nullifying the protection offered by
    encryption.  Thus, if the same input keying material is reused, then
    the salt parameter MUST be unique each time.  This ensures that the
    content encryption key is not reused.  An implementation SHOULD
    generate a random salt parameter for every message; a counter could
    achieve the same result.

So maybe the requirement is uniqueness, not randomness?

Editorial:

Abstract

Maybe this should mention that it also adds a mechanism to parametrize 
content codings in general?

3.1.  Encryption Header Field Parameters

   salt:  The "salt" parameter contains a base64url-encoded octets
       [RFC7515] that is used as salt in deriving a unique content
       encryption key (see Section 3.2).  ...

As "base64url" is defined upfront, citing here RFC7515 is a bit 
confusing (as when you follow the reference, you end up with a spec that 
happens to include that definition, but it largely about something else) 

3.2.  Content Encryption Key Derivation

    The info parameter to HKDF is set to the ASCII-encoded string
    "Content-Encoding: aesgcm", a single zero octet and an optional
    context string:

       cek_info = "Content-Encoding: aesgcm" || 0x00 || context

I was a bit confused by this until I realized that it's pseudo-code, not 
ABNF. Maybe "concat(....)" would be clearer than "||" (which, in the 
languages I use, definitively does not mean concatenation).

3.3.  Nonce Derivation

    The nonce input to AEAD_AES_128_GCM is constructed for each record.
    The nonce for each record is a 12 octet (96 bit) value is produced
    from the record sequence number and a value derived from the input
    keying material.

maybe ...value *that* is produced...

    The length (L) parameter is 12 octets.  The info parameter for the
    nonce is the ASCII-encoded string "Content-Encoding: nonce", a single
    zero octet and an context:

...*a* context...

References:

    [XMLENC]   Eastlake, D., Reagle, J., Imamura, T., Dillaway, B., and
               E. Simon, "XML Encryption Syntax and Processing", W3C
               REC , December 2002, <http://www.w3.org/TR/xmlenc-core/>.

Might be better to cite the slightly newer 
<https://www.w3.org/TR/2013/REC-xmlenc-core1-20130411/>. In any case, 
please add the W3C short name to the series element, such as in:

"Eastlake, D. and J. Reagle, “XML Encryption Syntax and Processing”, W3C 
Recommendation REC-xmlenc-core-20021210, December 2002, 
<http://www.w3.org/TR/2002/REC-xmlenc-core-20021210/>."

---

From: Kari Hurtta <[email-masked]>
Date: Sat, 15 Oct 2016 08:43:35 +0300 (EEST)

> It isn't, its "list of named dictionaries" and your format lacks the name.

Name can invented, I think. ☺

There is

2.  Definition of HTTP header Common Structure
https://tools.ietf.org/html/draft-kamp-httpbis-structure-00#section-2

|       common-structure = 1* ( identifier dictionary )
|
|       dictionary = * ( identifier value )
|
|       value = identifier /
|               number /
|               ascii_string /
|               unicode_string /
|               blob /
|               timestamp /
|               common-structure
|
|       identifier = (token / "*") [ token / "*" ]

3.  HTTP/1 serialization of HTTP header Common Structure
https://tools.ietf.org/html/draft-kamp-httpbis-structure-00#section-3

|       h1_common_structure = h1_element  * ("," h1_element)
|
|       h1_element = identifier * (";" identifier ["=" h1_value])
|
|       h1_value = identifier /
|               number /
|               h1_ascii_string /
|               h1_unicode_string /
|               h1_blob /
|               h1_timestamp /
|               h1_common-structure

If I guess correctly Martin Thomson's format gives

|       common-structure = 1* ( (identifier dictionary) / anonymous-dictionary )
|
|       dictionary = * ( identifier value )
|
|       anonymous-dictionary = 1* ( identifier non-empty-value )
|
|       non-empty-value = <value; empty value not allowed>

and

|       h1_common_structure = h1_element  * ("," h1_element)
|
|       h1_element = ( identifier * (";" identifier ["=" h1_value]) ) /
|                    ( 1* ( identifier "=" h1_value) )

This looks serializable, but is it parsable?

/ Kari Hurtta

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sat, 15 Oct 2016 08:58:30 +0000

--------
In message <[email-masked]>, Kari Hurtta wri

>This looks serializable, but is it parsable?

First, I think Martins format is wrong, with or without Common
Structure, and the "CE-param:" header is a better and more general idea.

Second, I think what you propose is parsable, but Getty's 3rd rule
("The only thing worse than generalizing from one example is
generalizing from no examples at all.") and 1st rule ("Do not add
new functionality unless an implementor cannot complete a real
application without it.") tells me to not add it.

---

From: Martin Thomson <[email-masked]>
Date: Wed, 19 Oct 2016 15:13:37 +1100

Thanks for the review Julian.

Thanks for making your comments actionable.  And sorry it took so
long, illness and other work conspired against me.

PR: https://github.com/httpwg/http-extensions/pull/249

I've just responded with commit IDs below if I agreed with your
comment and thought that it needed no discussion.

> fix-sized records", how then can then last record be shorter?

How about:
 The "aesgcm" content coding uses a fixed record size.  The resulting
encoding is
 any number of fixed-size records - which could be zero records - followed by a
 single partial record.  The partial record MUST be shorter than the
fixed record
 size.
[ddf5f8e]

> codings and this header field. I'll assume any new content coding needs to
> opt-in to use this field value as well, so it's clear how to remove entries
> when unwrapping codings.

I think that I have some text that will help.  I think that the answer is:

Content codings that use the Encryption header field MUST always include a
value for the header field when the content coding has been applied.  If no
parameters are needed, then a dummy value is necessary to avoid confusion about
which set of parameters applies to which content coding.  This requirement
applies to uses of the `aesgcm` content coding, which does not need a dummy
value because the `salt` parameter is mandatory.
[97b3c12] and [67b65df]

> ...of the same parameter name in a single encryption_params production...
[a0c51a2]

> (which reminds me that you can't have '_' in ABNF production names)
[1b376a8]

>    parameter name are invalid.
[a0c51a2]

> by a URI? (leftover from earlier versions?)
[bd0eef2]

> Maybe mention the alternative of not sending it at all.
[4e510be]

> So maybe the requirement is uniqueness, not randomness?

Yes, and it says that just above: "the salt parameter MUST be unique
each time".  The "SHOULD" here is advice (good advice I believe).

> Maybe [the abstract] should mention that it also adds a mechanism to parametrize

I don't believe that it does, or should.  See earlier email about
crypto being special (though we might argue that point if you like).

> As "base64url" is defined upfront, citing here RFC7515 is a bit confusing
> (as when you follow the reference, you end up with a spec that happens to
> include that definition, but it largely about something else)
[63850db]

> languages I use, definitively does not mean concatenation).

Maybe I can just explain it.  || is more compact.
[80225a4]

> maybe ...value *that* is produced...
[7f3a35c]

> ...*a* context...
[31c4bc4]

> <https://www.w3.org/TR/2013/REC-xmlenc-core1-20130411/>. In any case, please
> add the W3C short name to the series element, such as in:
[8ea4739]

Interesting thing I learned from this: there are rules for "fixing"
non-ASCII names in xml2rfc.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 19 Oct 2016 08:11:16 +0200

Thanks for the changes, Martin!

On 2016-10-19 06:13, Martin Thomson wrote:
> ...
> ...

"applies" -> "does not apply"?

In any case: this sounds like a band-aid. I think it would be good to 
discuss the whole parametrization of content codings...

> ...
> ...

But they are unreliable...

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 19 Oct 2016 06:27:01 +0000

--------
In message <[email-masked]>, Julian Reschke writes

>discuss the whole parametrization of content codings...

I have been pondering the "only encryptions will need parameters"
comment somebody made some days ago, and I have a hard time finding
out why that should be true.

Why is it that encryptions cannot prefix their necessary parameters
in the wame way compressions do (see gzip header) ?

Or to be concrete:  Why wouldn't this work:

   HTTP/1.1 200 OK
   Content-Type: text/html

   {magic marker}
   keyid="[email-masked]";
   {magic terminator}
   [encrypted payload]

---

From: Julian Reschke <[email-masked]>
Date: Wed, 19 Oct 2016 08:43:02 +0200

>    salt="m2hJ_NttRtFyUiMRPwfpHA"

Because you might want to ship the parameters somewhere else. See 
example in 
<https://greenbytes.de/tech/webdav/draft-reschke-http-oob-encoding-08.html#rfc.section.3.5.3>.

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 19 Oct 2016 14:13:28 +0000

--------
In message <[email-masked]>, Julian Reschke writes

><https://greenbytes.de/tech/webdav/draft-reschke-http-oob-encoding-08.html#rfc.section.3.5.3>.

Yeah, I thought about that, but the more I study it, the more I don't
see why HTTP needs to get involved in either activity.

All this stuff can be done with existing HTTP mechanisms, by defining
a new C-E which carries its own metadata in the body, like all other
C-E's, and the enourmous advantage of that is that it is backwards
compatible.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 19 Oct 2016 16:46:16 +0200

> C-E's, and the enourmous advantage of that is that it is backwards

But how would you handle the case describes above -- where the metadata 
(content type, encryption material) is served from a server different 
from the one having the (encrypted) payload?

Best regards, Julian

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 19 Oct 2016 15:33:15 +0000

--------
In message <[email-masked]>, Julian Reschke writes

>from the one having the (encrypted) payload?

I would put it into the inline metadata of the new C-E I defined.

---

From: Kari Hurtta <[email-masked]>
Date: Wed, 19 Oct 2016 19:01:19 +0300 (EEST)

> from the one having the (encrypted) payload?

https://lists.w3.org/Archives/Public/ietf-http-wg/2016OctDec/0196.html

Is this just about moving Encryption header to body?

   HTTP/1.1 200 OK
   Content-Type: text/html

   {magic marker}
   keyid="[email-masked]";

On https://greenbytes.de/tech/webdav/draft-reschke-http-oob-encoding-08.html#rfc.section.3.5.3
there is 

   Crypto-Key: keyid="a1"; aesgcm="csPJEXBYA5U-Tal9EdJi-w"

which you do not want move to body. That is different thing.

And content-type is just here, it is not moved to
server to where out-of-band points:

TTP/1.1 200 OK
Date: Thu, 14 May 2015 18:52:00 GMT
Content-Encoding: aesgcm, out-of-band
Content-Type: text/plain
Encryption: keyid="a1"; salt="vr0o6Uq3w_KDWeatc27mUg"
Crypto-Key: keyid="a1"; aesgcm="csPJEXBYA5U-Tal9EdJi-w"
Content-Length: 101
Vary: Accept-Encoding

{
  "sr": [
    { "r" :
      "http://example.net/bae27c36-fa6a-11e4-ae5d-00059a3c7a00"}
  ]
}

/ Kari Hurtta

---

From: Martin Thomson <[email-masked]>
Date: Thu, 20 Oct 2016 13:14:19 +1100

On 20 October 2016 at 01:46, Julian Reschke <[email-masked]> wrote:
> (content type, encryption material) is served from a server different from
> the one having the (encrypted) payload?

You know, I had the same thought as PHK after making that statement
and I can't think of a reason he is wrong. What was relevant is that
you need to *know* more stuff to get crypto right, but that's just the
key.  You don't need to parameterize the content coding otherwise.

If rs, salt and keyid were in the payload, I can't see how that would
be a real problem.  They are public information and inline avoids a
whole mess of issues.  Every secondary server would be serving the
values, but that's not fatal.  You wouldn't be able to "compress" them
by including one value across all potential secondaries (again, not a
real problem, and potentially a feature if you wanted different
encryptions across secondaries to avoid correlation).

The best I could come up with is that random access always requires
the first few octets.  But that's weak: it's either metadata or some
of the payload.  And we already take on that burden in other places:
it's actually a common restriction on resources that are acquired
piecemeal.  Some media container formats require the end to make sense
of the middle, which is much more tiresome.  Zip archives are also
like that.

Now, I don't know what to do about webpush, but if we are going to
make breaking changes, I can maybe get some efficiency gains from them
which should help there.  A new name should help avoid the worst parts
of the churn.

---

From: Julian Reschke <[email-masked]>
Date: Thu, 20 Oct 2016 08:16:42 +0200

> of the middle, which is much more tiresome.  Zip archives are also

We may want to try to make the parameter segment fixed-size...

> which should help there.  A new name should help avoid the worst parts

Could you clarify this? Is this about a technical issue, or about when 
webpush can be finalized?

Best regards, Julian

---

From: Martin Thomson <[email-masked]>
Date: Fri, 21 Oct 2016 07:57:55 +1100

> webpush can be finalized?

We have deployed code, lots of it, that sort of rely on the current
structure.  We would need a transition strategy.