Subject: Re: [Acme] Specify which JWS serialization is used

From: Richard Barnes <[email-masked]>
Date: Fri, 2 Mar 2018 09:50:21 -0500

Hey all,

I merged #395 last night (thanks, Logan!).  While I was reviewing that, I
noticed that we need to cover the case where the client sends an encoding
that the server doesn't understand.  So I've posted a follow-on that adds
an error code and requires its usage.  LMK if you have any objections,
otherwise I'll merge before submission on Monday.

https://github.com/ietf-wg-acme/acme/pull/398

Thanks,
--Richard

On Mon, Feb 12, 2018 at 2:37 PM, Logan Widick <[email-masked]>
wrote:

> I've created a new pull request (https://github.com/ietf-wg-
> acme/acme/pull/395) to partially address the lack of serialization format
> specification. This pull request requires use of the Content-Type HTTP
> header to indicate the serialization format of the outermost JWS. The pull
> request also includes restrictions on the serializations (no detached
> payload, no unencoded payload, no unprotected header, etc.). In addition,
> the pull request bans multiple signatures, regardless of the
> serialization used. The use of the Content-Type header, and the list of
> currently possible serializations, is mentioned in its own subsection of
> "Message Transport".
>
> The pull request does not contain advice on how to convert different
> serialization formats before and/or after use with a pre-existing JWS
> library. I have started on a separate conversion guide (
> https://github.com/uhhhh2/jwe-jws-serialization-conversion-guide) for
> that purpose.
>
> The pull request does not specify how a "nested" JWS should be
> serialized. However, I have included an outline of one possible approach to
> this in the pull request's description.
>
> Please let me know what you think about the pull request  (
> https://github.com/ietf-wg-acme/acme/pull/395), and the separate
> conversion guide (https://github.com/uhhhh2/jwe-jws-serialization-c
> onversion-guide)
>
> Logan
>
> On Fri, Jan 12, 2018 at 6:08 PM, Logan Widick <[email-masked]>
> wrote:
>
>> Last night, I briefly surveyed the listings of JWT implementations on
>> jwt.io. I could find only a small handful that appeared to support all
>> serializations, and even fewer that appeared to give programmers control
>> over what serialization was produced. Thus, assuming jwt.io is a
>> sufficiently accurate and comprehensive listing of implementations of all
>> and/or part of the JOSE specs, the developers of many ACME client and
>> server implementations may find themselves needing to convert between
>> serializations before and/or after using JOSE libraries. Such conversion
>> processes, if needed, should be well-documented somewhere.
>>
>> I've started on a very rough draft of a possible JWS and JWE
>> serialization conversion guide at https://github.com/uhhhh2/j
>> we-jws-serialization-conversion-guide. I made the conversion guide draft
>> by copying a few items from the ACME GitHub repository (the Markdown file,
>> the makefile, and the .gitignore), replacing the text from the Markdown
>> file, and renaming the Markdown file. I designed the conversion guide draft
>> to be non-ACME specific, so I've tried to include things like unencoded JWS
>> payloads, JWEs, multiple signatures, detached payloads, etc. If you have
>> any changes or suggestions, please let me know.
>>
>> Logan
>>
>> On Fri, Jan 5, 2018 at 7:11 PM, Jörn Heissler <
>> [email-masked]> wrote:
>>
>>> On Thu, Jan 04, 2018 at 08:03:55 -0600, Logan Widick wrote:
>>> > What do you think of the following:
>>>
>>> > Content type application/jose+json: MUST be supported. If used, the JWS
>>> > will need to be in the Flattened or General serialization. Flattened
>>> MUST
>>> > be supported; General MAY be supported.
>>>
>>> > Content type application/jose: MAY be supported. If used, the JWS MUST
>>> use
>>> > the Compact serialization. Or should this content type not be allowed?
>>>
>>> Agreed. I wouldn't disallow "compact". And it could be clarified:
>>>
>>> The server SHOULD use the "Content-Type" HTTP header as an indication
>>> for the request format.
>>>
>>> > JWS Unprotected Header: Not currently used in ACME. Should this be
>>> banned
>>> > in ACME?
>>>
>>> I don't see much sense in those. But some client implementations might
>>> automatically add an unprotected header like e.g. "cty".
>>> Maybe with a "SHOULD NOT"?
>>>
>>> > Multiple signatures: MAY be supported.
>>>
>>> > Should messages signed by both MAC keys and private keys be allowed?
>>>
>>> This is already forbidden.
>>>
>>> > What about Key IDs not issued by the CA?
>>> > Or are multiple signatures more trouble than they're worth to the
>>> point of
>>> > banning them entirely?
>>> >
>>> > Multiple signatures on messages that need to be signed by account key:
>>> At
>>> > least one signature MUST be from the account key
>>> >
>>> > Multiple signatures on revokeCert: Should this be allowed?
>>> >
>>> > Multiple signatures on externalAccountBinding field of newAccount:
>>> Should
>>> > it be possible to bind to multiple pre-existing accounts? Or should
>>> this
>>> > not be allowed?
>>> >
>>> > Multiple signatures on newAccount: Not allowed?
>>> >
>>> > Multiple signatures on keyChange: Not allowed for outer or inner JWS?
>>>
>>> I see no use case. All the authentication is based on accounts and those
>>> have exactly one keypair. Having multiple signatures would equal using
>>> multiple accounts at the same time. That makes no sense to me.
>>> Client libs would probably not generate multiple signatures
>>> automatically.
>>> Multiple signatures should be banned in my opinion.
>>>
>>> > JWS Unencoded Payload Option (RFC 7797): Not allowed?
>>>
>>> Yep, they would make things very complicated.
>>>
>>> > Conversion guide between the different JWS serialization formats: Is it
>>> > completely safe to assume that any and all programmers given the JWS
>>> RFC,
>>> > pre-existing JWS implementations with sufficient documentation, and
>>> > pre-existing JSON libraries with sufficient documentation could figure
>>> out
>>> > how to convert the serialization formats as needed?
>>>
>>> Why, yes! Of course every programmer can do that! ;-)
>>>
>>> > Or is the conversion
>>> > guide necessary? If the guide is necessary, then include a reference
>>> to a
>>> > separate new or pre-existing conversion guide. If the guide is
>>> necessary,
>>> > and there is no pre-existing conversion guide, how should the new
>>> > conversion guide be published? Should the new conversion guide be
>>> > ACME-specific, or more general (possibly with coverage of JWE as well
>>> as
>>> > JWS features not utilized in ACME)?
>>>
>>> It's not necessary, *most* programmers can figure it out. But it would
>>> doubtlessly be helpful. E.g. I didn't consider the possibility to do
>>> this conversion in an ACME implementation before/after using a
>>> preexisting
>>> JOSE lib.
>>> If such a guide were to be published, it should not be ACME-specific.
>>>
>>>
>>> Cheers
>>> Jörn
>>>
>>
>>
>
> _______________________________________________
> Acme mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/acme
>
>

---

From: Felipe Gasper <[email-masked]>
Date: Fri, 2 Mar 2018 10:47:23 -0500

Could there be some way of using a header like “Accept” for a server to indicate whether it supports jose, jose+json, or both?

-F

> On Mar 2, 2018, at 9:50 AM, Richard Barnes <[email-masked]> wrote:
> 
> Hey all,
> 
> I merged #395 last night (thanks, Logan!).  While I was reviewing that, I noticed that we need to cover the case where the client sends an encoding that the server doesn't understand.  So I've posted a follow-on that adds an error code and requires its usage.  LMK if you have any objections, otherwise I'll merge before submission on Monday.
> 
> On Mon, Feb 12, 2018 at 2:37 PM, Logan Widick <[email-masked]> wrote:
> I've created a new pull request (https://github.com/ietf-wg-acme/acme/pull/395) to partially address the lack of serialization format specification. This pull request requires use of the Content-Type HTTP header to indicate the serialization format of the outermost JWS. The pull request also includes restrictions on the serializations (no detached payload, no unencoded payload, no unprotected header, etc.). In addition, the pull request bans multiple signatures, regardless of the serialization used. The use of the Content-Type header, and the list of currently possible serializations, is mentioned in its own subsection of "Message Transport". 
> 
> The pull request does not contain advice on how to convert different serialization formats before and/or after use with a pre-existing JWS library. I have started on a separate conversion guide (https://github.com/uhhhh2/jwe-jws-serialization-conversion-guide) for that purpose. 
> 
> The pull request does not specify how a "nested" JWS should be serialized. However, I have included an outline of one possible approach to this in the pull request's description. 
> 
> Please let me know what you think about the pull request  (https://github.com/ietf-wg-acme/acme/pull/395), and the separate conversion guide (https://github.com/uhhhh2/jwe-jws-serialization-conversion-guide)
> 
> Logan
> 
> On Fri, Jan 12, 2018 at 6:08 PM, Logan Widick <[email-masked]> wrote:
> Last night, I briefly surveyed the listings of JWT implementations on jwt.io. I could find only a small handful that appeared to support all serializations, and even fewer that appeared to give programmers control over what serialization was produced. Thus, assuming jwt.io is a sufficiently accurate and comprehensive listing of implementations of all and/or part of the JOSE specs, the developers of many ACME client and server implementations may find themselves needing to convert between serializations before and/or after using JOSE libraries. Such conversion processes, if needed, should be well-documented somewhere. 
> 
> I've started on a very rough draft of a possible JWS and JWE serialization conversion guide at https://github.com/uhhhh2/jwe-jws-serialization-conversion-guide. I made the conversion guide draft by copying a few items from the ACME GitHub repository (the Markdown file, the makefile, and the .gitignore), replacing the text from the Markdown file, and renaming the Markdown file. I designed the conversion guide draft to be non-ACME specific, so I've tried to include things like unencoded JWS payloads, JWEs, multiple signatures, detached payloads, etc. If you have any changes or suggestions, please let me know. 
> 
> Logan
> 
> On Fri, Jan 5, 2018 at 7:11 PM, Jörn Heissler <[email-masked]> wrote:
> > will need to be in the Flattened or General serialization. Flattened MUST
> > Content type application/jose: MAY be supported. If used, the JWS MUST use
> > JWS Unprotected Header: Not currently used in ACME. Should this be banned
> > in ACME?
> 
> > Or are multiple signatures more trouble than they're worth to the point of
> > banning them entirely?
> >
> > Multiple signatures on messages that need to be signed by account key: At
> > Multiple signatures on externalAccountBinding field of newAccount: Should
> > it be possible to bind to multiple pre-existing accounts? Or should this
> > not be allowed?
> >
> > completely safe to assume that any and all programmers given the JWS RFC,
> > pre-existing JSON libraries with sufficient documentation could figure out
> > guide necessary? If the guide is necessary, then include a reference to a
> > separate new or pre-existing conversion guide. If the guide is necessary,
> > ACME-specific, or more general (possibly with coverage of JWE as well as
> this conversion in an ACME implementation before/after using a preexisting
> JOSE lib.

---

From: Logan Widick <[email-masked]>
Date: Fri, 2 Mar 2018 13:29:41 -0600

I think the follow-on (#398) includes the Accept header in error responses
(to requests with unacceptable serializations).

Also, there may need to be another follow-on regarding how to serialize
"nested" JWS instances (like the externalAccountBinding and the inner JWS
for a key change).

Logan

On Mar 2, 2018 9:47 AM, "Felipe Gasper" <[email-masked]> wrote:

Could there be some way of using a header like “Accept” for a server to
indicate whether it supports jose, jose+json, or both?

-F

acme/acme/pull/395) to partially address the lack of serialization format
specification. This pull request requires use of the Content-Type HTTP
header to indicate the serialization format of the outermost JWS. The pull
request also includes restrictions on the serializations (no detached
payload, no unencoded payload, no unprotected header, etc.). In addition,
the pull request bans multiple signatures, regardless of the serialization
used. The use of the Content-Type header, and the list of currently
possible serializations, is mentioned in its own subsection of "Message
Transport".
serialization formats before and/or after use with a pre-existing JWS
library. I have started on a separate conversion guide (
https://github.com/uhhhh2/jwe-jws-serialization-conversion-guide) for that
purpose.
> Please let me know what you think about the pull request  (
https://github.com/ietf-wg-acme/acme/pull/395), and the separate conversion
guide (https://github.com/uhhhh2/jwe-jws-serialization-conversion-guide)
> I've started on a very rough draft of a possible JWS and JWE
serialization conversion guide at https://github.com/uhhhh2/jwe-
jws-serialization-conversion-guide. I made the conversion guide draft by
copying a few items from the ACME GitHub repository (the Markdown file, the
makefile, and the .gitignore), replacing the text from the Markdown file,
and renaming the Markdown file. I designed the conversion guide draft to be
non-ACME specific, so I've tried to include things like unencoded JWS
payloads, JWEs, multiple signatures, detached payloads, etc. If you have
any changes or suggestions, please let me know.
MUST
use
banned
At
Should
RFC,
out
necessary,

---

From: Richard Barnes <[email-masked]>
Date: Fri, 2 Mar 2018 17:29:04 -0500

> (to requests with unacceptable serializations).

Indeed it does!

> "nested" JWS instances (like the externalAccountBinding and the inner JWS

If you have suggestions here, a PR would be welcome.  TBH, this issue kind
of makes me want to reverse course and say "all flattened JSON", but I get
the sense that other folks disagree?

--Richard

> > I merged #395 last night (thanks, Logan!).  While I was reviewing that,
> I noticed that we need to cover the case where the client sends an encoding
> > I've created a new pull request (https://github.com/ietf-wg-ac
> me/acme/pull/395) to partially address the lack of serialization format
> conversion guide (https://github.com/uhhhh2/jwe
> -jws-serialization-conversion-guide)
> >
> > Logan
> >

---

From: Jacob Hoffman-Andrews <[email-masked]>
Date: Fri, 2 Mar 2018 14:51:08 -0800

> If you have suggestions here, a PR would be welcome.  TBH, this issue
> kind of makes me want to reverse course and say "all flattened JSON",
> but I get the sense that other folks disagree?
I'd be very much in favor of "all flattened JSON." When we started ACME,
I think there was some concern that libraries might support only Compact
Serialization (https://tools.ietf.org/html/rfc7515#section-7.1). But
experience has shown that Flattened JSON Serialization
(https://tools.ietf.org/html/rfc7515#section-7.2.2) is pretty well
supported across languages.

FWIW, Let's Encrypt's ACMEv2 endpoint only supports Flattened JSON
Serialization.

---

From: Logan Widick <[email-masked]>
Date: Fri, 2 Mar 2018 19:46:31 -0600

I don't really have a preference for which serializations are used, as long
as all ACME implementations (regardless of the programming language used)
can support the chosen serializations, and new serializations can be added
later if necessary.

When I searched the listings on GitHub and jwt.io a month or two ago, I
found some JOSE libraries that didn't support reading all serialization
formats, and/or didn't provide programmers with control over the format
produced. Thus, serialization may still be an issue unless one can convert
between serializations before and/or after using such libraries.

Here is a PR for one possible way to resolve the "root JWS" and "nested
JWS" issue: https://github.com/ietf-wg-acme/acme/pull/403 . This PR
includes the following:

   - Initial definitions of "root JWS" and "nested JWS"
   - Updated requirement for "root JWS" and "nested JWS" under the two
   currently defined content types (application/jose and application/jose+json)
   - Revisions to the registries already defined in the spec (to label
   nested JWS things as such and indicate that the actual types may change
   depending on the serialization formats used)
   - A new serializations registry (so that serializations can be added as
   needed)
      - I put it as a new registry under the "New Registries" section
      because it includes ACME-specific constraints on existing content types
      instead of new content types.
      - I have no experience adding new material to an IANA Considerations
      section. I'm not sure if I put the new material in the right place,
      formatted it correctly, included all required information (like what
      guidance, if any, should be provided), etc.

If it is decided to change course and stick with one serialization format
for now, I can make the following changes to this PR:

   - Clean up the "JWS Serialization Formats" section:
      - In PR #403 as it is today:
         -  "One of the following Content-Type header values MUST be used
         to indicate the JWS serialization format"
         - This section includes a list describing the requirements for
         both JSON (Flattened and General) and Compact Serializations
      - To change course:
         - "The following Content-Type header value MUST be used to
         indicate the JWS serialization format"
         - Delete the unused serialization format from the list
      - Delete the unnecessary format from the new serializations registry

Let me know what you think of this PR.

Logan

On Fri, Mar 2, 2018 at 4:51 PM, Jacob Hoffman-Andrews <[email-masked]> wrote:

> I'd be very much in favor of "all flattened JSON." When we started ACME, I
> think there was some concern that libraries might support only Compact
> experience has shown that Flattened JSON Serialization (
> https://tools.ietf.org/html/rfc7515#section-7.2.2) is pretty well

---

From: Jörn Heissler <[email-masked]>
Date: Sun, 4 Mar 2018 12:53:08 +0100

On Thu, Jan 04, 2018 at 00:07:34 +0100, Jörn Heissler wrote:
> https://ietf-wg-acme.github.io/acme/draft-ietf-acme-acme.html#rfc.section.6.2
> states:
> 
>     In the examples below, JWS objects are shown in the JSON or
>     flattened JSON serialization
> 
> All examples in the ACME specification use only the flattened serialization.

I opened a PR for this: https://github.com/ietf-wg-acme/acme/pull/406

---

From: Jörn Heissler <[email-masked]>
Date: Sun, 4 Mar 2018 13:58:54 +0100

> > (to requests with unacceptable serializations).

The way I understand https://tools.ietf.org/html/rfc7231#section-6.5.6
the 406 status and the Accept header are only valid to negotiate server-to-client
content types.

Is it really okay to use them for client-to-server?
I think code 415 "Unsupported Media Type" is more appropriate:
https://tools.ietf.org/html/rfc7231#section-6.5.13
Together with an error document specifying the supported JWS
serializations.

Cheers
Joern Heissler

---

From: Logan Widick <[email-masked]>
Date: Sun, 4 Mar 2018 07:45:36 -0600

Good catch. Should it be 415 (Unsupported Media Type) plus which of the
following (or which combination of the following):

   - A new problem document field (tentatively named
   "supportedSerializations": an array of media type strings)?
   - A new directory field (tentatively named "supportedSerializations": an
   array of media type strings)?
      - Should this go in the directory's "meta" object, or in the
      directory object itself?
   - A HTTP header?
   - Something else?

Logan

---

From: Jörn Heissler <[email-masked]>
Date: Sun, 4 Mar 2018 15:33:00 +0100

>       - Should this go in the directory's "meta" object, or in the

I like the directory approach with meta. Then a client could
use this information before sending the first POST. Else the client
would need to change an internal state after receiving the error
message. For my own client, I'm planning to support the OpenPGP smart
card. It takes 3 seconds to generate a signature. If a signature is
wasted to find out that the default serialization is not supported, it
would be annoying. Having to write a configuration file "use compact by
default for CA foo" would be stupid too.

This, and the problem document field. "supportedSerializations" sounds
fine.

Should the two features be OPTIONAL?

I don't like HTTP headers, it's quite complicated to parse them correctly.
JSON is so much easier.

Or... specify that flattened MUST BE used :-)

Cheers
Joern Heissler

---

From: Logan Widick <[email-masked]>
Date: Sun, 4 Mar 2018 09:43:11 -0600

How about this: Specify a default format (either "application/jose" for
Compact Serialization, or "application/jose+json" with Flattened
Serialization - I have no preference which one), with optional support for
other formats if needed? Even with JOSE libraries that don't support all
serializations and/or don't provide control over which serialization is
used, a programmer would at least need to know (or experimentally find out)
if a JSON serialization or if the compact one is being produced. If a JSON
serialization is selected as the default, a programmer should be able to
convert between the two JSON serializations easily as needed before and/or
after using a JOSE library. If a JSON format is declared as the default but
the JOSE library only has the compact one, or vice-versa, conversion before
and/or after the JOSE library would be more complex but should still be
doable with guidance.

The directory meta item could be defined as something like:

   - supportedSerializations: An array of supported serialization formats
   as described in {{jws-serialization-formats}}. If this is not specified,
   assume that the server only supports [insert selected default here].

Then, the JWS Serialization Formats section could be changed to something
like the following:

The JSON Web Signature (JWS) specification {{!RFC7515}} contains multiple
JWS serialization formats. When sending an ACME request with a non-empty
body, an ACME client implementation SHOULD use the HTTP Content-Type
{{!RFC7231}} header to indicate which JWS serialization format is used for
encapsulating the ACME request payload.

Each serialization format defined for use in ACME is described with a
content type, and a series of ACME-specific restrictions on root JWS and
nested JWS instances.  A "root JWS" is a JWS used to encapsulate an entire
ACME request payload, and a "nested JWS" is a JWS contained within the ACME
request payload (such as the "externalAccountBinding" described in
{{external-account-binding}} or the "key-change" object described in
{{account-key-roll-over}}). Below are the JWS serialization formats that
are defined for use in ACME:

[same list as before but with the default format coming first]

If no Content-Type is provided, the default serialization type is [insert
selected default here]. Servers MUST support [insert selected default
here]. [NOTE: If a JSON format is selected as the default, say that a
server SHOULD support the other JSON format.] A server MAY support
additional serializations, such as [insert serialization(s) not picked
here], by including a "supportedSerializations" field in the directory
"meta" object as described in {{directory}}.

If a server receives a request using a serialization it does not support,
the server MUST send a response with HTTP status code 415 (Unacceptable
Media Type) and a problem document with error type
"unsupportedSerialization". This problem document SHOULD contain a
"supportedSerializations" array of strings indicating the acceptable
serialization content types.

[TODO: If a client uses the General JSON Serialization but it turns out the
server only supports the Flattened JSON Serialization (or vice-versa),
explain that a 415 response indicates that the client will need to switch
JSON formats]

[TODO: Insert a sentence or two specifying what happens if a supported
serialization is used but the serialization is malformed? Should this be
400 Bad Request + malformed error code + supportedSerializations?]

In the examples below, JWS objects are shown in the Flattened JSON
serialization, with the protected header and payload expressed as
base64url(content) instead of the actual base64-encoded value, so that the
content is readable. [Example readability is a very high priority
regardless of which serialization format is actually chosen as the default,
and the current convention of Flattened JSON + base64url(content) is about
as readable as it gets, so I don't think any changes will need to be made
here]

---

From: Richard Barnes <[email-masked]>
Date: Sun, 4 Mar 2018 17:25:25 -0500

The lengths of the emails in this thread illustrate the complexity risk
here :)

In the interest of simplicity, I would really like to stick to Flattened
JSON unless someone has **strong** objections.

Logan, to your point about library compatibility, two notes: (1) it's OK if
we front-run libraries a little.  It's not hard for libraries to upgrade;
this is only formatting, no crypto changes needed.  (2) Empirically, this
must not be too big a blocker for people, since as Jacob notes, Let's
Encrypt only supports Flattened JSON right now and they've got a bunch of
clients talking to them.

As far as headers / response codes: You're correct that 406 is wrong / 415
is right.  But ISTM that Accept is still the right header to say what is
right.  So the server should return 415+Accept.  Copying Thomson to check
our work here.

--Richard

On Sun, Mar 4, 2018 at 10:43 AM, Logan Widick <[email-masked]>
wrote:

> [TODO: If a client uses the General JSON Serialization but it turns out
> the server only supports the Flattened JSON Serialization (or vice-versa),
> On Sun, Mar 4, 2018 at 8:33 AM, Jörn Heissler <

---

From: Richard Barnes <[email-masked]>
Date: Sun, 4 Mar 2018 23:38:59 -0500

I also note that there's no issue with Accept if we require the use of the
Flattened JSON serialization.

https://i.imgflip.com/25r2ui.jpg

https://github.com/ietf-wg-acme/acme/pull/410

On Sun, Mar 4, 2018 at 6:28 PM, Martin Thomson <[email-masked]>
wrote:

> That's a bit silly.  I'll follow-up with httpbis.  I think that's an
> error, though probably only an error of omission.  7694 was so fixated
> on solving the content-coding issue, it neglected the obvious
> accompanying fix.
>
> On Mon, Mar 5, 2018 at 9:38 AM, Richard Barnes <[email-masked]> wrote:
> > How about Accept?  It looks like 7694 gives the server a way to specify
> > encodings, but not the content type.  But 7231 says that Accept only
> replies
> > to response media types.
> >
> > On Sun, Mar 4, 2018 at 5:33 PM, Martin Thomson <[email-masked]
> >
> > wrote:
> >>
> >> 415 is for the case where a client provides bad request content, so yes.
> >> See rfc7694 for details.
> >>
> >>  406 is for failed conneg. Not something you expect to see much here.
> >>
> >>
> >> On 5 Mar. 2018 09:25, "Richard Barnes" <[email-masked]> wrote:
> >>
> >> Logan, to your point about library compatibility, two notes: (1) it's OK
> >> if we front-run libraries a little.  It's not hard for libraries to
> upgrade;
> >>> supportedSerializations: An array of supported serialization formats as
> >>> described in {{jws-serialization-formats}}. If this is not specified,
> assume
> >>> that the server only supports [insert selected default here].
> >>>
> >>> {{account-key-roll-over}}). Below are the JWS serialization formats
> that are
> >>> defined for use in ACME:
> >>>
> here].
> >>> [NOTE: If a JSON format is selected as the default, say that a server
> SHOULD
> >>> support the other JSON format.] A server MAY support additional
> >>> serializations, such as [insert serialization(s) not picked here], by
> >>> including a "supportedSerializations" field in the directory "meta"
> object
> >>> as described in {{directory}}.
> >>>
> >>> serialization is used but the serialization is malformed? Should this
> be 400
> >>> Bad Request + malformed error code + supportedSerializations?]
> >>>
> regardless
> >>> of which serialization format is actually chosen as the default, and
> the
> >>> current convention of Flattened JSON + base64url(content) is about as
> >>> readable as it gets, so I don't think any changes will need to be made
> here]
> >>>
> >>>
> >>> On Sun, Mar 4, 2018 at 8:33 AM, Jörn Heissler
> >>> <[email-masked]> wrote:
> >>>>

---

From: Martin Thomson <[email-masked]>
Date: Mon, 5 Mar 2018 16:02:43 +1100

Unless you believe that an alternative format is ever desirable.  CBOR
for version 2 might be a terrible idea, but I know the IETF well
enough not to rule that out entirely.

On Mon, Mar 5, 2018 at 3:38 PM, Richard Barnes <[email-masked]> wrote:
>> > On Sun, Mar 4, 2018 at 5:33 PM, Martin Thomson
>> > <[email-masked]>
>> > wrote:
>> >>

---

From: Felipe Gasper <[email-masked]>
Date: Mon, 5 Mar 2018 09:11:02 -0500

For what it’s worth:

Regarding alternative formats, I think ACME over WebSocket would be a great thing. Replay-nonce would go away, and clients wouldn’t need to poll for the certificate unless the connection dropped. The server could send the certificate as soon as it’s ready. A simple handshake at the start could take the place of JWS, too.

-F

> On Mar 5, 2018, at 12:02 AM, Martin Thomson <[email-masked]> wrote:
>

---

From: Jörn Heissler <[email-masked]>
Date: Mon, 5 Mar 2018 15:35:31 +0100

> Regarding alternative formats, I think ACME over WebSocket would be a great thing. Replay-nonce would go away, and clients wouldn’t need to poll for the certificate unless the connection dropped. T…

Moving to websocket would mean having to specify a new websocket based
protocol and reimplementing all servers/clients.
You'd also need to consider MitM (e.g. by CDN or expensive enterprise MitM
appliances). Doing a handshake at the beginning won't be enough to keep
those from taking over a session after the handshake.

If you wanted to eliminate the polling, it would be possible to change
the finalize endpoint to return the certificate directly, even if it
takes a while until the HTTP response is sent.

Anyway, I'm satisfied with the current protocol. Complexity should (and
can) be addressed by client implementors.

---

From: Felipe Gasper <[email-masked]>
Date: Mon, 5 Mar 2018 10:28:25 -0500

> On Mar 5, 2018, at 9:35 AM, Jörn Heissler <[email-masked]> wrote:
> 

I would think “ACME/bidi” could live alongside ACME/HTTP. The bidirectional variant could be described in a separate RFC that gives the differences--which would basically just be how to translate the HTTP-specific parts of ACME/HTTP to “pure messages”.

WebSocket wouldn’t necessarily be the only persistent-connection format; any reliable, message-oriented protocol (e.g., SCTP or WAMP “RawSocket”) could work.

> takes a while until the HTTP response is sent.

This would alleviate the polling for the certificate but not for authz status. If I have an Order with, say, 5 authzs, I’ll still need to poll for the state of those.

A transport layer that allows asynchronous server-to-client communication would facilitate quicker feedback to the client and a smoother user experience overall. 

-F

---

From: "Matthew D. Hardeman" <[email-masked]>
Date: Mon, 5 Mar 2018 12:13:30 -0600

My working experience is primarily outside the PKI space, but I can offer some perspectives on scalability and deployment architecture issues.

WebSocket is entirely appropriate for real-time or near-real-time bidirectional communications of an asynchronous nature.

The overhead of WebSocket as a pure transport for the various ACME messages back and forth between the client and the server could be done, but would any server side implementation ever take full advantage with the appropriate async callbacks to actually achieve faster certificate delivery.

Especially with CT logging being a pragmatic requirement, time-to-delivery for certificates is likely to increase (slightly) rather than decrease.

In the kind of pattern that getting a certificate represents it seems WebSocket would add significant overhead without a lot of performance advantage.

If the content and nature of the messages would change based on WebSocket used as a transport, then you’ve got a whole new set of threats to model and consider.

Perhaps before going down that road, it would be useful to poll server-side implementers and deployers and see if there’s interest in a WebSocket delivery channel.

> On Mar 5, 2018, at 8:11 AM, Felipe Gasper <[email-masked]> wrote:
> 
> For what it’s worth:
>

---

From: Felipe Gasper <[email-masked]>
Date: Mon, 5 Mar 2018 16:50:22 -0500

> On Mar 5, 2018, at 1:13 PM, Matthew D. Hardeman <[email-masked]> wrote:
> 

Quick point: the alleviation of polling would go for authz status as well as to certificate delivery.

A certificate order that has 10 domains needs to poll for the status of all 10 of those domains’ authorizations as well as the certificate issuance. “ACME/bidi” would remove all 11 of those needs to poll.

Thanks for those who have given this suggestion their consideration. I don’t mean to “gum up the gears” for the main ACME work, but as I’ve been writing ACME clients the polling stuff has stuck out to me like a sore thumb.

It’s worth noting, too, that concerns about overhead may be alleviated if we do get a usable WebSocket-over-HTTP/2 implementation. Or, maybe someone will expose an SCTP endpoint, or a raw TCP endpoint that implements a simple message-boundary layer. I think the question of pure-message, bidirectional transport is more relevant than a specific transport implementation.

-F

---

From: Richard Barnes <[email-masked]>
Date: Mon, 5 Mar 2018 17:27:28 -0500

Thomson: Could h2 push replace some of the polling here?

On Mon, Mar 5, 2018 at 4:50 PM, Felipe Gasper <[email-masked]>
wrote:

> > Especially with CT logging being a pragmatic requirement,
> time-to-delivery for certificates is likely to increase (slightly) rather
> than decrease.
>
> A certificate order that has 10 domains needs to poll for the status of
> all 10 of those domains’ authorizations as well as the certificate
> issuance. “ACME/bidi” would remove all 11 of those needs to poll.
>
> Thanks for those who have given this suggestion their consideration. I
> don’t mean to “gum up the gears” for the main ACME work, but as I’ve been
> writing ACME clients the polling stuff has stuck out to me like a sore
> thumb.
>
> It’s worth noting, too, that concerns about overhead may be alleviated if
> we do get a usable WebSocket-over-HTTP/2 implementation. Or, maybe someone
> will expose an SCTP endpoint, or a raw TCP endpoint that implements a
> simple message-boundary layer. I think the question of pure-message,
> bidirectional transport is more relevant than a specific transport
> implementation.
>
> -F

---

From: Martin Thomson <[email-masked]>
Date: Tue, 6 Mar 2018 09:31:26 +1100

Sure.  Plenty of ways to do that.

If your primary concern is issuance, then you don't even need server
push, you can just long-poll.  In HTTP/1.1, that's gross because it
ties up a connection and has some disgusting keep-alive properties.
In h2 there is no opportunity cost to worry about, the connection
remains usable for other things while you wait.  (You can use
prefer:wait to control timing if you want a standards-based solution:
https://tools.ietf.org/html/rfc7240#section-4.3)

On Tue, Mar 6, 2018 at 9:27 AM, Richard Barnes <[email-masked]> wrote:
>> all 10 of those domains’ authorizations as well as the certificate issuance.
>> “ACME/bidi” would remove all 11 of those needs to poll.
>>
>> will expose an SCTP endpoint, or a raw TCP endpoint that implements a simple
>> message-boundary layer. I think the question of pure-message, bidirectional
>> transport is more relevant than a specific transport implementation.
>>
>> -F
>
>

---

From: "Matthew D. Hardeman" <[email-masked]>
Date: Mon, 5 Mar 2018 16:58:39 -0600

> On Mar 5, 2018, at 3:50 PM, Felipe Gasper <[email-masked]> wrote:
> 
> 

While it eliminates the need for those 11 targets to poll for, it adds the not insignificant infrastructure cost of nailing up a stateful connection.  It’s worth mention that the past decade has seen massive improvements in the network stack from the low layer up through application layer apis as it would pertain to handling a great many of these connections, but there is still some cost to nailing up large numbers of concurrent connections.

As others note, h2 offers numerous models under which you could achieve the same benefits of getting asynchronous return of results over a single TCP session.

I would think that even if you have a clear, consistent, bidirectional communication channel that it would still make sense for the individual message content to remain unchanged.  (Not optimizing out replay-nonce, etc.)  As a general matter, protocols dealing with security and integrity probably achieve better and stronger implementations in the field when variability and supported cases are reduced to the minimum required to achieve the protocol’s goals.  Where possible, I would think removal of edge cases, exceptions, and different handling for different messaging protocols would be a positive value.

---

From: Felipe Gasper <[email-masked]>
Date: Mon, 5 Mar 2018 19:56:53 -0500

> On Mar 5, 2018, at 5:58 PM, Matthew D. Hardeman <[email-masked]> wrote:
> 

I would think the grand majority of such connections would go away once certificate renewals were done. There’d be no reason to hold onto the connection afterward, as it ties up resources for the client as well as the server.

> As others note, h2 offers numerous models under which you could achieve the same benefits of getting asynchronous return of results over a single TCP session.

I’ve not seen much discussion of h2 server push in this context; its use seems (by design) limited to the case of pre-sending resources that the client will either accept or reject via RST_STREAM.

There is SSE (Server-Sent Events), but my understanding is that that protocol is effectively abandoned, supported browser implementations notwithstanding. (Edge/IE in particular doesn’t support it.)

Is there another mechanism that you and Thomson have in mind, or what am I missing?

Thank you,
-Felipe Gasper
Mississauga, Ontario