Subject: [jose] Fwd: New Version Notification for draft-rundgren-predictable-serialization-for-json-00.txt

From: Anders Rundgren <[email-masked]>
Date: Wed, 4 Nov 2015 15:23:35 +0100

A new version of I-D, draft-rundgren-predictable-serialization-for-json-00.txt
has been successfully submitted by Anders Rundgren and posted to the
IETF repository.

Name:		draft-rundgren-predictable-serialization-for-json
Revision:	00
Title:		Predictable Serialization for JSON Tools
Document date:	2015-11-04
Group:		Individual Submission
Pages:		8
URL:            https://www.ietf.org/internet-drafts/draft-rundgren-predictable-serialization-for-json-00.txt
Status:         https://datatracker.ietf.org/doc/draft-rundgren-predictable-serialization-for-json/
Htmlized:       https://tools.ietf.org/html/draft-rundgren-predictable-serialization-for-json-00

Abstract:
    This specification outlines an optional characteristic of JSON tools
    like parsers, serving two entirely different purposes: 1) Making
    information-rich JSON messages more human-readable by honoring the
    originator's conventions.  2) Facilitating simple "Signed JSON"
    schemes without necessarily needing specific signature text-
    processing software.  Finally, there is a section containing
    recommendations for interoperability with systems based on EcmaScript
    V6 (AKA JavaScript).

---

From: "Manger, James" <[email-masked]>
Date: Mon, 9 Nov 2015 11:12:14 +1100

Comments on draft-rundgren-predictable-serialization-for-json:

1. It is useful to have a canonical form of JSON defined, even if most applications should avoid relying on a canonical form where possible.

2. Signing JSON is a primary use-case for a canonical form, but should be specified separately. Defining the "signature" member in section 3 is too incomplete (which key, which algorithm etc) so it should be omitted.
[I wonder if an easy way to define how to embed a signature in the JSON value being signed would be to define, say, a "jws" member that holds "protected", "header", and "signature" members. Those member are as per JWS [RFC7515], except "protected" is not base64-encoded as its canonical form is used. The payload is the canonical form once "jws" is removed.]

3. The sample in the introduction should show a few more features: some string escapes, and changing whitespace. How about:
  {
    "device": "Pump 1\/2 \u0022Red\u0022",
    "value": 0.000000000000000001
  }
and
  { "value":1e-18, "device":"Pump 1/2 \"Red\"" }

4. Don't require "empty properties" to be rejected. It's not clear if an empty property is one with a null value, or "" as its name; both of which are valid JSON that should be accepted.

5. You don't need to say the "'\/' escape sequence MUST be honoured on input" as this is no different from all the other valid escape sequences that also need to be honoured by a JSON parser (unless you are implying \/ needs to be preserved when creating the canonical form). Suggested text:
  2.2.2 String Normalization
  The canonical form of a JSON string uses escape sequences as follows:
  * \" \\ \b \f \n \r and \t MUST be used to represent their respective characters
  * \u00xx (where xx are lowercase hex digits) MUST be used to represent the other 27 control characters in the range U+0000 to U+001F 
  * All other characters MUST NOT be escaped
  Note: the canonical form does not use the \/ escape sequence, nor does it escape the line and paragraph separator characters U+2028 and U+2029.

6. Preserving the form of numbers is too unnatural for most uses of JSON (where JSON is parsed into primitive types such as a 64-bit IEEE float). We can and should specify a canonical form for decimals in general (exponent iff value outside [1e-6, 1e21), no leading or trailing zeros, "e" for exponent, + or - on exponent); plus a canonical form for translating from binary floating point formats (use shortest then closest significand that parses to the float). Plus advice about rounding floats to lower precision (eg 14 digits) to improve chances of interop.

7. For interop with ECMAScript JSON.parse/stringify you don't have to forbid integer-like member names, eg "1". It is sufficient to require they occur first and in numeric order.

8. Offering specific code in 3.3 to minimise interop issues from ECMAScript's leniency is nice, but shouldn't be "REQUIRED". Need to clarify that this sort of code should only be used when initially creating JSON (eg signing), not when re-creating JSON after parsing it (eg verifying).

9. It is curious that the special value the code uses for rounding to zero (< 2.22507385850721E-308) isn't the smallest non-subnormal 64-bit float, but the 4th smallest.

--
James Manger

-----Original Message-----
From: jose [mailto:[email-masked]] On Behalf Of Anders Rundgren
Sent: Thursday, 5 November 2015 1:24 AM
To: [email-masked]
Subject: [jose] Fwd: New Version Notification for draft-rundgren-predictable-serialization-for-json-00.txt

A new version of I-D, draft-rundgren-predictable-serialization-for-json-00.txt
has been successfully submitted by Anders Rundgren and posted to the IETF repository.

Abstract:

_______________________________________________
jose mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/jose

---

From: Anders Rundgren <[email-masked]>
Date: Mon, 9 Nov 2015 07:29:23 +0100

On 2015-11-09 01:12, Manger, James wrote:

Thanx James for your detailed comments!

I guess the main problem is separating the use-cases from the specification.
As a hands-on-guy I tend to hover around use-cases.

Comments-in-line:

> 1. It is useful to have a canonical form of JSON defined, even if most
 > applications should avoid relying on a canonical form where possible.

I'm not really prescribing canonicalization but predictable serialization
which is useful for any system although the motives for (re)defining property
order the way EMCA did was due to misunderstandings by practitioners rather
than for supporting signatures or human consumption ("debugging") which
were my sole motivations.

> 2. Signing JSON is a primary use-case for a canonical form, but should
 > be specified separately. Defining the "signature" member in section 3
 > is too incomplete (which key, which algorithm etc) so it should be omitted.

The purpose was to show the principles for an "in-object" signature.

> [I wonder if an easy way to define how to embed a signature in the
 > JSON value being signed would be to define, say, a "jws" member that
 > holds "protected", "header", and "signature" members. Those member
 > are as per JWS [RFC7515], except "protected" is not base64-encoded
 > as its canonical form is used. The payload is the canonical form once
 > "jws" is removed.]

It would be easy but personally I would drop "header" and assume that all
members except the signature value property would be a part of the signed data.
This is what my JCS scheme [1] already does although I have taken the liberty
to give the solution more structure and verbosity.

IF there IS an interest in creating a variant of JWS along these lines, I could
consider updating JCS but I really hate the idea of bringing in legacy constructs
like "x5c" which use Base64 rather than Base64URL.  Personally I think supporting
the JOSE algorithms is the most crucial part.

But I'm pragmatic, so what do you JOSE guys out there think???

>
> 3. The sample in the introduction should show a few more features:
 > some string escapes, and changing whitespace. How about:
>    {

NP.

> 4. Don't require "empty properties" to be rejected. It's not clear if
 > an empty property is one with a null value, or "" as its name; both of
 > which are valid JSON that should be accepted.

You are right.  I just considered "":5 as plain stupid :-)

>
> 5. You don't need to say the "'\/' escape sequence MUST be honoured on input"
 > as this is no different from all the other valid escape sequences that also
 > need to be honoured by a JSON parser (unless you are implying \/ needs to be
 > preserved when creating the canonical form). Suggested text:
>    * \u00xx (where xx are lowercase hex digits) MUST be used to represent the
 >   other 27 control characters in the range U+0000 to U+001F
>    Note: the canonical form does not use the \/ escape sequence, nor does it
 >    escape the line and paragraph separator characters U+2028 and U+2029.

Hum...I actually meant the same thing, I just expressed it differently and probably
with less "sharpness".   I have indeed seen JSON serializers outputting \/.

>
> 6. Preserving the form of numbers is too unnatural for most uses of JSON
 > (where JSON is parsed into primitive types such as a 64-bit IEEE float).

OK.  Maybe the whole spec should be redesigned for ES6?

 > We can and should specify a canonical form for decimals in general
 > (exponent iff value outside [1e-6, 1e21), no leading or trailing zeros,
 > "e" for exponent, + or - on exponent); plus a canonical form for translating
 > from binary floating point formats (use shortest then closest significand
 > that parses to the float). Plus advice about rounding floats to lower
 > precision (eg 14 digits) to improve chances of interop.

I guess this is the true pain point.  Fortunately it turned out that your
assumption that most ES6 implementations [2] use the "better" approach as
mentioned in the ES6 specification.  I just finished a test with 100M random
values without hiccups.

Rounding as I suggested in the I-D (using toPrecision) didn't work well in practice :-(
I ran into endless trail-and-error situations which is quite uncool so it is better to
rely on a canonicalized format.

> 7. For interop with ECMAScript JSON.parse/stringify you don't have
 > to forbid integer-like member names, eg "1". It is sufficient to
 > require they occur first and in numeric order.

True but I was rather thinking about the other end.  Requiring free-standing
JSON parsers like mine to implement a scheme that only makes sense in a
JavaScript-environment is not my cup of tea.  It's a very small price to pay.
At least I wouldn't RECOMMEND using numeric property names unless they align
both with creation order AND ES6-order.

> 8. Offering specific code in 3.3 to minimise interop issues from
 > ECMAScript's leniency is nice, but shouldn't be "REQUIRED". Need
 > to clarify that this sort of code should only be used when initially
 > creating JSON (eg signing), not when re-creating JSON after parsing it (eg verifying).

I intend to drop this part entirely.  See your point #6.

>
> 9. It is curious that the special value the code uses for rounding
 > to zero (< 2.22507385850721E-308) isn't the smallest non-subnormal
 > 64-bit float, but the 4th smallest.

This was a good intention.  It didn't really work as expected.  Point #6.

Cheers,
Anders Rundgren

1] https://cyberphone.github.io/openkeystore/resources/docs/jcs.html#Sample_Signature

2] https://mailarchive.ietf.org/arch/msg/jose/6r82zYZseVkEC1DDFxAkUEvLNqc