Subject: Consensus call for qlog serialization format (issue #144)

From: Lucas Pardue <[email-masked]>
Date: Mon, 2 Aug 2021 14:23:42 +0100

Hello QUIC WG,

During the IETF 111 meeting discussion of qlog, we discussed the
serialization format tracked on issue #114 [1].

The feeling in the room was to keep the JSON serialization format. Noting
that implementations can use their own intermediate formats and transform
to and from JSON as needed, and that future documents could specify other
interop formats if there is sufficient interest.

The proposed resolution for this matter is to keep the JSON serialization
format as the canonical interoperable format. This email seeks to establish
consensus for this. If you have comments, for or against, please respond on
the issue. The call will run until EoD August 9 2021.

Best Regards
Matt & Lucas
QUIC WG Chairs

[1] - https://github.com/quicwg/qlog/issues/144

---

From: Nick Banks <[email-masked]>
Date: Mon, 2 Aug 2021 13:37:44 +0000

I support this plan.

Thanks,
Nick Banks

Sent from Outlook<http://aka.ms/weboutlook>

________________________________
From: QUIC <[email-masked]> on behalf of Lucas Pardue <[email-masked]>
Sent: Monday, August 2, 2021 8:23 AM
To: QUIC WG <[email-masked]>
Cc: QUIC WG Chairs <[email-masked]>
Subject: Consensus call for qlog serialization format (issue #144)

Hello QUIC WG,

During the IETF 111 meeting discussion of qlog, we discussed the serialization format tracked on issue #114 [1].

The feeling in the room was to keep the JSON serialization format. Noting that implementations can use their own intermediate formats and transform to and from JSON as needed, and that future documents could specify other interop formats if there is sufficient interest.

The proposed resolution for this matter is to keep the JSON serialization format as the canonical interoperable format. This email seeks to establish consensus for this. If you have comments, for or against, please respond on the issue. The call will run until EoD August 9 2021.

Best Regards
Matt & Lucas
QUIC WG Chairs

[1] - https://github.com/quicwg/qlog/issues/144<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fquicwg%2Fqlog%2Fissues%2F144&data=04%7C01%7Cnibanks%40microsoft.com%7C14d9566093f6401187fc08d955b8d200%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637635074580404372%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Tr0Q1WxiHTf52di1o6Vr3IzGwdEN6LWYyWa10XNhI5E%3D&reserved=0>

---

From: James <[email-masked]>
Date: Mon, 2 Aug 2021 17:14:30 +0200

I support this plan assuming that requirements for serialisation and 
value representation remain part of the document (which Section 6 of 
draft-ietf-quic-qlog-main-schema-00 does already).

- J

On 02-08-2021 15:23, Lucas Pardue wrote:
> The feeling in the room was to keep the JSON serialization format. 
> Noting that implementations can use their own intermediate formats and 
> transform to and from JSON as needed, and that future documents could 
> specify other interop formats if there is sufficient interest.
>
> The proposed resolution for this matter is to keep the JSON 
> serialization format as the canonical interoperable format. This email 
> seeks to establish consensus for this. If you have comments, for or 
> against, please respond on the issue. The call will run until EoD 
> August 9 2021.
>
> Best Regards
> Matt & Lucas
> QUIC WG Chairs
>
> <https://github.com/quicwg/qlog/issues/144>

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 2 Aug 2021 15:59:41 +0000

> The proposed resolution for this matter is to keep the JSON serialization format as the canonical interoperable format. This email seeks to establish consensus for this. If you have comments, for o…

Which JSON serialization is being discussed here? If it is the one in Section 6.1, it is explicitly not canonical, and not interoperable. For example, 6.1.1 gives QLOG emitters a choice of writing large numbers a numbers or strings. Section 6.1.2.1 gives multiple ways of truncating byte strings. Section 6.1.4 allows QLOG emitters use non-standard JSON (trailing commas). There are also other SHOULD-level requirements for JSON emitters in other parts of the draft, such as in section 3.

I would support having a JSON serialization format as the canonical interoperable format, but only if there is eventually a canonical JSON format, which there is not currently. Without a canonical JSON serialization, there can be no interoperability with other formats.

--Paul HOffman

---

From: Lucas Pardue <[email-masked]>
Date: Mon, 2 Aug 2021 18:52:31 +0100

Hi Paul,

Response in-line:

On Mon, Aug 2, 2021 at 4:59 PM Paul Hoffman <[email-masked]> wrote:

> against, please respond on the issue. The call will run until EoD August 9
> 2021.
>
> Which JSON serialization is being discussed here? If it is the one in
> Section 6.1, it is explicitly not canonical, and not interoperable. For
> example, 6.1.1 gives QLOG emitters a choice of writing large numbers a
> numbers or strings. Section 6.1.2.1 gives multiple ways of truncating byte
> strings. Section 6.1.4 allows QLOG emitters use non-standard JSON (trailing
> commas). There are also other SHOULD-level requirements for JSON emitters
> in other parts of the draft, such as in section 3.

> I would support having a JSON serialization format as the canonical
> interoperable format, but only if there is eventually a canonical JSON
> format, which there is not currently. Without a canonical JSON
> serialization, there can be no interoperability with other formats.
>

Great question!

The intention here is to determine consensus on using _a JSON_
serialization and not a completely different format. The current draft is a
starting point but is not intended to be the final solution.

Should the WG determine consensus on using _a JSON_ serialization, the
editors and WG will unblocked on iterating the finer details of _the JSON_
serialization. The specific examples you provide are good, they can be
tracked as issues against the draft.

I'd also like to clarify that this consensus call relates to only a JSON
serialization. Although the issue mentions streaming serialization (and
NDJSON) and Robin presented some slides on the topic during the IETF 111
meeting, our intention continue that discussion separately pending the
outcome of this call. I've created a new issue to make this clear [1].

Does that clarify things sufficiently?

Cheers
Lucas

[1] - https://github.com/quicwg/qlog/issues/172

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 2 Aug 2021 18:38:04 +0000

On Aug 2, 2021, at 10:52 AM, Lucas Pardue <[email-masked]> wrote:
> The intention here is to determine consensus on using _a JSON_ serialization and not a completely different format.

The associated question, which was not asked on this thread is "should there be any serialization format chosen, or just a data definition?". I would have leaned towards that, particularly because of the thorny JSON serialization issues brought up in the draft. However, that discussion often gets too meta for folks who want to start debugging and logging now.

> The current draft is a starting point but is not intended to be the final solution. 
> 
> Should the WG determine consensus on using _a JSON_ serialization, the editors and WG will unblocked on iterating the finer details of _the JSON_ serialization. The specific examples you provide are good, they can be tracked as issues against the draft.
> 
> I'd also like to clarify that this consensus call relates to only a JSON serialization. Although the issue mentions streaming serialization (and NDJSON) and Robin presented some slides on the topic during the IETF 111 meeting, our intention continue that discussion separately pending the outcome of this call. I've created a new issue to make this clear [1].
> 

It does, thanks! If the WG wants to have a canonical, interoperable serialization instead of a set of data types (such as Section 1.1 in the draft), I support JSON because of its commonness and mostly-useful mapping to the problem space. 

--Paul Hoffman

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 5 Aug 2021 13:20:31 +0100

Hi Paul,

On Mon, Aug 2, 2021 at 7:38 PM Paul Hoffman <[email-masked]> wrote:

> serialization and not a completely different format.
>
> The associated question, which was not asked on this thread is "should
> there be any serialization format chosen, or just a data definition?". I
> would have leaned towards that, particularly because of the thorny JSON
> serialization issues brought up in the draft. However, that discussion
> often gets too meta for folks who want to start debugging and logging now.
>

There's quite a large population of people already using JSON for qlog,
prior to WG adoption. So far, there has been a practical benefit to an
interoperable serialization format that was strongly linked to the schema
definition by draft version. During the adoption call I don't recall there
being suggestions for dropping serialization altogether from
draft-ietf-quic-qlog-main-schema.

Wearing no hats: I think including at least one serialization format as
part of the work is a useful function to drive design and development
decisions. There might be a question about whether
draft-ietf-quic-qlog-main-schema should include the concrete serialization
or if it should be spun out to a separate document. However, I think this
question is tangential to the current call.

Cheers,
Lucas

---

From: Roberto Peon <[email-masked]>
Date: Thu, 5 Aug 2021 16:54:54 +0000

Including at least one interoperable format is good.
Having it be ascii is annoying (not very performant, requires more I/O, and thus decreases the fidelity at which we can ultimately capture events)
…but it is probably better than nothing.

-=R

From: QUIC <[email-masked]> on behalf of Lucas Pardue <[email-masked]>
Date: Thursday, August 5, 2021 at 5:21 AM
To: Paul Hoffman <[email-masked]>
Cc: QUIC WG <[email-masked]>, QUIC WG Chairs <[email-masked]>
Subject: Re: [Ext] Consensus call for qlog serialization format (issue #144)

Hi Paul,

On Mon, Aug 2, 2021 at 7:38 PM Paul Hoffman <[email-masked]<mailto:[email-masked]>> wrote:

There's quite a large population of people already using JSON for qlog, prior to WG adoption. So far, there has been a practical benefit to an interoperable serialization format that was strongly linked to the schema definition by draft version. During the adoption call I don't recall there being suggestions for dropping serialization altogether from draft-ietf-quic-qlog-main-schema.

Wearing no hats: I think including at least one serialization format as part of the work is a useful function to drive design and development decisions. There might be a question about whether draft-ietf-quic-qlog-main-schema should include the concrete serialization or if it should be spun out to a separate document. However, I think this question is tangential to the current call.

Cheers,
Lucas

---

From: Paul Hoffman <[email-masked]>
Date: Thu, 5 Aug 2021 16:41:52 +0000

> The proposed resolution for this matter is to keep the JSON serialization format as the canonical interoperable format. This email seeks to establish consensus for this. If you have comments, for o…

The call asked whether the (now "a") JSON serialization format will be the canonical interoperable format. That is quite different than "including at least one serialization format". As a concrete example:

- The JSON serialization says that numbers such as packet_number may be represented as a number or a string

- The data format says that the numbers such as packet_number are uint64

If the JSON serialization format is the canonical interoperable format, and I'm writing a CBOR emitter, I would be allowed to write packet_number as a number or string because both could convert to JSON. However, if the data format is the canonical interoperable format, I would only be able to write it as a number.

Thus, it is critical for interoperability between formats to specify if the data definition is canonical, or if the JSON serialization is canonical. If the latter, there really is no need for the data definitions at all; if this choice is made, interoperability would be more likely if the data definitions were removed.

I hope this helps clarify why the WG needs to choose one or the other.

--Paul Hoffman

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Thu, 5 Aug 2021 12:17:59 -0500

On this point ...

On Thu, Aug 5, 2021 at 11:55 AM Roberto Peon <fenix=[email-masked]>
wrote:

> Having it be ascii is annoying (not very performant, requires more I/O,
> and thus decreases the fidelity at which we can ultimately capture events)

Having at least one interoperable format is excellent, since you and I can
always exchange data using that format.  Not having something that's fairly
widely implemented, at least among early implementers, really is "nothing",
so this is "better than nothing".

Having the baseline format be annoying can be OK, because that motivates
people to explore other formats while using the baseline format, if it's
TOO annoying.

I'm not saying it would have been a great plan, but at IETF 83 during
discussions about RTCWeb codec selections, when the choice between VP8 and
H.264 seemed deadlocked, the working group had a serious conversation about
picking H.261 as an annoying (or worse) baseline so that we didn't have
complete communication breakdowns, and then implementations could negotiate
for a better choice if that seemed necessary.

(see https://www.ietf.org/proceedings/83/minutes/minutes-83-rtcweb.txt for
proof).

Best,

Spencer

---

From: Roberto Peon <[email-masked]>
Date: Thu, 5 Aug 2021 17:22:22 +0000

You can consider my prior statement as grudging support, if that wasn’t clear.
I’d prefer a format with zero representational ambiguity, because we’ve all seen how fun that ambiguity ends up being to deal with in the ecosystem, but that doesn’t stop it being true that zero is worse than one.
-=R

From: QUIC <[email-masked]> on behalf of Spencer Dawkins at IETF <[email-masked]>
Date: Thursday, August 5, 2021 at 10:18 AM
To: Roberto Peon <fenix=[email-masked]>
Cc: Paul Hoffman <[email-masked]>, QUIC WG <[email-masked]>, QUIC WG Chairs <[email-masked]>, Lucas Pardue <[email-masked]>
Subject: Re: [Ext] Consensus call for qlog serialization format (issue #144)

On this point ...

On Thu, Aug 5, 2021 at 11:55 AM Roberto Peon <fenix=[email-masked]<mailto:[email-masked]>> wrote:
Including at least one interoperable format is good.
Having it be ascii is annoying (not very performant, requires more I/O, and thus decreases the fidelity at which we can ultimately capture events)
…but it is probably better than nothing.

Having at least one interoperable format is excellent, since you and I can always exchange data using that format.  Not having something that's fairly widely implemented, at least among early implementers, really is "nothing", so this is "better than nothing".

Having the baseline format be annoying can be OK, because that motivates people to explore other formats while using the baseline format, if it's TOO annoying.

I'm not saying it would have been a great plan, but at IETF 83 during discussions about RTCWeb codec selections, when the choice between VP8 and H.264 seemed deadlocked, the working group had a serious conversation about picking H.261 as an annoying (or worse) baseline so that we didn't have complete communication breakdowns, and then implementations could negotiate for a better choice if that seemed necessary.

(see https://www.ietf.org/proceedings/83/minutes/minutes-83-rtcweb.txt<https://www.ietf.org/proceedings/83/minutes/minutes-83-rtcweb.txt> for proof).

Best,

Spencer

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 5 Aug 2021 20:38:31 +0100

Hi Paul,

On Thu, Aug 5, 2021 at 5:41 PM Paul Hoffman <[email-masked]> wrote:

> The call asked whether the (now "a") JSON serialization format will be the
> canonical interoperable format. That is quite different than "including at
> least one serialization format". As a concrete example:
>

> If the JSON serialization format is the canonical interoperable format,
> and I'm writing a CBOR emitter, I would be allowed to write packet_number
> as a number or string because both could convert to JSON. However, if the
> data format is the canonical interoperable format, I would only be able to
> write it as a number.
>
> Thus, it is critical for interoperability between formats to specify if
> the data definition is canonical, or if the JSON serialization is
> canonical. If the latter, there really is no need for the data definitions
> at all; if this choice is made, interoperability would be more likely if
> the data definitions were removed.
>

Thanks for the additional commentary. I fear my use of canonical hasn't
helped clarity much (blame my co-chair for allowing me to use big words
:-)).

The adopted qlog documents included a TypeScript-based data model (schema)
and a JSON serialization. There have been some discussions about changing
either of these. As you mentioned upthread, the two are associated. In an
attempt to keep discussions focused, we've somewhat avoided mentioning the
schema. The aim of finding consensus on the serialization format was so
that we could use that as input into deciding on schema definition. Picking
JSON for serialization would, hypothetically speaking for now, allow us to
rewrite the schema in CDDL. Using an IETF format for schema definition
would assuage some of the concerns that have been expressed about
Typescript or other options. And I anticipate a schema rewrite would
require the WG tackle some of the thornier issues in good time. Speaking
personally, I think that nailing down JSON as the working format du jour
will allow us to tackle data definition aspects that have been hanging over
the spec for a while.

qlog has an objective to support alternative serializations formats and
robust transformations between them. If I understand your comments
correctly. There's an argument to say we might be approaching things
backwards and that the group should pick a data definition language first,
and then the serialization stuff might just come out in the wash. It's a
bit chicken and egg. Given we have interoperability today between QUIC
implementations generating qlogs in JSON and tools consuming JSON, for many
people the decision about data definition is of less immediate practical
importance. But that's not to downplay the long term importance of creating
a robust data definition.

Speaking as a co-chair, getting clear consensus about the topic of schema
and serialization is important at this stage of the document's lifecycle. I
want to make sure we're asking the right questions now, and we understand
the potential impacts of things we might be trying to defer. So thanks for
picking at this. Does the intent to revist (soon) the qlog schema and
(possibly) extract the serialization address some of your concerns? Is
there a clearer way we could articulate the question to ensure the WG
understands what it is agreeing to?

Cheers,
Lucas

---

From: Robin MARX <[email-masked]>
Date: Fri, 6 Aug 2021 16:31:14 +0200

Hello all,

Thank you for the excellent discussion on this and sorry for my silence.

As Lucas has said multiple times, I don't think we should take the current
text too much as direct gospel for this consensus call.
To me, it is indeed much more a question of agreement on going for "a" JSON
option, as opposed to CBOR/protobuf/pcapng/etc. and not "the" specific JSON
serialization currently specified.

Much of the current JSON oddities come from:
     a) problems with JSON implementation limitations in browsers (e.g., no
true 64-bit number support) and
     b) the use of TypeScript to specify the data format (a decision taken
at a time where I was unaware of things like CDDL)
I am certainly not averse to and indeed proposing to actively revisit these
things.

Concretely to address some of Paul's original points:
Section 6.1.1: large numbers as strings or numbers
    This is because browser JavaScript engines typically only do up to
2^53-1 instead of 64 bit. I hope that with BigInt support this will
eventually change and for qlog it is probably sufficient to just use
numbers and note this as a caveat
    ("if you're targeting browser-based tooling, large numbers might be
truncated. Shouldn't happen in 99.9% of cases, but if you're doing weird
stuff, beware").

6.1.2.1 multiple ways of truncating byte strings
   I am unaware of a canonical JSON structure that would allow doing this.
If there is such a thing, kindly enlighten me.
   If the problem is mainly with the fact that I currently define multiple
options (e.g., with/without raw_length etc.), that's certainly something
I'm willing to revise.

6.1.4 allowing trailing comma's
   I'm a bit confused by this one, since the qlog text explicitly states
that trailing comma's aren't allowed. The intent is to work around this by
requiring -tools- to accept empty objects to make output easier.
   I'm not sure if this type of practical tooling guidance is
necessary/wanted in an actual RFC, but this and other similar guidances
have been quite useful in the early days.

6.2 NDJSON
   As Lucas has indicated, this imo isn't part of the current consensus
call, but the decision of what to do about a streaming format will be
influenced by the choice of main serialization format of course.
   If JSON would be chosen, the current idea is to probably switch to JSON
text sequences (https://datatracker.ietf.org/doc/html/rfc7464, see also
https://github.com/quicwg/qlog/issues/172)

With regards to having a canonical serialization format (JSON) or canonical
data definition (as Lucas said, probably CDDL-based), I personally have a
very large preference for the latter.
As indicated by Lucas, Spencer and Roberto, I fully expect there to be
other serialization formats than JSON emerging down the line if qlog
continues to grow/be used.
As such, the canonical data definition should help make such alternatives
possible/easier down the line.
However, I also see a great benefit in having a single, "default" and
standardized serialization format, even if it's just for the purposes of
having a concrete example (e.g., like how we have New Reno as the
congestion controller for QUIC),
though I think the current usage shows it's also already useful for several
other purposes as well (e.g., conversion from/to, tooling development)
As the "default" serialization that we standardize (first), I feel JSON is
our best option for the reasons outlined during the presentation and in the
related Github issues.

To make all that clearer/easier, it might indeed be interesting to split
the serialization format from the main schema document, but to me that's
something that can be easily done down the line, since it's mostly
editorial work.
For now, I think we'd benefit from keeping things together in the main
schema, especially for new implementers/people updating their qlog support
to new versions down the line.

I think that's mainly echoing what Lucas has been saying, but in a slightly
different way, hoping to make things even clearer.
Please let me know if you still have issues with this proposed direction!

With best regards,
Robin

On Thu, 5 Aug 2021 at 21:39, Lucas Pardue <[email-masked]>
wrote:

>> The call asked whether the (now "a") JSON serialization format will be
>> the canonical interoperable format. That is quite different than "including
>> at least one serialization format". As a concrete example:
>>

-- 

dr. Robin Marx
Postdoc researcher - Web protocols
Expertise centre for Digital Media

*Cellphone *+32(0)497 72 86 94

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 6 Aug 2021 14:59:48 +0000

Slicing it down to a minimum:

On Aug 6, 2021, at 7:31 AM, Robin MARX <robin.marx=[email-masked]> wrote:
> With regards to having a canonical serialization format (JSON) or canonical data definition (as Lucas said, probably CDDL-based), I personally have a very large preference for the latter. 

Excellent! We agree. It is still good to have the eventual JSON serialization format in the documents, because it will be the easiest to grok for those of us who are "learn by example".

If this is the way the WG goes, I volunteer to do a CBOR serialization. If the WG loves it so much that it also puts it into the documents, great, but if not, it can easily be a standalone document that developed in parallel with the data-definition-plus-JSON document. I also volunteer to help with the JSON, if needed (I was previoulsy co-chair if the JSON WG).

--Paul Hoffman

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 12 Aug 2021 23:12:58 +0100

Hello QUIC WG,

> [1] - https://github.com/quicwg/qlog/issues/144

Thanks to those that responded on the list.

Seeing no pushback against keeping JSON, consensus is declared.

The chairs and editors have noted the additional worthwhile discussion and
suggestions for improvement areas in the document. These will be handled as
follow up work, separate to this call.

Kind regards
Matt & Lucas
QUIC WG Chairs