Subject: QUIC logging and tooling : moving towards qlog

From: Robin MARX <[email-masked]>
Date: Tue, 16 Apr 2019 12:50:53 +0200

Hello everyone,

In Prague, there were three separate presentations focusing on logging and
tooling for QUIC and HTTP/3 (video:
https://www.youtube.com/watch?v=LiNLz1QuT0s&feature=youtu.be&t=3190,
slides: https://datatracker.ietf.org/meeting/104/session/tsvarea).

There seemed to be a lot of interest in moving this forward. The currently
agreed upon approach is to focus on creating a single logging format and
schema. This will make the creation of reusable tools easier. As a basis
for this, the qlog proposal was chosen.

Per recommendation of Brian Trammell, we have created two I-Ds for this:
1) The high-level schema: this is (semi-) protocol agnostic and could
eventually lead to a BOF for a more general wg on this topic. For now, we
focus on the QUIC/H3 use case.
2) The concrete QUIC and HTTP/3 event definitions: event semantics for
these protocols.

These documents and examples can be found at:
https://github.com/quiclog/internet-drafts

We welcome new collaborators and co-authors for these drafts.
There are also several issues in the Github repo that could use some
insightful comments.
In the quicdev slack, there is also a #qlog channel.

---------------------------------------------------------------------------------------------------------------

Even though qlog will be the basis for the "standard" format, several
implementers have started logging in the quic-trace format as well (
https://github.com/google/quic-trace).

To help the transition and show that qlog is/should become general enough
to support the use cases from other formats, we have written a small
quic-trace to qlog converter webpage:

https://quic.edm.uhasselt.be/qtr-to-qlog/

This tool ingests the binary .qtr quic-trace protobuf files and converts
them to json-based qlog.
It also includes a rudimentary version of the quic-trace
visualization/tool, as a simple example of how to ingest and process qlog
in web-based tools.

In time, Victor Vasiliev also plans to add qlog support to the quic-trace
tool. Jari and Marcus of the spindump tool also plan to add qlog support,
as will Maxime Piraux for quic-tracker. Finally, Dmitri and Stephen from
Litespeed already integrated basic qlog support in lsquic, which can be
seen live at https://http3check.net/. We doubt Wireshark will add qlog
output directly, but we also wrote a pcap-to-qlog converter for an early
version of qlog.

We expect most implementers will not log qlog/json directly, but write
automated transformers from their (binary) formats into qlog. The
qtr-to-qlog and pcap-to-qlog examples shows that this is a perfectly valid
approach as well.

--------------------------------------------------------------------------------------------------------------------

At this moment in time, the original qlog tools (see
https://quic.edm.uhasselt.be) are not yet compatible with the new qlog
format. This is something we will be working on in the next few weeks.

We welcome any and all feedback and ideas for tools and logging amenities.

With best regards,
Robin

-- 

Robin Marx
PhD researcher - web performance
Expertise centre for Digital Media

T +32(0)11 26 84 79 - GSM +32(0)497 72 86 94

www.uhasselt.be
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Tue, 16 Apr 2019 20:02:07 +0000

I haven’t studied qlog yet, but if you settle on a protobuf derivef JSON format, it would be good to make it FlatBuffers compatible as well. I’d be happy to help with that.

________________________________
Fra: QUIC <[email-masked]> på vegne af Robin MARX <[email-masked]>
Sendt: tirsdag, april 16, 2019 12:51 PM
Til: IETF QUIC WG
Emne: QUIC logging and tooling : moving towards qlog

Hello everyone,

In Prague, there were three separate presentations focusing on logging and tooling for QUIC and HTTP/3 (video: https://www..youtube.com/watch?v=LiNLz1QuT0s&feature=youtu.be&t=3190<https://www.youtube..com/watch?v=LiNLz1QuT0s&feature=youtu.be&t=3190>, slides: https://datatracker.ietf.org/meeting/104/session/tsvarea).

There seemed to be a lot of interest in moving this forward. The currently agreed upon approach is to focus on creating a single logging format and schema. This will make the creation of reusable tools easier. As a basis for this, the qlog proposal was chosen.

Per recommendation of Brian Trammell, we have created two I-Ds for this:
1) The high-level schema: this is (semi-) protocol agnostic and could eventually lead to a BOF for a more general wg on this topic. For now, we focus on the QUIC/H3 use case.
2) The concrete QUIC and HTTP/3 event definitions: event semantics for these protocols.

These documents and examples can be found at: https://github.com/quiclog/internet-drafts

We welcome new collaborators and co-authors for these drafts.
There are also several issues in the Github repo that could use some insightful comments.
In the quicdev slack, there is also a #qlog channel.

Even though qlog will be the basis for the "standard" format, several implementers have started logging in the quic-trace format as well (https://github.com/google/quic-trace).

To help the transition and show that qlog is/should become general enough to support the use cases from other formats, we have written a small quic-trace to qlog converter webpage:

This tool ingests the binary .qtr quic-trace protobuf files and converts them to json-based qlog.
It also includes a rudimentary version of the quic-trace visualization/tool, as a simple example of how to ingest and process qlog in web-based tools.

In time, Victor Vasiliev also plans to add qlog support to the quic-trace tool. Jari and Marcus of the spindump tool also plan to add qlog support, as will Maxime Piraux for quic-tracker. Finally, Dmitri and Stephen from Litespeed already integrated basic qlog support in lsquic, which can be seen live at https://http3check.net/. We doubt Wireshark will add qlog output directly, but we also wrote a pcap-to-qlog converter for an early version of qlog.

We expect most implementers will not log qlog/json directly, but write automated transformers from their (binary) formats into qlog. The qtr-to-qlog and pcap-to-qlog examples shows that this is a perfectly valid approach as well.

At this moment in time, the original qlog tools (see https://quic.edm.uhasselt.be) are not yet compatible with the new qlog format. This is something we will be working on in the next few weeks.

With best regards,
Robin

--

Robin Marx

www.uhasselt.be<http://www.uhasselt.be>
Universiteit Hasselt - Campus Diepenbeek
Agoralaan Gebouw D - B-3590 Diepenbeek
Kantoor EDM-2.05

[https://www.uhasselt.be/images/nieuwsbrief/dcm/UHasselt-liggend.jpg]

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 18 Apr 2019 08:47:59 +0900

Hi,

Thank you for your efforts.

quic-trace has already become an indispensable tool for us when testing
quicly. I am excited to see a common logging format emerging; it would help
us use other tools, and more generally, foster the development of tools to
analyze what is going on, that'd help the QUIC stacks mature.

One comment regarding how the frames are being logged.

I'm not sure what the `class` definitions in the event logs are, but I
assume that they represent particular form of JSON objects.

I think it might be worth considering to log the frames of QUIC as they are
(either in binary or hex-encoded binary), with the exception that the
Stream Data field of STREAM frames would be omitted.

I suggest this because in my view, the binary encoding described in the
transport document is what we are already standardizing; I'd prefer using
that for logging rather than having a separate schema just for logging.

If we are concerned about the introduction of QUIC extensions breaking the
analysis tools, the QUIC stacks can log each QUIC frame as a different
hex-encoded JSON string. That'd allow the analysis tools to ignore unknown
frame types.

WDYT?

2019年4月16日(火) 19:51 Robin MARX <[email-masked]>:

> https://www..youtube.com/watch?v=LiNLz1QuT0s&feature=youtu.be&t=3190
> <https://www.youtube..com/watch?v=LiNLz1QuT0s&feature=youtu.be&t=3190>,

-- 
Kazuho Oku

---

From: Lucas Pardue <[email-masked]>
Date: Thu, 18 Apr 2019 01:28:32 +0100

Hi Kazuho,

Responding to some points inline. This is just my opinion so I stand to be
corrected by Robin.

On Thu, 18 Apr 2019, 00:48 Kazuho Oku, <[email-masked]> wrote:

>
> I think it might be worth considering to log the frames of QUIC as they
> are (either in binary or hex-encoded binary), with the exception that the

I can see the appeal of that. However, part of the attraction of logging is
that it can give further insight into applications. I had in mind logging
in the HTTP/3, which in my implementation is a few steps removed from QUIC
frames or packets. More so, much of the time my logging only cares for
HTTP/3 frame payload, not complete frames.

But perhaps the intent of your suggestion is to compliment the other logs,
not replace them?

> that for logging rather than having a separate schema just for logging.

As I understood it, logging adds detail not present in wire image. And it
does so in a way that could be compatible with general purpose datastore,
analytics etc.

> hex-encoded JSON string. That'd allow the analysis tools to ignore unknown

I think the current intention is to place unknown data handling burden on
tools. For example, a tool focused on congestion control analysis probably
has no interest in HTTP/3 fine details.

Lucas

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 18 Apr 2019 09:55:22 +0900

2019年4月18日(木) 9:28 Lucas Pardue <[email-masked]>:
>
> Hi Kazuho,
>
> Responding to some points inline. This is just my opinion so I stand to be corrected by Robin.
>
>> I think it might be worth considering to log the frames of QUIC as they are (either in binary or hex-encoded binary), with the exception that the Stream Data field of STREAM frames would be omitted.
>
>
> I can see the appeal of that. However, part of the attraction of logging is that it can give further insight into applications. I had in mind logging in the HTTP/3, which in my implementation is a few steps removed from QUIC frames or packets. More so, much of the time my logging only cares for HTTP/3 frame payload, not complete frames.

I think I might have caused confusion due to specifically talking
about QUIC frames, but I think the same argument applies for H3 as
well.

I think we can log the HTTP/3 frames as they are in binary or as a
hex-encoded binary (with the exception that we'd omit the payload of
DATA frames (and possibly also HEADERS frame)) in the log format.

> But perhaps the intent of your suggestion is to compliment the other logs, not replace them?

My point is that it would be easier to use the wire-format defined in
QUIC or HTTP/3 when logging what is being transmitted.

We need to log other sorts of information (e.g., packet loss timer
firing), so using the wire-format does not help everything. But at
least, it would remove the requirement on QUIC stacks to support two
ways of serializing what is transmitted (the two way being the QUIC
wire format and the log format).

>> I suggest this because in my view, the binary encoding described in the transport document is what we are already standardizing; I'd prefer using that for logging rather than having a separate schema just for logging.
>
>
> As I understood it, logging adds detail not present in wire image.

Yes, logging information only known to the endpoints do help (even
though traditional tools (e.g., wireshark) don't). I think it's
beneficial to define a format that can log not only what's being sent
/ received but also what's happening within the endpoint.

> And it does so in a way that could be compatible with general purpose datastore, analytics etc.

What I am suggesting is to log each frame (either QUIC frame or H3
frame) as binary or binary encoded as hex string. I think it would be
compatible with general purpose datastore. For QUIC, I'd assume that
we'd generally be building tools specifically for the protocol, so I
think using the wire-format does not make things complicated on the
analytics side (it's just about dealing with binary-encoded
information rather than JSON).

OTOH, for HTTP/3, if your intent is to use the same logging format for
HTTP1, 2, 3, then I can see your point on using a wire-independent
logging format.

Though, even if that is the case for HTTP-level logging, I'd still
argue that it would be worthwhile to allow QUIC / H3 stacks to log the
QUIC or the H3 frames using the wire-format. There's no reason to
require each stack to implement the logging format. Allowing the
stacks to log using the wire-format would lower the entry bar of the
stacks supporting the analysis tools.

>
>>
>> If we are concerned about the introduction of QUIC extensions breaking the analysis tools, the QUIC stacks can log each QUIC frame as a different hex-encoded JSON string. That'd allow the analysis tools to ignore unknown frame types.
>
>
> I think the current intention is to place unknown data handling burden on tools. For example, a tool focused on congestion control analysis probably has no interest in HTTP/3 fine details.
>
> Lucas
>>
>>

-- 
Kazuho Oku

---

From: Christian Huitema <[email-masked]>
Date: Wed, 17 Apr 2019 18:19:26 -0700

> stacks supporting the analysis tools.

I am a bit concerned that the proposed logging formats are rather
verbose, using text encodings and lots of detailed events. That useful
when debugging basic interop issues, but we are somewhat past that. As
QUIC is maturing, I expect to see a much greater emphasis on debugging
performance issues: software performance of course, but also performance
of recovery and congestion control. You need good traces to debug the
recovery and congestion issues, but you lose if turning on tracing slows
down the system so much that the behavior under traces does not reflect
normal behavior. And I am concerned that using verbose formats will do
just that.

-- Christian Huitema

---

From: Robin MARX <[email-masked]>
Date: Thu, 18 Apr 2019 12:39:57 +0200

Hello everyone,

Thank you for the thoughtful comments.
I think I can answer most of the remarks made with one statement: you can
log whatever you want in whatever format, and then write a transformer into
the qlog format.

A better way to describe qlog is probably "*a common input data format for
tools*" rather than a standard *logging* format.
However, you can certainly use qlog as a basic logging format as well,
which is exactly what quicker and (partially) lsquic are currently doing.
When you need higher performance or don't want to write two serializers in
the implementation etc., you can log in other ways and then transform to
qlog for ingestion in the tools.
This is usually relatively trivial and fast, e.g., see the quictrace2qlog
and pcap2qlog converters we've already started (and what Jana is also doing
for quicly iiuc),
As you say, I don't think it's realistic to get every implementation to
agree on how exactly they will log, whether it's a verbose .json or
optimized flatbuffer setup.

A few more in-depth comments:

1) I do not want tools to have to parse the binary wire image themselves
    - I envision many tool developers/maintainers will be students, interns
or other non-QUIC experts. Having them write/extend a parser is possibly
not the best idea.
    - The binary formats are sometimes quite non-intuitive (e.g., ACK
frame) which can lead to mistakes in interpretation.
    - Each tool platform/language (e.g., web, C++, python, ...) would need
its own frame parsing library (for JSON, that's usually readily available)
    - Parsing the binary in for example web-based tools (which I envision
will be the largest group, as they allow ease of use and sharing), will
lead to performance problems there.
    - As Lucas said, ideally the logs would go much further than just the
wire image. I'm more interested in when and how exactly an ACK is
processed, rather than just what came in on the wire.
          Taking that into consideration, the logs would contain two
formats, one for binary events, one for all the rest, again adding to
complexity.

    - That being said, the original qlog proposal had the concept of also
logging raw values, and it's certainly possible to re-incorporate that as
an optional extra field (see issue 11
<https://github.com/quiclog/internet-drafts/issues/11>).
        e.g., you could just log the binary hex-string from the
implementation, and then write a post-processing converter that parses the
binary values and transforms them into the expected qlog fields.

2) The verbosity is indeed a potential problem, but it's manageable
   - As stated at the beginning, for high-performance tracing, you could
only log the select events you want or use a custom format and post-process
it into qlog for tooling
   - Even within the JSON setup, there are many ways to reduce the
file-size and logging overhead (e.g., the technique used by the Netlog
format, also discussable in issue 8
<https://github.com/quiclog/internet-drafts/issues/8>)
        Logging individual events should not take more overhead than normal
textual logging that is now also done to stdout
   - I agree that current (v1) stacks are going beyond basic interop, but
this is meant to be more general purpose and long-term
          Think of teaching new people, implementing new features (v2,
other app-layer besides H3), major performance-related refactoring, etc.
          For those use cases, a highly optimized format is in my opinion
more of a hindrance than a more verbose format is to congestion control /
performance evaluation
   - Just like with normal logging, qlog has the concept of log levels
(e.g., Verbose vs Error-only), but then for categories. e.g., only
debugging HTTP/3 right now, not logging any of the QUIC stuff.

Smaller remarks:
- The "class" fields are just because we are using TypeScript semantics (see
issue 5 <https://github.com/quiclog/internet-drafts/issues/5>). They do not
mean that you cannot simply drop unknown frames.
     In fact, one of the key tenets of qlog is that you can add as many
custom events as you want for easy extensibility.
- Tools would be required to clearly list the types of events they look for
and the data they expect in the log.
     You can then see for yourself what it takes to be able to use that
tool and only log (or transform to qlog) that exact info.

I hope this helps resolve some of the questions.
The current proposal is the one I feel strikes the best balance between
performance and usability, flexibility and pragmatism.

Feel free to disagree with my points above and to go more in-depth on the
github issues.

With best regards,
Robin

-- 

Robin Marx

www.uhasselt.be

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Thu, 18 Apr 2019 03:48:53 -0700

It doesn’t really reflect on my earlier suggestion on making JSON
flatbuffer compatible.

The thing is that thare are parsers from JSON to protobuf and from JSON to
flatbuffers where have you do not have to do anything other than writing
some driver logic to get the conversion.

A protobuf schema will often have a JSON format that is very similar to
what is accepted by similar schema in FlatBuffers, so it suffice to define
a protobuf schema and a FlatBufffers schema, and all conversion can happen
automatically via JSON.

However, if you rely on specific features such as Any or OneOf, it might
not work, and if you use Base64 to encode strings instead of binary arrays,
it will not work. But, it is generally not difficult to specify schema
where this is not a problem.

Googles Flatbuffer tool flatc convert json to binary given a schema.
FlatBuffers for C (flatcc) generates JSON C parsers from a schema which can
process JSON very fast into FlatBuffers (I’m the author).

It is unlikely that you will see someone handcoding a JSON parser to
convert it into FlatBuffers.

Mikkel

On 18 April 2019 at 12.40.34, Robin MARX ([email-masked]) wrote:

Hello everyone,

Smaller remarks:
- The "class" fields are just because we are using TypeScript semantics (see
issue 5 <https://github.com/quiclog/internet-drafts/issues/5>).. They do
not mean that you cannot simply drop unknown frames.
     In fact, one of the key tenets of qlog is that you can add as many
custom events as you want for easy extensibility.
- Tools would be required to clearly list the types of events they look for
and the data they expect in the log.
     You can then see for yourself what it takes to be able to use that
tool and only log (or transform to qlog) that exact info.

With best regards,
Robin

--

Robin Marx

www.uhasselt.be

---

From: Robin MARX <[email-masked]>
Date: Thu, 18 Apr 2019 16:09:20 +0200

Hello Mikkel,

Sorry for not addressing your first reply.
At this point, I was not really considering making the format
protobuf-compatible.
I know that you can trivially transform from a binary to a json-based
protobuf representation (that's what I'm doing in the quictrace2qlog tool
<https://quic.edm.uhasselt.be/qtr-to-qlog/>)

I don't know all that much about protobuf, but from what I've learned from
the quictrace format and its default json output, it seems like that would
end up -a lot- more verbose than the current qlog proposal.
E.g., quic-trace of 823KB to json clocks in at 14MB, due to a lot of
repeated fields (see also issue 4
<https://github.com/quiclog/internet-drafts/issues/4>).

I'm not sure if protobuf supports the tricks we apply to bring down
filesize (i.e., csv-alike having "column headers" that say what's in each
column so not each entry needs those field names repeated).
This approach is described in the current draft in section 3.3.4
<https://quiclog.github.io/internet-drafts/draft-marx-quic-logging-main-schema.html#commonfields-and-eventfields>.
(see also the common_fields trick with group_ids and issue 8
<https://github.com/quiclog/internet-drafts/issues/8>).

If that type of approach is possible in protobuf/flatbuffer, it would
certainly be interesting to consider it.
If it's not, the (transformed) .json files would be much larger than what
we have now, which would make it even more difficult to create scalable
web-based tools.
(Counterpoint: loading large json files is certainly possible in the
browser, it's primarily rendering of many small elements that will be the
bottleneck).

An alternative argument you could make is that if you have the binary
protobuf, you do not need to transform to json at all: just load the binary
file using something like protobuf.js
<https://github.com/dcodeIO/ProtoBuf.js>.
However, this would make the logs unreadable to the human eye without using
the tools, which makes it a little less applicable as a general logging
format (e.g., to (partially) replace the textlogs that most implementation
now output in an ad-hoc fashion).
My personal feeling is that I would prefer a readable json file so I don't
always need the tools, but possibly this is a trade-off that can be made
(see .pcap).

So some input would be welcome on:
- Usefulness of human readability
- Willingness to output protobuf/flatbuffer directly from implementations
(if you have to transform from whatever your app is doing into protobuf as
post-process anyway, you might as well transform to JSON imo)

With best regards,
Robin

On Thu, 18 Apr 2019 at 12:48, Mikkel Fahnøe Jørgensen <[email-masked]>
wrote:

> A better way to describe qlog is probably "*a common input data format
> for tools*" rather than a standard *logging* format.
>     - I envision many tool developers/maintainers will be students,
> interns or other non-QUIC experts. Having them write/extend a parser is
> possibly not the best idea.
>         Logging individual events should not take more overhead than
> normal textual logging that is now also done to stdout
> - Tools would be required to clearly list the types of events they look
> for and the data they expect in the log.

-- 

Robin Marx

www.uhasselt.be

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Thu, 18 Apr 2019 07:38:47 -0700

My impression was that you already used protobuf and it seems that you do,
but perhaps not as prominent as I thought but for internal use.

For humans, obviously JSON is the way to go. For machines it is less clear.

I am not an expert on protobuf although I do know about the encoding format.
I do know a lot about FlatBuffers.

Protobuf provides are rather compact binary format more than JSON and
FlatBuffers, and possibly more compact than CSV.
You do not have to store fields that have default values. Protobuf does not
have implicit meaning like CSV fields. Each value is preceeded by a
tag/type value, but that can is usually quite compact, and unused fields
can be entirely omitted.

However, protobuf is not directly accessible, it must be parsed into a host
language structure. Depending on implementation this comes at a certain
overhead.

FlatBuffers is not nearly as compact - it can be both smaller and larger
than JSON. All field names are implicit but there are pointers (offsets)
that take up space.

FlatBuffers does not store fields that have default values. FlatBuffers can
be navigated directly without any allocation, for example from a memory
mapped file. I do this on fairly large data sets with latency critical
lookups, and it is very fast, although here, speed may not be a primary
concern here.

JSON is somewhat like protobuf in that it must be parsed and cannot be
accessed directly. JSON parsers generally need to allocate a lot of memory,
like between 3 and 10 times the file size, I would guess.

Parsing JSON to FlatBuffers is interesting because JSON needs an in memory
representation and this representation can be FlatBuffers. Thus, if the
JSON format corresponds to a FlatBuffers schema, parsing can be extremely
fast. Much faster than you can normally parse protobuf into a custom
in-memory structure because you avoid a lot of allocations - if you used
the code generated for C, and otherwise, it is likely still quite fast.

With a bit of care, the JSON format can be designed to work with both
protobuf and FlatBuffers.

JSON fields that have default values are normally omitted in a FlatBuffers
compatible configuration. This can save a lot of space, but for humans it
might in some cases be confusing. You can force values to appear.

FlatBuffers is not as widely supported as protobuf, but the list is growing
with Javascript, Python, C/C++ Java as some of the most well known.

If you find that another JSON format is much better and have good reasons
for doing that, that is perfectly fine. What I am trying to argue is that
if you have a format that is almost compatible with FlatBuffers and
ProtoBuf, it would be sad not to go all the way and make it work.

Both FlatBuffers, ProtoBuf and JSON are to an extend robust towards changes
in the schema supporting backwards and forward schema evolution. However, a
hand written JSON parser might not be equally robust towards changes.

On 18 April 2019 at 16.09.33, Robin MARX ([email-masked]) wrote:

Hello Mikkel,

With best regards,
Robin

--

Robin Marx

www.uhasselt.be

---

From: Dmitri Tikhonov <[email-masked]>
Date: Thu, 18 Apr 2019 11:03:55 -0400

> hand written JSON parser might not be equally robust towards changes.

There are several reasons for sticking with JSON:

    - Robin and his crew have written much code that uses JSON.

    - There are probably a couple of orders of magnitude more
      JSON implementations than proto bufs and flatbuffers
      implementations combined.

    - JSON is human readable.

    - JSON is greppable.  It can be processed by command-line
      tools without having to spend half an hour reading docs
      and installing special software.  (This is actually a pretty
      high barrier to entry: most people won't go that far unless
      they _really_ have to.)

    - JSON has a standard MIME type.  For example, Firefox makes
      responses of type application/json look pretty [1].

I propose we stop arguing about this and agree that JSON, while not
perfect, is a pretty good choice with which we should stick.

  - Dmitri.

1. https://www.http3check.net/?host=www.http3check.net&format=qlog

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Thu, 18 Apr 2019 08:11:45 -0700

You are missing the point.

JSON can be the format of choice. The point is if the JSON format should be
compatible with binary protocol parsers.

On 18 April 2019 at 17.04.03, Dmitri Tikhonov ([email-masked])
wrote:

- Dmitri.

---

From: Dmitri Tikhonov <[email-masked]>
Date: Thu, 18 Apr 2019 11:27:36 -0400

> compatible with binary protocol parsers.

Hmm.  I suppose this was your point, then:

    " With a bit of care, the JSON format can be designed to work
    " with both protobuf and FlatBuffers.

A "free" performance boost is good.  On the other hand, those who
want to consume qlog information as a protobuf can just write a
converter, no?

I am not familiar with protobuf or flat buffers and so I cannot
judge the magnitude of "bit" in the "bit of care."  Still, at this
stage, optimizing seems premature.

  - Dmitri.

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Thu, 18 Apr 2019 09:04:52 -0700

Ideally the JSON format is designed such that there exists a compatible
protobuf schema and a compatible FlatBuffers schema.
If this is the case, there ought to be tooling to convert JSON to protobuf
or to FlatBuffers with very little effort.

If the JSON format is not designed with this in mind, it will be a lot more
work to design a converter and you might not be able to get an exact
representation and each language might end up with its own parser and own
interpretation.

On 18 April 2019 at 17.27.40, Dmitri Tikhonov ([email-masked])
wrote:

- Dmitri.

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 19 Apr 2019 09:17:04 +0900

Hi Robin,

Thank you for your responses.

I can see that requiring every analysis tool to implement their own frame
decoder is counterproductive. It's mirror image of my comment stating that
requiring every QUIC stack to implement their own qlog encoders is
counterproductive.

And as a solution, I think I agree that defining both is a good idea. Thank
you for opening issue #11.

It would be a win-win situation if all the QUIC / H3 stacks and analysis
tools could rely on a common converter that translates the binary frame
format to a JSON-based one.

2019年4月18日(木) 19:40 Robin MARX <[email-masked]>:

> issue 5 <https://github.com/quiclog/internet-drafts/issues/5>). They do

-- 
Kazuho Oku