Subject: Re: [apps-discuss] Gen-ART review of draft-bormann-cbor-04

From: Carsten Bormann <[email-masked]>
Date: Thu, 8 Aug 2013 21:58:17 +0200

On Jul 30, 2013, at 09:05, Martin Thomson <[email-masked]> wrote:

> What would cause this to be tragic, is if publication of this were
> used to prevent other work in this area from subsequently being
> published.

Indeed.

As Paul and I have repeatedly said, CBOR is not trying to be the final, definitive, binary object representation for all purposes.  It was written to some specific objectives, clearly stated in the document.  It is being proposed for standards-track because specific ongoing work that works well with these objectives will benefit greatly from being able to reference a common specification.  If the objectives for other work are different, that work may benefit from using a different format, existing or newly designed.

We had some offline discussion of what can be done to make this more apparent from the document, and decided to start at the most visible part, the abstract.

In -04, the abstract says:

   The Concise Binary Object Representation (CBOR) is a data format
   whose design goals include the possibility of extremely small code
   size, fairly small message size, and extensibility without the need
   for version negotiation.  These design goals make it different from
   earlier binary serializations such as ASN.1 and MessagePack.

Now this seemed fairly clear to the authors, but without context it might be read as trying to be the final, end-all format because it only talks about earlier formats.
We propose to fix this (at the danger of being slightly tautological) by making it clear other formats will be created in the future:

   The Concise Binary Object Representation (CBOR) is a data format
   whose design goals include the possibility of extremely small code
   size, fairly small message size, and extensibility without the need
   for version negotiation.  These design goals make it different from
   earlier binary serializations such as ASN.1 and MessagePack, or other
   binary serializations that may be created in the future.

Serialization/encoding of data structures for transmission continues to be an exciting (if sometimes arcane) subject of computer science and its long history will not end with CBOR.

(CC to [email-masked] because some discussion there appears to be related.)

Grüße, Carsten

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 9 Aug 2013 16:23:49 -0400

On Thu, Aug 8, 2013 at 3:58 PM, Carsten Bormann <[email-masked]> wrote:

> As Paul and I have repeatedly said, CBOR is not trying to be the final,
> definitive, binary object representation for all purposes.  It was written
> to some specific objectives, clearly stated in the document.  It is being
> proposed for standards-track because specific ongoing work that works well
> with these objectives will benefit greatly from being able to reference a
> common specification.  If the objectives for other work are different, that
> work may benefit from using a different format, existing or newly designed.
>

I do not expect you to succeed in being the final definitive format.

But if you are not prepared to try then I don't want the result.

You keep talking about your design requirements, but where are the use
cases that drive them? When I am discussing a specification I am always
referring to illustrative use cases that motivate the requirements. When
you are discussing your spec you wave away every objection by saying it
isn't a design requirement but I have never seen you give the rationale.

Coming back to Martin's comments about the type system. I think that is
what JSON does so well. It has exactly as much type system as is useful and
no more. The developers realized that there was no need to distinguish
lists and sets because both are going to serialize as lists on the wire.

-- 
Website: http://hallambaker.com/

---

From: Tony Finch <[email-masked]>
Date: Tue, 13 Aug 2013 12:14:20 +0100

>    binary serializations that may be created in the future.

I have a small problem with this abstract: the three goals that it picks
out are satisfied by MessagePack better than CBOR, since MessagePack is
simpler so will need even less code. As far as I can tell what makes CBOR
different from MessagePack is support for richer types, especially
strings. Streaming support is useful but it wasn't one of the original
goals.

Regarding PHB's criticisms of complexity: Type tags don't really need to
be part of the serialization format: they can be encoded in a simpler
format by the application. The advantage of doing so would be a more
direct translation into typical dynamic language data structures, and
better interop with JSON. The disadvantage is slightly more verbosity. You
would need a clear tagging convention to replace the syntactic framework.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Forties, Cromarty: East, veering southeast, 4 or 5, occasionally 6 at first.
Rough, becoming slight or moderate. Showers, rain at first. Moderate or good,
occasionally poor at first.

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 13 Aug 2013 16:28:17 +0200

On Aug 13, 2013, at 13:14, Tony Finch <[email-masked]> wrote:

> Type tags don't really need to
> format by the application. 

Yes, and we also can get rid of maps {"a": 1, "b": 2}.

Just represent them as arrays of two-element arrays [["a", 1], ["b", 2]].
Or, actually, as arrays where the even-indexed elements are the keys ["a", 1, "b", 2].
(This is, in the end, close to what happens in the encoder anyway.)

There is a reason why we provide some type tagging (here: distinguish arrays and maps) in the encoding: The decoder may be able to factor out some common handling that otherwise would need to be done on the application.

In other words, removing maps in the name of simplicity just moves the complexity elsewhere, in this case to the application.

Now, the application may already be prepared to handle the complexity, and if you come from a JSON world, you are probably *used* to having it handle the equivalent of CBOR's tags in the application: JSON doesn't have tags.  Supporting applications that do that is a reason why the use of tags is optional.

Grüße, Carsten

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 13 Aug 2013 08:12:17 -0700

On Aug 13, 2013, at 4:14 AM, Tony Finch <[email-masked]> wrote:

> simpler so will need even less code.

MsgPack is not extensible. If it was, we might have been able to avoid all of this.

> As far as I can tell what makes CBOR

It *was* part of the original goals; it wasn't part of the original format. Some people on the AppsAWG list pointed out to us that our format was not meeting its goals by not supporting streaming, so we added it. Yes, this added a bit of complexity for decoders, but we handled that as best we could.

--Paul Hoffman

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 14 Aug 2013 22:46:19 +0200

On Aug 13, 2013, at 13:14, Tony Finch <[email-masked]> wrote:

> MessagePack is simpler so will need even less code

FWIW, earlier today I had a nice afternoon with the msgpack-ruby C code, converting it to encoding and decoding CBOR instead.

Saved ~ 250 lines of C code.

Of course, I'm filling in that gap now by implementing some of the optional tags of CBOR.
In the end, the implementation of a generic CBOR encoder/decoder with a sizable collection of optional tags will probably indeed be slightly larger.
But if you just need the features of MessagePack, I now have one data point where CBOR measurably wins on the implementation complexity metric.

Grüße, Carsten

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 15 Aug 2013 08:41:44 +0100

On Wed, Aug 14, 2013 at 9:46 PM, Carsten Bormann <[email-masked]> wrote:

> FWIW, earlier today I had a nice afternoon with the msgpack-ruby C code,
> converting it to encoding and decoding CBOR instead.
>
> In the end, the implementation of a generic CBOR encoder/decoder with a
> sizable collection of optional tags will probably indeed be slightly larger.
> But if you just need the features of MessagePack, I now have one data
> point where CBOR measurably wins on the implementation complexity metric.
>
> Grüße, Carsten
>
>
How does it compare to JSON-B which is considerably simpler than CBOR?

Getting Proposed Standard this way is not going to count for anything
because your proposal that you named after yourself was never in
competition with anything else. No other competitors were allowed.

And, no, no apologies for complaining about this farce.