Subject: [Cose] Encoding sizes

From: "Jim Schaad" <[email-masked]>
Date: Sun, 5 Apr 2015 19:25:27 -0700

I have been trying to look at a number of different encoding types for a
message to figure if there is any reason to choose something other than
CBOR.

What I did was not a very deep look at the different formats,  I did just
enough to encode one message (a signature on content) and got some idea of
what the structures looked like and could make some guesses about what the
difference in sizes would be on things going forward.

JOSE-C

I encoded a message that in CBOR takes 237 bytes and in JOSE-B it took 250
bytes.  I did not do any swapping of strings to integers in either case
here.  The major difference in lengths appears to be the fact that most of
the strings have tag and length in 2 bytes for JOSE-B but only in one byte
for CBOR.

I did not do the string folding that is available in JOSE-C because it would
not give any change in the size.  None of the strings occur more than once
so unless there was a pre-defined map of values to strings there would be no
shrinking.   This would require the ability to both add items to the map and
declare which map is used.  This is done in the CBOR by using a pre-defined
mapping which is contained in the IANA registry.  There will be potential
problems for the CBOR encoding if new items are added later.   Even if this
is done I believe that the CBOR encoding will be smaller as the integers are
encoded in one byte for CBOR and two bytes for JOSE-C

BSON

I started to encode and quickly stopped as the fact that many fields are
prefixed with a 32-bit integer which contains the length means that the
message size is going to be much larger.  There does not appear to be any
way to switch from strings to integers for the keys.  However, it would be
possible to use single character strings which would be encoded in two bytes
for the element name (character followed by zero terminator).

Smile

I found the Smile description to be mostly unreadable.  It is a tighter
version of BSON and encoded my message in 249 bytes.  There is a 3 byte
prefix that could probably be dropped.  It was unclear if one could do an
integer for the element name using the standard decoder or not.  Strings are
encoded in a similar method to CBOR, but using more tags so that even more
string can be encoded with a one character tag.  From that point it will
produce smaller items.   Smile does not have the same nicety of CBOR of
having a similar structure for all items so the code size of the
encoder/decoder is probably larger.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 06 Apr 2015 14:32:04 +0200

> BSON

The domain of application for BSON is a database where you want to do
in-place modification of some data (e.g., increase a counter).  Hence,
BSON does not employ variable-length encoding as much as other formats.
(See also Appendix E.3 of RFC 7049.)

I think the main reason why it is sometimes cited where a "binary JSON"
as an interchange format is needed is that it has the cool name.

> Smile

Smile is really baroque.  (I had the same problems extracting
information from the specification, which is why it's not in Appendix E.)

More information about formats can be found in other sections of Appendix E.

By the way, there are various implementations of compression of
duplicate strings or items for CBOR.  E.g., see tag 25 [1, 2].
These schemes are most useful if you are using a JSON-like
representation for a table.  They are less useful for many types of
communication seen by constrained nodes.  More powerful nodes can also
get many of the advantages (and some more) from applying a lean
compression scheme such as LZ4 to the encoded CBOR.

It would be trivial to define something like JSON-C's hash for a string
map as a tag for CBOR.  So far, nobody has wanted to.  I think this is
because the scheme has all the usual problems of referencing external
data that the recipient may or may not have available to it (may need a
way to retrieve, and cache, previously unknown values; standalone data
lose their usefulness; ...).

Grüße, Carsten

[1] https://www.iana.org/assignments/cbor-tags/cbor-tags.xhtml
[2] http://pod.tst.eu/http://cvs.schmorp.de/CBOR-XS/doc/stringref.pod