Subject: [Cbor] Artart early review of draft-ietf-cbor-file-magic-02

From: Bernard Aboba via Datatracker <[email-masked]>
Date: Tue, 03 Aug 2021 12:26:30 -0700

Reviewer: Bernard Aboba
Review result: Ready with Issues

Review of draft-ietf-cbor-file-magic-02
Reviewer: Bernard Aboba
Result: Ready with Issues

An overall comment. This document seems to propose more than one option in
several cases.  I wonder whether the multiple options will turn out to be
used in practice.  This makes me wonder if a document status of Experimental
might be a better choice (so we could try it out and see what turns out to be
needed), rather than BCP.

Section 2.

   A magic number is ideally a unique fingerprint, present in the first
   4 or 8 bytes of the file, which does not change when the contents
   change, and does not depend upon the length of the file.

[BA] Why have two supported lengths? I realize that they can be distinguished,
but having two potential lengths is likely to lead to one of them being less
widely supported or tested.

Section 3.2

[BA] Why support both CBOR Tag Wrapped and CBOR Tag Sequence?
The logic is explained in Section 4, but I'm not sure I buy it:

   "The use of CBOR Tag Wrapped format is easier to retrofit to an
   existing format with existing and unchangeable on-disk format."

[BA] Overall, it seems more likely to me that CBOR will be used for new file
formats than being retrofitted to new ones (which would be complicated by
backward compatibility issues). Or are there specific cases where a retrofit
is being seriously considered?

NITs:

Section 3.3:

"
   3.  The three byte CBOR byte string containing 0x42_4F_52.  When
       encoded it shows up as "CBOR"

...

   The third part is a constant value 0x43_42_4f_52, "CBOR".  That is,
   the CBOR encoded data item for the three byte sequence 0x42_4f_52
   ("BOR").  This is the data item that is tagged.
"

[BA] The latter text is more clear than the former, since 0x42_4F_52
is indeed "BOR". I'd suggest deleting the second sentence in 3, "When
encoded..."

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 4 Aug 2021 21:58:43 +0200

Hi Bernard,

thank you for this thoughtful review.

> On 2021-08-03, at 21:26, Bernard Aboba via Datatracker <[email-masked]> wrote:
> 
> Reviewer: Bernard Aboba

We started out with the intention of having just one way.
It turned out that the flexibility of the CBOR-sequence-based approach is needed, but also the compatibility of supporting implementations that do not support CBOR sequences.
Both will be used in practice, as only the CBOR-sequence-based approach is applicable to file formats encoding a CBOR-sequence, and only the CBOR-data-item-based approach has the compatibility with data-item-only implementations.

> widely supported or tested.

This section provides a quick look at the requirements, or maybe at potential goals that we could set ourselves.
Applications such as file(1) that interpret magic numbers actually have significantly more flexibility than just 4 or 8 bytes.

> is being seriously considered?

Many CBOR-based structures are already defined, usually mainly considering the needs for interchange over protocols such as CoAP and HTTP.
The formats differ in their characteristics (e.g., we have RFC 8949 data items and RFC 8742 CBOR sequences).
But, for all of them, there is a need to store them at some point, if only for diagnostics.
To support both data items and sequences, we best have magic number formats that are appropriate for each.

Most likely, a single application (that is not like file(1)) will simply need to recognize its own file format or a few related ones.

I added "even if the program only
supports a single data item.”, as this was the main argument why the CBOR-sequence-based magic number could not cover all applications.

> is indeed "BOR". I'd suggest deleting the second sentence in 3, "When

Thanks.  I deleted the sentence as suggested and clarified some more of the text.

All the changes I am proposing are in
https://github.com/cbor-wg/cbor-magic-number/pull/8

Grüße, Carsten

---

From: [email-masked] (Dale R. Worley)
Date: Wed, 25 Aug 2021 23:11:39 -0400

There's an odd characteristic about this document, not in its content,
but in its style.  It's written as a proposal (a word which is used
several times), and yet it's intended to be a BCP, which tells people
what do to.  And various passages are written as if the reader is a
co-designer.  For instance,

(Though it would be more accurate to say, "... present at a defined
location in the first few bytes of the file ...".)  If this was a
specification, it would say essentially the same fact as "This scheme
has the desired property of magic numbers of having a unique fingerprint
at a fixed location near the beginning of the data."

I get the sense that the authors are ambivalent whether they are saying
what implementers should do, or just floating an idea.  Though I don't
know if that is accurate.

As far as content goes, I favor the "tag wrapped" version over the "tag
sequence", as it has assured upward-compatibility (if the implementation
follows the requirement to ignore unknown tags).  In particular, if the
data consists of a sequence of CBOR items, each one could be tagged (if
they might differ) or only the first one be tagged (if they don't).  But
that's a different matter.

Dale

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 26 Aug 2021 20:00:20 +0200

On 2021-08-26, at 05:11, Dale R. Worley <[email-masked]> wrote:
> co-designer.  

I can see what you mean.

Typically, specs like this start as a trial balloon, keeping the design space wide open, and encouraging discussion.
Once this discussion happens, it focuses on the normative parts of the text, which then get all the attention.
Once those are consensus, nobody wants to disturb that by changing the introductory parts and the discussion parts.

> For instance,

It seems a major editorial round is in order.

> know if that is accurate.

No, we are quite interested in giving actionable guidelines; the “floating an idea” phase is over.  But the text may indeed not reflect that.

> that's a different matter.

If the data is a CBOR sequence, tagging any one of the items doesn’t tell you much about the others.  So for CBOR sequences, the 55800 approach is the one I’d recommend.  For single-item files, both approaches are open (tag-wrapping and turning the single item into a sequence by prepending a 55800 data item), and it depends on details of the circumstances which one one would select.

Grüße, Carsten

---

From: Michael Richardson <[email-masked]>
Date: Thu, 26 Aug 2021 14:27:38 -0400

Dale R. Worley <[email-masked]> wrote:
    > what do to.

Yes, fair point.
1) It was written as an invididual draft originally, and sought feedback.
   (Request for Comments...)

2) It's a protocol for writing a protocol really.
   Each author that has something they want to mark up are told to think
   about the different parts, and then register something and document it in
   their document.
   So it's more of a framework for CBOR authors to follow.
   That's perhaps why it's not as authoritative as normal.

    > at a fixed location near the beginning of the data."

Good comments.
I can re-edit to be more authoritative if you like.

    > follows the requirement to ignore unknown tags).

I also prefer it, although I implemented the tag sequence in the protocol
that made me write this document.

    > In particular, if the data consists of a sequence of CBOR items, each one could be tagged (if

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide