Subject: Re: [Cbor] Simple values 24..31 (was Re: List of not-well-formed CBOR and test vectors)

From: Laurence Lundblade <[email-masked]>
Date: Thu, 1 Aug 2019 10:18:50 -0700

> On Jul 31, 2019, at 11:52 PM, Carsten Bormann <[email-masked]> wrote:
> 
> On Aug 1, 2019, at 05:01, Laurence Lundblade <[email-masked]> wrote:
>> 
>> Why wouldn’t 24 through 255 be considered unassigned and encoded 0xf8 0x18 through 0xf8 0xff?
> 
> Because we considered those values confusing, and decided to give up on 8 potential values in exchange for reducing this confusion.  See Section 2.3 of RFC 7049 (parenthesis in first paragraph on page 13).
> 
> Whether that was a particularly wise decision can be discussed.  Since we already have to check for too-low values in the second byte of 7.24, checking for 32 is not more onerous than checking for 24 (see pseudocode).  Giving up 8 values seems to be of limited concern given the number of allocations of Simple values so far.  Changing this now, however, would be a backward compatibility issue that we need to avoid.
> 
> Grüße, Carsten
> 

I see the logic, though I kind of think the opposite. Carving out a chunk of simple values as never-to-be-used is also confusing. Confused me :-). Another reason it is confusing is that it makes them (just slightly) different from low-value integers.

Probably a lot of decoders today cannot decode these simple values, so allowing them now would break backwards compatibility.

I think they should be labeled different than “reserved” because that implies they are going to be used at a later time or for some special purpose. That’s why I went looking at extension points and IANA. Here’s a few suggestions:
Never to be used
Disallowed
Forbidden

Agreed that implementation cost is insignificant for any choice here.

LL

---

From: Laurence Lundblade <[email-masked]>
Date: Sat, 3 Aug 2019 09:54:31 -0700

I’ve added:
    - The simple values 24-31, the ones marked as “reserved" in 7409 (I’d prefer to call them “permanently disallowed” or such)
    - Half-precision, float and short values that are missing some bytes

Will turn this in to an appendix for CBORbis in about a week. 

Thanks for any comments, additions, confirmations.

LL

> On Jul 29, 2019, at 11:49 AM, Laurence Lundblade <[email-masked]> wrote:
> 
> I think I’ve made a comprehensive list of all things that are not well formed by going through the latest draft and my decoder. There are about a dozen of them. I’ve also created 110 test vectors that cover them pretty thoroughly. 
> 
> Everything is here <https://github.com/laurencelundblade/QCBOR/blob/not_well_formed/test/not_well_formed_cbor.h> in a C header. The dozen types of non-well-formedness are listed as comments in the header file. The test vectors are in an array that can be used for testing. It is BSD-3 license.
> 
> I’ve turned up one bug in the RFC’s pseudo code. It doesn’t catch an indefinite length string as a segment in another indefinite length string. 
> 
> I’d like to get some review, some folks to try it out and such to see if I’ve missed anything and all is right. When that is done I’ll make a pull request for the draft out it. Probably in about two weeks.
> 
> So please look it over and try it against your implementation.
> 
> LL
>

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 3 Aug 2019 19:07:30 +0200

> Will turn this in to an appendix for CBORbis in about a week. 

Do you think the test vectors need to be in the document?
I’d rather leave them in a resource that directly can be used by implementers.

> marked as “reserved” in 7409 (I’d prefer to call them “permanently disallowed”

Well, reserved means “not available for allocation”.  I know that the term has occasionally been misused, but that is not a reason to excise it from all documents now.  These numbers are exactly as reserved as ai=28, 29, 30 are.  Not legal now, but very much available for a future standards effort if there ever is one that extends CBOR beyond its premeditated extension points.

Grüße, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Sat, 3 Aug 2019 22:12:17 +0200

On Aug 3, 2019, at 19:07, Carsten Bormann <[email-masked]> wrote:
> 
> Well, reserved means “not available for allocation”. 

(Forgot to cite chapter and verse:)

https://tools.ietf.org/html/rfc8126#section-6

Grüße, Carsten

---

From: Laurence Lundblade <[email-masked]>
Date: Sat, 3 Aug 2019 17:00:55 -0700

OK. I didn’t realize that “Reserved” was a special key word per 8126. I was going by the dictionary definition. Only super hard-core IETF people will ever look at 8126, so most will get this wrong. 

The 5-bit values 28..30 are listed as reserved in one place and unassigned in another. Probably they should be listed as reserved everywhere (change Table 1).There is no assignment procedure for 28..30 in the IANA section.

LL

> On Aug 3, 2019, at 1:12 PM, Carsten Bormann <[email-masked]> wrote:
> 
> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor

---

From: Laurence Lundblade <[email-masked]>
Date: Sat, 3 Aug 2019 17:02:56 -0700

> On Aug 3, 2019, at 10:07 AM, Carsten Bormann <[email-masked]> wrote:
> 
> On Aug 3, 2019, at 18:54, Laurence Lundblade <[email-masked] <mailto:[email-masked]>> wrote:
>> 

I was thinking both. The RFC is so much more widely available and read than any .h file or such that I think it is worth putting them in. I was thinking an appendix, formatted in a three column table so it doesn’t take up pages.

LL

---

From: Jim Schaad <[email-masked]>
Date: Sat, 3 Aug 2019 19:06:46 -0700

I would agree.  I think it is worthwhile to put the reasoning for the reserve into the document.  Perhaps as part of the DE instructions so that it does not accidently get assigned after you retire and stop doing the work.

Jim

-----Original Message-----
From: CBOR <[email-masked]> On Behalf Of Laurence Lundblade
Sent: Saturday, August 3, 2019 5:01 PM
To: Carsten Bormann <[email-masked]>
Cc: [email-masked]
Subject: Re: [Cbor] List of not-well-formed CBOR and test vectors

LL