Subject: [Cbor] Failure Testcases (was: RE: Simple Value Example)

From: Richter, Jörg <[email-masked]>
Date: Wed, 30 Jan 2019 12:55:19 +0000

> > BTW: Is there a collection of “corrupt” CBOR messages that must produce
> > a decode error?
> 
> We have talked about this:
> 
> https://github.com/cbor-wg/CBORbis/issues/16
> 
> But so far nobody has started collecting instances.
> Do you have some we could start with?
> I was planning to scan some of the open source CBOR packages for tests of
> failing decoding.

Okay. I've been thinking about it for a while.

One set of test cases can be easily constructed from valid test cases. 
Just decode a prefix of a valid test case and the decoder should fail with 
an "end of message" error.  Just generate them during normal testing.

I think a second set of failing test cases can be constructed by just 
considering which CBOR prefixes are invalid. This test cases should of course 
fail with a different error than "end of message".

I wrote and attached a simple Lua script to generate all invalid 1 and 2 byte 
prefixes.

This script outputs 578 test cases.  Also attached.

Another set of test cases can be constructed when using tags.  But I think
this is more involved and not clearly defined.  But I might be wrong.
Consider for example a bigfloat. Is it allowed that the array elements themselves
are using a tag?  

Or what if a tag is followed by an element that is not allowed for this tag.  
Like following a tag for a set (tag=258) with a string instead of an array?  Should this 
cause a decode error?

Or is it allowed that Tag 55799 (0xd9d9f7) is the only content of a CBOR file?

- Jörg

---

From: Joe Hildebrand <[email-masked]>
Date: Wed, 30 Jan 2019 13:08:36 -0700

Fantastic.  I already see a couple of places I need to check from this.  PR to the test-vectors project here:

https://github.com/cbor/test-vectors/pull/5

That's just a quick JSONification of your list.

> On Jan 30, 2019, at 5:55 AM, Richter, Jörg <[email-masked]> wrote:
> 
> <generate.lua><fail.txt>_______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor

---

From: Joe Hildebrand <[email-masked]>
Date: Wed, 30 Jan 2019 13:16:25 -0700

Jörg's test cases pointed out a bullet point that we might want to add to section 4.3.2.

- The Indefinite-Length encoding is used for major type 0, 1, 6, or 7.

-- 
Joe Hildebrand

---

From: Jeffrey Yasskin <[email-masked]>
Date: Wed, 30 Jan 2019 12:35:56 -0800

https://github.com/cbor-wg/CBORbis/pull/17 deletes that (non-normative)
section entirely in favor of saying it at the top of section 3 (
https://github.com/cbor-wg/CBORbis/pull/17/files#diff-1db0e3aacf579921911ded2cd0e6e4f5R433
).

Jeffrey

On Wed, Jan 30, 2019 at 12:16 PM Joe Hildebrand <[email-masked]>
wrote:

> _______________________________________________
> CBOR mailing list
> [email-masked]

---

From: Joe Hildebrand <[email-masked]>
Date: Wed, 30 Jan 2019 13:39:27 -0700

Sorry, just 0, 1, or 6.  MT 7 with Indefinite Encoding is FF or "Break".

> On Jan 30, 2019, at 1:16 PM, Joe Hildebrand <[email-masked]> wrote:
>

---

From: Richter, Jörg <[email-masked]>
Date: Wed, 30 Jan 2019 20:46:32 +0000

You mean the opposite? It is used for 2,3,4 and 5.

________________________________________
Von: Joe Hildebrand <[email-masked]>
Gesendet: Mittwoch, 30. Januar 2019 21:39:27
An: [email-masked]
Betreff: Re: [Cbor] 4.3.2 Malformed Indefinite-Length Items

---

From: Joe Hildebrand <[email-masked]>
Date: Wed, 30 Jan 2019 13:55:55 -0700

Yes.  I meant it is *invalid* for 0, 1, or 6.

> On Jan 30, 2019, at 1:46 PM, Richter, Jörg <[email-masked]> wrote:
>

---

From: Jeffrey Yasskin <[email-masked]>
Date: Wed, 30 Jan 2019 13:05:36 -0800

> Like following a tag for a set (tag=258) with a string instead of an

<https://github.com/cbor-wg/CBORbis/pull/18>
https://github.com/cbor-wg/CBORbis/pull/18 tries to specify which contents
for tags are valid, and https://github.com/cbor-wg/CBORbis/pull/17 says
that generic decoders don't necessarily reject invalid-but-well-formed data.

Or is it allowed that Tag 55799 (0xd9d9f7) is the only content of a CBOR
> file?

No, because any well-formed tag contains another CBOR item.

Jeffrey

---

From: Richter, Jörg <[email-masked]>
Date: Wed, 30 Jan 2019 21:15:50 +0000

I had of course some mistakes in my script. Attached is the fixed 
version with now only 568 test cases.
Every other single- and two-byte message should fail with 
"end of message" or should succeed with decoding.

Note that it is important to differentiate the failures into 
"end of message" and something else. The attached list 
contains only the other failures.

I am fairly sure that most, if not all, possible decoder problems 
for invalid CBOR should be found with this list. CBOR is so simple, 
that all possible problems can be triggered with the first two 
bytes of an element. 
This does not apply to decoders that, for example, run through 
different code paths for map elements than for the root element. 
But a straight forward recursive decoder would use the same code path.

- Jörg