Subject: [Cbor] cddl validation - tracking errors

From: Jim Hague <[email-masked]>
Date: Tue, 20 Dec 2016 16:16:44 +0000

Hi,

I'm working on a draft 
(https://tools.ietf.org/html/draft-dickinson-dnsop-dns-capture-format-00) 
which describes a CBOR based file format for recording DNS traffic. The 
draft includes a CDDL definition. I'm also working on an implementation.

I'm using the 'cddl' tool to validate that the files I'm outputting 
match the CDDL in the draft.

First, a minor thing. It would make including an output-matches-CDDL 
test in the regular test suite easier if 'cddl' exited 1 if validation 
fails. Currently it only seems to exit 1 if the CDDL is invalid.

Second, and rather bigger. Does anyone have any tips for how to track 
down where a validation actually fails? The file format in question 
isn't simple, and error messages therefore become:

CDDL validation failure (nil for [
<500+ lines>
, 2=>44878, 3=>2329, 4=>2, 5=>64, 6=>75, 7=>1602, 11=>67, 8=>499, 9=>{}, 
10=>{2=>3, 3=>24}}], 4=>[]}]]):
[32769,
  [:prim, 3],
  "[32769, [:prim, 3], \"occur not reached in array [\\\"C-DNS\\\",
<500+ lines>
1=>67, 8=>499, 9=>{}, 10=>{2=>3, 3=>24}}], 4=>[]}]] for [:array, 
[:member, 1, 1, [:text, \"file-type-id\"], [:prim, 3]], [:member, 1, 1, 
[:text, \"file-preamble\"],
<60 lines>
  [:prim, 0]], [:member, 1, 1, [:int, 2], [:prim, 0]], [:member, 1, 1, 
[:int, 3], [:prim, 0]]]]]]]]]]]"]

which basically seems to be telling me there's a problem somewhere in or 
below the array that contains the entire format.

Thanks.
-- 
Jim Hague - [email-masked]          Never trust a computer you can't lift.

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 21 Dec 2016 12:38:58 +0100

On 20 Dec 2016, at 17:16, Jim Hague <[email-masked]> wrote:
> 
> Hi,
> 
> I'm working on a draft (https://tools.ietf.org/html/draft-dickinson-dnsop-dns-capture-format-00) which describes a CBOR based file format for recording DNS traffic. The draft includes a CDDL definition. I'm also working on an implementation.
> 
> I'm using the 'cddl' tool to validate that the files I'm outputting match the CDDL in the draft.

Great to hear!

> First, a minor thing. It would make including an output-matches-CDDL test in the regular test suite easier if 'cddl' exited 1 if validation fails. Currently it only seems to exit 1 if the CDDL is invalid.

Good point.  Fixed in 0.8.4.

(Jim had earlier alerted me to this before, but it never made in on the list…).

> Second, and rather bigger. Does anyone have any tips for how to track down where a validation actually fails?

As you have noticed, error messages are not something that the current tool does very well.

> The file format in question isn't simple, and error messages therefore become:
> 
> , 2=>44878, 3=>2329, 4=>2, 5=>64, 6=>75, 7=>1602, 11=>67, 8=>499, 9=>{}, 10=>{2=>3, 3=>24}}], 4=>[]}]]):
> [32769,
> [:prim, 3],
> 1=>67, 8=>499, 9=>{}, 10=>{2=>3, 3=>24}}], 4=>[]}]] for [:array, [:member, 1, 1, [:text, \"file-type-id\"], [:prim, 3]], [:member, 1, 1, [:text, \"file-preamble\"],
> <60 lines>
> [:prim, 0]], [:member, 1, 1, [:int, 2], [:prim, 0]], [:member, 1, 1, [:int, 3], [:prim, 0]]]]]]]]]]]"]
> 
> which basically seems to be telling me there's a problem somewhere in or below the array that contains the entire format.

Actually it also tells you that an occurrence indicator wasn’t reached.

There also was an attempt to match up 32769 with a text string ([:prim, 3] — primitive with major type 3).

Getting better error output is definitely an objective for further development of the tool.
Since there is some backtracking going on, it is not always obvious what exactly triggered the validation failure.  The current tool is a bit like Kernighan’s car.  It may be useful to extend CDDL by a few mechanisms that would cut back on the backtracking to make this easier.

If you have a particularly hard nut to crack, please do send it (CDDL file, CBOR instance) to me so I can develop a little set of examples from which I can improve the error handling.

Grüße, Carsten