Subject: [Cbor] Interoperable CBOR protocol issue?

From: Laurence Lundblade <[email-masked]>
Date: Mon, 16 Jul 2018 11:03:38 -0400

Seems to me that is it possible to have non-interoperable CWT or COSE implementations that are fully compliant with the specs due to the allowed CBOR encoding variations. A CWT sender could only send indefinite lengths. A CWT receiver is not required to support indefinite lengths.  Similar for integer encoding. 

This seems true for every CBOR protocol unless the protocol’s specification addresses it. I suspect this hasn’t been a problem much because most implementations send canonical and receive most variants, but technically speaking it is a gap in the specifications.

I created a PR for EAT <https://github.com/eat-ietf-wg/eat/pulls> that should address it for EAT. I choose to put all the burden on the server. The EAT server MUST support all the encoding variants. The client gets to do what it wants since it could be a tiny constrained device. 

Seems like the CBOR RFC should instruct CBOR protocol designer to put a CBOR interoperability section in their specification.  CWT could do similar to EAT. I’m not sure what the right answer for COSE is. 

LL

---

From: Thiago Macieira <[email-masked]>
Date: Mon, 16 Jul 2018 08:36:47 -0700

On Monday, 16 July 2018 08:03:38 PDT Laurence Lundblade wrote:
> Seems to me that is it possible to have non-interoperable CWT or COSE
> implementations that are fully compliant with the specs due to the allowed
> CBOR encoding variations. A CWT sender could only send indefinite lengths.
> A CWT receiver is not required to support indefinite lengths.  Similar for
> integer encoding.

I don't agree. If the protocol doesn't say "encoders must not use indefinite 
length", then indefinite length is a possibility and decoders must be able to 
read it. Then they remove interoperable.

> This seems true for every CBOR protocol unless the protocol’s specification
> addresses it. I suspect this hasn’t been a problem much because most
> implementations send canonical and receive most variants, but technically
> speaking it is a gap in the specifications.

I disagree.

> I created a PR for EAT <https://github.com/eat-ietf-wg/eat/pulls> that
> should address it for EAT. I choose to put all the burden on the server.
> The EAT server MUST support all the encoding variants. The client gets to
> do what it wants since it could be a tiny constrained device.

That seems like a good compromise: put the burden on the bigger of the 
devices. Which one that is (encoder or decoder) depends on the protocol in 
question.

> Seems like the CBOR RFC should instruct CBOR protocol designer to put a CBOR
> interoperability section in their specification.  CWT could do similar to
> EAT. I’m not sure what the right answer for COSE is.

That looks like a good recommendation.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

---

From: Thiago Macieira <[email-masked]>
Date: Mon, 16 Jul 2018 08:45:48 -0700

On Monday, 16 July 2018 08:36:47 PDT Thiago Macieira wrote:
> length", then indefinite length is a possibility and decoders must be able
> to read it. Then they remove interoperable.

The last sentence didn't make sense due to typo. I meant to say: then they 
remain interoperable.

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Jul 2018 12:01:46 -0400

> The last sentence didn't make sense due to typo. I meant to say: then they 

Right.  If a spec based on CBOR does not restrict representation variants, then it doesn’t restrict them.  We could be more explicit about that.  (See my recent message about representation variants.)

Apart from floating point, this hasn’t been a problem that I have heard of from the CBOR community.

Grüße, Carsten

---

From: Laurence Lundblade <[email-masked]>
Date: Mon, 16 Jul 2018 14:37:42 -0400

> On Jul 16, 2018, at 12:01 PM, Carsten Bormann <[email-masked]> wrote:
> 

For this to interop, the receiver MUST implement all encoding variants unless a specification for a specific CBOR-based protocol says otherwise. 

The CBOR specification should be explicit about this. Some text in section 3 should say something like:

“Unless the specification of a CBOR-based protocol explicitly restricts the encoding variants used, the decoder MUST support all encoding variants”

This seems fine for indefinite lengths and less-than-shortest integer and length encoding that are easy to implement, but I’m less sure about more exotic types like bignum, fractions and maybe even floating point epoch time.  If this rule includes these exotics, then most every protocol specification should have a restriction section to exclude these.

LL

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Jul 2018 15:16:37 -0400

On Jul 16, 2018, at 14:37, Laurence Lundblade <[email-masked]> wrote:
> 
> This seems fine for indefinite lengths and less-than-shortest integer and length encoding that are easy to implement, but I’m less sure about more exotic types like bignum, fractions and maybe even floating point epoch time.  

These are not really representation variants: they are visible at the data model level.
If you don’t want Bignums, don’t include them in the specific data model.

If you don’t want floating point times, instead of using the prelude’s

                  time = #6.1(number)

you can define your own CDDL type such as

		  itime = #6.1(int)

Or even:   
		  uitime = #6.1(uint)

> If this rule includes these exotics, then most every protocol specification should have a restriction section to exclude these.

Recommending to always have this section is probably a good idea anyway, but I’m pretty sure that this will be empty in many quite frugal protocols.

Grüße, Carsten

---

From: Thiago Macieira <[email-masked]>
Date: Mon, 16 Jul 2018 12:24:22 -0700

On Monday, 16 July 2018 11:37:42 PDT Laurence Lundblade wrote:
> This seems fine for indefinite lengths and less-than-shortest integer and
> length encoding that are easy to implement, but I’m less sure about more
> exotic types like bignum, fractions and maybe even floating point epoch
> time.  If this rule includes these exotics, then most every protocol
> specification should have a restriction section to exclude these.

That's a good point, but like Carsten says, tagged content is not exactly the 
same as the core encoding. Handling of bignums and bigfloats is very 
implementation-specific and will often require specialised libraries (like 
GMP) to support them. Those won't be found in generic encoders and decoders.

In any case, I recommend expanding the wording about the floating point width 
too. Support for 16-bit hardware floating point is not very widespread, so it 
often involves software conversion, which could be expensive. Similarly, 
constrained devices may opt to have only single-precision floating point 
emulation instead of single- and double-precision. Unlike the encoding 
variants for numbers, those *can* be described in CDDL, but it wouldn't hurt 
to have some explicit text in the spec saying protocols may want to think 
about this.

Finally, I don't know if this is worth discussing, but there's the question of 
the bit patterns for NaN and infinities. Should a protocol restrict what 
encodings are allowed for those values? Or only when they are part of map 
keys?

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 16 Jul 2018 15:34:52 -0400

> encodings are allowed for those values? Or only when they are part of map 

Is there an ambiguity about infinities?

NaN, indeed, is somewhat special.

For canonical, we say (7049 3.9):

	If NaN is an allowed value, it must always be represented as 0xf97e00.

That is almost always a good idea for NaN in general.

(Unless you are using NaN encoding…  Do we want to cater for that?  It is not part of the generic data model.)

Grüße, Carsten

---

From: Thiago Macieira <[email-masked]>
Date: Mon, 16 Jul 2018 12:53:47 -0700

On Monday, 16 July 2018 12:34:52 PDT Carsten Bormann wrote:
> > Finally, I don't know if this is worth discussing, but there's the
> > question of the bit patterns for NaN and infinities. Should a protocol
> > restrict what encodings are allowed for those values? Or only when they
> > are part of map keys?
> 

Oops, no, there isn't. Infinities always have zero significand, so there's 
only one bit pattern for each.

> not part of the generic data model.)

The point is that the code that generated the NaN on the sender side may have 
used a different bit pattern, like using signalling vs quiet NaN. The encoder 
may not have done the conversion to a strictly-controlled value, so this bit 
pattern may end up sent.