Subject: [Cbor] Invalid decimal fraction / big float?

From: Laurence Lundblade <[email-masked]>
Date: Sun, 25 Aug 2019 02:57:56 -0700

Here’s text from section 3.4.5:

   The exponent e MUST be represented in an integer of major
   type 0 or 1, while the mantissa also can be a bignum (Section 3.4.4 <https://tools.ietf.org/html/draft-ietf-cbor-7049bis-06#section-3.4.4>).
   Contained items with other structures are invalid.

For the most part it seems clear. If the second item in the array is not an integer or a bignum the decimal fraction / big float is invalid.

However, I’ve heard comments that tags are only hints and might be optional if the meaning is clear without them. For example CWT prohibits using the tag value 1 with “exp”, “nbf” and “iat” claims.  In the case of a bignum vs an integer, the meaning is clear without a tag value 2 or 3 explicitly indicating it is a bignum.

Is this a valid decimal fraction?

C4            # tag(4)
   82         # array(2)
      01      # unsigned(1)
      42      # bytes(2)
         0102 # "\x01\x02"

Presumably this is valid, but if we did like CWT, it would be invalid.

C4               # tag(4)
   82            # array(2)
      01         # unsigned(1)
      C2         # tag(2)
         42      # bytes(2)
            0102 # "\x01\x02"

My inclination would be that they are both valid. If that is the choice, then decoders MUST handle both and that is not obvious from the text.

LL

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 25 Aug 2019 15:19:44 +0200

> My inclination would be that they are both valid. If that is the choice, then decoders MUST handle both and that is not obvious from the text.

The intent (and the text) of RFC 7049 require that the second element of the array is either a basic integer (major type 0/1) or a tag 2/3 bignum.  

The meaning of your example with a byte string in that position may be clear to you, but it isn’t to me.  (It also isn’t allowed by RFC 7049.)  I would not propose that we start assigning that meaning to it after the fact.

Grüße, Carsten

---

From: Laurence Lundblade <[email-masked]>
Date: Sun, 25 Aug 2019 12:22:06 -0700

> On Aug 25, 2019, at 6:19 AM, Carsten Bormann <[email-masked]> wrote:
> 

OK. Tag 2/3 is required. 

It seemed worth checking to me because of 1) comments and emails during the recent Prague meeting about tags being hints and optional and 2) CWT expressly forbidding them. Wanted to be sure there was no assumptions being made. 

There seems to be no rule about when tags are present or not. Each protocol gets to decide between:
a) required — for example decimal fraction
b) forbidden — for example time CWT
c) optional — (I don’t know of an example)

A good generic decoders will handle all three, probably with some feature in the API to say which of the above scenarios to use.

But understood that a decimal fraction with an untagged big num is invalid.

LL

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 25 Aug 2019 21:45:21 +0200

On Aug 25, 2019, at 21:22, Laurence Lundblade <[email-masked]> wrote:
> […] OK. Tag 2/3 is required. 
> 
> It seemed worth checking to me because of 1) comments and emails during the recent Prague meeting about tags being hints and optional and

That language is a bit weird.

Tags are elements of the CBOR data model.
Their use is indeed optional, like with any other element of the CBOR data model.
Some applications may be able to make sense of a CBOR data item while ignoring Tags that actually are present, but this requires a lot of foresight on the side of the application designer.

I don’t know why Tags would be “hints” in a general sense (well, some of them are designed to be exactly that, but that is then the semantics of that specific Tag — e.g., JSON encoding hints).

> 2) CWT expressly forbidding them.

That is not the case at all.
Only those 7 claims defined in RFC 8392 are defined to not use Tags (because they don’t need them, and nothing is gained by allowing them).

I cite from Section 5 of RFC 8392:

…  this does not prohibit future claim
   definitions from requiring the use of CBOR tags for those specific
   claims.

> Wanted to be sure there was no assumptions being made. 
> 

CWT can do that because the exp/nbf/iat claims are simply defined that way.
There is nothing “forbidden” here.  If you really want to use this word, you are also “forbidden” to tag a claim 1 (“iss”) as a URI (or as a geographic coordinate, for that matter) — this claim always gets a text string.
(Note that CWT defines Tag 61 as “CWT” and also uses COSE Tags, which makes it even more obvious Tags are not “forbidden” with CWT.)

> c) optional — (I don’t know of an example)

Any protocol that uses CDDL `unsigned` has “optional” Tags:
Numbers below 2**64 do not use Tags, numbers equal to or greater than 2**64 do.

> A good generic decoders will handle all three, probably with some feature in the API to say which of the above scenarios to use.

Since the generic decoder hands the application the decoded data item, the application can handle the Tags and their being required/optional/forbidden.

			.oOo.

I think that these are good comments, because they can help us get rid of some weasel wording in RFC 7049.  (In 2013, the concept of Tags was somewhat revolutionary, and maybe the authors were a bit too cautious foisting it on the CBOR community.  By now, people have become familiar with Tags, so maybe we can say in simpler words how they work.)

Grüße, Carsten

---

From: Laurence Lundblade <[email-masked]>
Date: Sun, 25 Aug 2019 14:04:28 -0700

> On Aug 25, 2019, at 12:45 PM, Carsten Bormann <[email-masked]> wrote:
> 

Yes, I misused the word “hints” here.

> Only those 7 claims defined in RFC 8392 are defined to not use Tags (because they don’t need them, and nothing is gained by allowing them).

I mean forbids them only for the claims it defines:

   The claim values defined in this specification MUST NOT be prefixed
   with any CBOR tag. For instance, while CBOR tag 1 (epoch-based date/
   time) could logically be prefixed to values of the "exp", "nbf", and
   "iat" claims

Digging in a little deeper…

The claims “exp”… are defined by reference to NumericDate in JWT. They are not actually defined as a CBOR tag 1 date without a tag. This makes the comment about them not being tagged in 8392 a bit mis guided (but not problematic).

CWT aside, by my understanding, it is allowed for CBOR protocols to make use of the data types defined in “Optional Tagging of Items” section of 7049 without explicitly adding a tag. 

For example, I could say in the EAT definition that the claim labeled “teetime”  is a tag 1 epoch-based date as defined in 7049. EAT could (should?) then say which of a), b) or c) is allowed for use of the tag 1. Right? Since the data type is known from the map label / key any of a), b) or c) will work.

Maybe a bit more guidance on selecting between a), b) and c) in a protocol design would be useful?

>   definitions from requiring the use of CBOR tags for those specific

Agreed

> (Note that CWT defines Tag 61 as “CWT” and also uses COSE Tags, which makes it even more obvious Tags are not “forbidden” with CWT.)

When CWT says “MUST NOT be prefixed with any CBOR tag”, it sounds semantically like “forbidden” to me.

> Since the generic decoder hands the application the decoded data item, the application can handle the Tags and their being required/optional/forbidden.

Generic decoders can handle tagged types inside the decoder itself. Mine does. That means those types of generic decoders have to know about case a), b) and c). As the number of data types expands and becomes more useful, having generic decoders handle them seems beneficial. 

> I think that these are good comments, because they can help us get rid of some weasel wording in RFC 7049.  (In 2013, the concept of Tags was somewhat revolutionary, and maybe the authors were a bi…

Despite my nit picking, I really do like CBOR a lot

LL

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 25 Aug 2019 23:32:40 +0200

> CWT aside, by my understanding, it is allowed for CBOR protocols to make use of the data types defined in “Optional Tagging of Items” section of 7049 without explicitly adding a tag. 

Absolutely.  Making that easier to do was one reason we introduced ~ (unwrap) in CDDL.

> 
> For example, I could say in the EAT definition that the claim labeled “teetime”  is a tag 1 epoch-based date as defined in 7049. EAT could (should?) then say which of a), b) or c) is allowed for use of the tag 1. Right?

Well, EAT should simply say what the claim is.  There is no need for a/b/c here.
Either you use the tag or just its definition of the semantics applied to the enclosed data item; I can’t imagine a case where anything is gained by allowing both.

> When CWT says “MUST NOT be prefixed with any CBOR tag”, it sounds semantically like “forbidden” to me.

Yes.  But it’s not that CWT forbids tags, it is that it defines the first seven claims for CWT in a way that does not make use of tags.  It would be as misleading to say that “COSE is forbidden in CWT” because the first seven claims don’t use COSE.

> Generic decoders can handle tagged types inside the decoder itself.

They never can for all tags, because all tags haven’t been invented yet.

> Mine does. That means those types of generic decoders have to know about case a), b) and c).

I still don’t think so.  That would only be the case if the decoder doesn’t “handle” the tag, but “hides” it.  That would indeed lead to problems.

> Despite my nit picking, I really do like CBOR a lot

Thanks.  We need to have these discussions to ensure we don’t have language in the spec that turns out to be misleading.  I just pushed a pull request for CBORbis that massively rearranges the language around tags, please have a look:

https://github.com/cbor-wg/CBORbis/pull/109

Grüße, Carsten

---

From: Laurence Lundblade <[email-masked]>
Date: Tue, 27 Aug 2019 21:30:59 -0700

Here's text I think might be useful to add to the Creating CBOR-based Protocols section:

Protocols using data types defined in the section Tagging of Items section can use them with explicit tags (type 6 enclosing data items) or with no tag and their type implied (no type 6 enclosing data item). If there is no explicit tag, the protocol design should be sure the data type can be unambiguously recognized in all use case. This is often accomplished by saying a member of a map with a particular label is always of a particular type.

Explicit tagging should only be used when it is actually necessary to clearly distinguish the type.

Some protocol designs may forbid explicit tagging of particular data items.

Protocol designs should directly state whether explicit tagging is required, disallowed or optional for each use of these data types.

One of the reasons I’m bring this up is that in a way section 3.4 Tagging of Items  has the horse before the cart. The definition of the new data types seems more important than the tagging as the tagging is the optional part. Alternative titles for the section might be Additional Data Types or Compound Data Types. The somewhat unclear (to me) characterization of the optionality of tagging is why I asked about tags in decimal fractions.

I think it is useful and practical for generic decoders to directly support tagged data types rather than just passing them on to the caller with the tag.

In some cases the tagged data types may have natural representations in the language or the platform (e.g. time formats). In some cases the tagged data types are complex enough that some CBOR expertise it helpful (e.g., array tags) when implementing. The decoder author might also want to encapsulate more of CBOR so the caller doesn’t have to know as much about CBOR.

Obviously, it can’t be for tags that aren’t invented yet, some tagged data types have use that is too narrow, and some may be poorly designed in the opinion of the author of the decoder. Generic decoders should also support the caller implementing tagged data types on their own and a mode to pass tags through.

That’s all just my view and no particular issue with the text. I mention it in part to say that generic decoders supporting tagged data types may wish to have features in their API to handle explicit / implicit tagging.

LL

> On Aug 25, 2019, at 2:32 PM, Carsten Bormann <[email-masked]> wrote:
> 
> 
> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 28 Aug 2019 07:11:30 +0200

Hi Laurence,

good point.
However, tagging is tagging, there can be no tagging without tagging.
So “implicit tagging” is an oxymoron to me.

What we could say:

The data types that are being defined in the form of CBOR tag definitions may also be useful to data definition languages, which may provide a way to employ the type definitions at specific locations identified by their context by means of the data definition language, without a representation of the actual tag being exchanged (for instance, the “unwrap” operator defined for CDDL in Section 3.7 of RFC 8610 can be used for this purpose: `~time` could stand for a number representing a Posix time without actually encoding Tag 1).
A CBOR-based protocol definition based on such a data definition language may still want to employ tags to enable automatic processing of tags in generic decoders, and to provide a distinguishing semantics where needed (e.g., to distinguish a time from another use of numbers possible in the same place).
Generic decoders may want to provide their tag data processing capabilities (e.g., converting a number into a time) in an unbundled form to the application in order to enable the processing of such data types identified by context, and not by an explicit tag.

(Unrelated:)
A CBOR-based protocol definition will typically define exactly where tags are and are not to be used, just as with other containers such as arrays and maps (giving rise to a “structural” view of application level validity).
Alternatively, it could describe its protocol in terms of the data types that are conceptually “returned” by a tag, e.g., it could treat all arrays the same, independently of whether they are represented as classical CBOR arrays or as typed arrays created by specific tags [I-D.ietf-cbor-array-tags]; we could term this as a “semantic” view of application level validity.  Note that the present specification does not define a “semantic” type system that could be employed for this; such definitions are left to further work.

The above needs a lot more word smithing (and may benefit from creating contexts where it is easier to say these things), but you get the idea where I think this should be heading.

Generic decoders will always need to be able to hand (e.g., unknown) tags to the application in an unprocessed manner.  Where processing *is* performed, it may be necessary to indicate to the application that a tag was employed (to keep the distinguishing semantics), except where we explicitly exclude this (e.g., bignums should never be used with distinguishing semantics).

Grüße, Carsten

> On Aug 28, 2019, at 06:30, Laurence Lundblade <[email-masked]> wrote:
>

---

From: Laurence Lundblade <[email-masked]>
Date: Wed, 28 Aug 2019 18:33:58 -0700

I meant to say “cart before the horse” below.

It seems possible that use of these data types without explicit tags could actually be more common than use with the explicit tags. CWT is one example. EAT will probably be similar. I recall ASN.1 protocols having a lot of implicit stuff.

Anyway, I think you have my meaning on this.

LL

> On Aug 27, 2019, at 10:11 PM, Carsten Bormann <[email-masked]> wrote:
> 
> Hi Laurence,
> 
> good point.