Subject: [Cbor] Request for IANA registry addition for cbor tag

From: Kio Smallwood <[email-masked]>
Date: Tue, 21 Jan 2020 11:14:39 +0000

Hi CBOR list,

I'd like to register the following tag for representing key-value maps 
where the order of keys is significant and must be passed through 
encode/decode steps without re-ordering:

Requesting tag #272
Data item: array
Semantics: Ordered Map
Point of contact: [email-masked]
Description of semantics (URL): 
https://github.com/Sekenre/cbor-ordered-map-spec/blob/master/CBOR_Ordered_Map.md

Thanks!

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 21 Jan 2020 12:26:53 +0100

Hi Kio,

overall, your registration request looks good to me.
(Nit: The way you are using single quotes in your diagnostic notation triggers appendix G.2 of RFC 8610 [0], which apparently is not what you want.)
In the editorial revision of RFC 7049 that we are creating right now [1], we are moving away from the concept of a strict decoder; instead we are identifying the specific checking tasks in more detail (here: map validity [2]).

Thank you for also bringing this to the WG.
A lot of people have been suggesting one or another form of ordered map (including ordered multimaps, which would remove the need for avoiding duplicate keys).  Maybe we can have a short discussion if the registration of this kind of ordered maps is a good reason to complete registration of related kinds.

Grüße, Carsten

[0]: https://tools.ietf.org/html/rfc8610#appendix-G.2
[1]: https://github.com/cbor-wg/CBORbis
[2]: https://cbor-wg.github.io/CBORbis/draft-ietf-cbor-7049bis.html#rfc.section.5.3.1

> On 2020-01-21, at 12:14, Kio Smallwood <[email-masked]> wrote:
> 
> Hi CBOR list,
> 
> I'd like to register the following tag for representing key-value maps where the order of keys is significant and must be passed through encode/decode steps without re-ordering:
> 
> 
> Requesting tag #272
> Data item: array
> Semantics: Ordered Map
> Description of semantics (URL): https://github.com/Sekenre/cbor-ordered-map-spec/blob/master/CBOR_Ordered_Map.md
> 
> 
> Thanks!
> 
> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor

---

From: Kio Smallwood <[email-masked]>
Date: Tue, 21 Jan 2020 11:59:19 +0000

Hi Carsten,

Thanks I've update the diagnostic notation.

I've never looked at multimaps (Python replaces values with duplicate 
keys in it's native dict so it's never come up for us)

Cheers,
Kio

On 2020-01-21 11:26, Carsten Bormann wrote:

> (Nit: The way you are using single quotes in your diagnostic notation 
> triggers appendix G.2 of RFC 8610 [0], which apparently is not what you 
> want.)
> In the editorial revision of RFC 7049 that we are creating right now 
> [1], we are moving away from the concept of a strict decoder; instead 
> we are identifying the specific checking tasks in more detail (here: 
> map validity [2]).
> 
> A lot of people have been suggesting one or another form of ordered map 
> (including ordered multimaps, which would remove the need for avoiding 
> duplicate keys).  Maybe we can have a short discussion if the 
> registration of this kind of ordered maps is a good reason to complete 
> registration of related kinds.
> 
> Grüße, Carsten
> 
> [2]: 
> https://cbor-wg.github.io/CBORbis/draft-ietf-cbor-7049bis.html#rfc.section.5.3.1
>

---

From: Laurence Lundblade <[email-masked]>
Date: Tue, 21 Jan 2020 09:58:40 -0800

Here’s some decoders types in relation to this tag:

1) It recognizes the tag, supports it and does the right thing.

2) It recognizes the tag, knows it can’t do what is required and errors out. For this decoder this type is invalid.

3) It doesn’t recognizes the tag and happens to do the wrong thing. That is perfectly OK.

4) It doesn’t recognize the tag and happens to do the right thing.

So anyone using this tag really needs to be sure they are using a decoder of type 1. Decoder types 3 and 4 are really the same in that there is no way to know which one you’ve got.

I don’t think we want some sort of implication that decoder type 3 is defective creating a situation where all decoders are forced to support this.

I think this should be spelled out in the description of this tag.

LL

> On Jan 21, 2020, at 3:26 AM, Carsten Bormann <[email-masked]> wrote:
> 
> Hi Kio,
>

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 21 Jan 2020 21:02:41 +0100

Hi Laurence,

I’m not sure I follow.  Decoders either implement a tag or they don’t.

> On 2020-01-21, at 18:58, Laurence Lundblade <[email-masked]> wrote:
> 

That would be a decoder that implements the tag.

> 2) It recognizes the tag, knows it can’t do what is required and errors out. For this decoder this type is invalid.

That would be a decoder that fundamentally doesn’t accept this tag.
Not particularly useful.
What does “can’t do what is required” mean?

> 3) It doesn’t recognizes the tag and happens to do the wrong thing. That is perfectly OK.

How would that be OK?  But maybe I don’t understand that “do the wrong thing” means.

> 4) It doesn’t recognize the tag and happens to do the right thing.

Which would be to present the tag number and the tag content to the application.

> I don’t think we want some sort of implication that decoder type 3 is defective creating a situation where all decoders are forced to support this.

Now I’m completely lost about your type 3.  “Do the wrong thing” and “OK” don’t mix that well for me…

> I think this should be spelled out in the description of this tag.

I think 7049bis might need some more text about handling unimplemented tags in decoders.

Right now, it says:
  Implementations receiving an unknown tag number can choose to simply ignore
  it or to process it as an unknown tag number wrapping the enclosed data
  item. The IANA registry in {{ianatags}} is the appropriate way to

But it also says:
Generic encoders provide an application interface that allows the
application to specify any well-formed value, including simple values
and tags unknown to the encoder.

So a generic encoder would not have problems with unimplemented tags; but other (e.g., task-specific) implementations might.

Grüße, Carsten

---

From: Jim Schaad <[email-masked]>
Date: Tue, 21 Jan 2020 17:05:36 -0800

-----Original Message-----
From: CBOR <[email-masked]> On Behalf Of Carsten Bormann
Sent: Tuesday, January 21, 2020 12:03 PM
To: Laurence Lundblade <[email-masked]>
Cc: [email-masked]; Kio Smallwood <[email-masked]>
Subject: Re: [Cbor] Request for IANA registry addition for cbor tag

Hi Laurence,

Right now, it says:

[JLS]
Carsten,

I would argue that the decode that I use obeys the text above and would fall into the category of decode that is going to do one of "bad" things Laurence discussed above.  It will return a tree with the following 
<TAG  new tag number>
   <MAP>
       [<key> <value>]+

However, depending on what the final language that the group comes up with for the duplicate tag language, it will either take the first, the last or throw and exception when a duplicate tag is found.  That says that it can not do the first option (hey dup tags are not done will for hash based maps) and thus will either do 3  or totally fail.

I would not really be a huge fan of this tag.  If this is really what you want to do then an array seems to me to be a better answer.

Jim

But it also says:
Generic encoders provide an application interface that allows the application to specify any well-formed value, including simple values and tags unknown to the encoder.

Grüße, Carsten

---

From: Laurence Lundblade <[email-masked]>
Date: Tue, 21 Jan 2020 18:25:30 -0800

My bad. I read the proposal wrong.  Let me try again.

1) Decoder recognizes and supports the tag. A nice ordered map will be returned to the caller.

2) Decoder knows nothing of the tag. A tag plus an array (not a map) will be returned to the caller in order.

3) The programing environment can’t support returning an ordered map and the implementor considers doing something about the tag

3a) The mellow implementor opts for 2).

3b) The knows-what-is-best implementor considers the input invalid and returns an error.

3c) The lets-all-get-along implementor offers an option to behave as 2) or 3b).

This tag is really aimed at type 1) so it is sort of the same, if you use this tag, be sure the decoder supports it. Same with just about any tag. Seems OK.

Separately...

So it appears to me that no where does it say “arrays are ordered” in 7049 or CBORbis. This seems like an omission that should be fixed. They are ordered aren’t they? COSE would break if they weren’t.

Also, I noticed recently that JWT requires either duplicate detection or take-last. CWT is silent on this including inheriting this behavior from JWT.

LL

> On Jan 21, 2020, at 5:05 PM, Jim Schaad <[email-masked]> wrote:
> 
> 
>

---

From: Jim Schaad <[email-masked]>
Date: Tue, 21 Jan 2020 18:45:13 -0800

From: Laurence Lundblade <[email-masked]> 
Sent: Tuesday, January 21, 2020 6:26 PM
To: Jim Schaad <[email-masked]>
Cc: Carsten Bormann <[email-masked]>; [email-masked]; Kio Smallwood <[email-masked]>
Subject: Re: [Cbor] Request for IANA registry addition for cbor tag

 

Separately...

[JLS]  I think that this is implicit in the definition.  Everything would break if arrays are not ordered.

[JLS] I would actually say that CWT is going to inherit behavior from 7049 not from JWT.

Jim

LL

On Jan 21, 2020, at 5:05 PM, Jim Schaad <[email-masked] <mailto:[email-masked]> > wrote:

-----Original Message-----
From: CBOR <[email-masked] <mailto:[email-masked]> > On Behalf Of Carsten Bormann
Sent: Tuesday, January 21, 2020 12:03 PM
To: Laurence Lundblade <[email-masked] <mailto:[email-masked]> >
Cc: [email-masked] <mailto:[email-masked]> ; Kio Smallwood <[email-masked] <mailto:[email-masked]> >
Subject: Re: [Cbor] Request for IANA registry addition for cbor tag

Hi Laurence,

On 2020-01-21, at 18:58, Laurence Lundblade <[email-masked] <mailto:[email-masked]> > wrote:

Right now, it says:

[JLS]
Carsten,

Jim

But it also says:

Grüße, Carsten

_______________________________________________
CBOR mailing list
[email-masked] <mailto:[email-masked]> 
https://www.ietf.org/mailman/listinfo/cbor

---

From: Kio Smallwood <[email-masked]>
Date: Wed, 22 Jan 2020 10:12:18 +0000

Reply below; 

On 2020-01-21 20:02, Carsten Bormann wrote:

> Hi Laurence,
> 
> I'm not sure I follow.  Decoders either implement a tag or they don't. 
> 
> <snip> 
> 

I agree, 

As a maintainer of a generic decoder, unknown tags are passed to the
calling application as a CBORTag(tag, value) [1] instance. Value is
always the corresponding Python datatype to the basic CBOR datatype.
This is serialized by the encoder into the same bytes, so it's
round-trip safe. 

A more specialized decoder should raise an error if it encounters an
unknown tag since that specialization implies some sort of schema that
the decoder expects. 

I think the 7049bis should specify that a generic decoder should be able
to produce objects that will be serialized identically (with the
possible exception of decoding in relaxed mode, then encoding
canonically) regardless of whether those objects have meaning to the
application. 

Cheers, 

Kio 

Links:
------
[1]
https://cbor2.readthedocs.io/en/latest/modules/types.html#cbor2.types.CBORTag