Subject: [lp-wan] Discussion about Target Value data structure

From: Charlie Perkins <[email-masked]>
Date: Mon, 26 Nov 2018 20:45:22 -0800

Hello folks,

The current draft does not provide details about how a Target Value is 
represented in a rule (e.g., as depicted in Figure 6). Without some 
specification for the data format, it is difficult to see how different 
vendors could enable interoperable Rule configuration for senders and 
receivers.

During discussion with some of the authors, that much was agreed.  It 
was suggested that the matter belongs in a separate document.  I like 
that suggestion, but it just moves the problem elsewhere.  And, if the 
goal is to enable interoperable implementations, then the current SCHC 
draft would require a normative dependency on the new document.

Another possibility is to restrict the kinds of Target Values that can 
be used in the Rules as specified by the current document.  For example, 
there could be a 3-bit "TV_type prefix" for Target Value that determines 
the data type of the Target Value (0 = byte, 1 = word, 2 = string, ..., 
6=JSON, 7=CBOR).  I just made this up and don't pretend to say that 3 
bits are enough, or that the array of values needs to start with those 
three types. Since the type values are not transmitted over the air in 
SCHC packets, I suppose the number of bits in the type descriptor 
doesn't need to be restricted.

A last possibility is to somehow state that the current document is not 
intended to fully enable interoperable implementations. I'm not sure 
about that sort of thing, but to me it seems wrong to require a 
Hackathon to decide what's what before interoperability can occur.

Regards,
Charlie P.

---

From: ana minaburo <[email-masked]>
Date: Tue, 27 Nov 2018 10:32:17 +0100

Hello,

For my understanding, the type of TV is implied in its definition together
with the FID. If FID common value is an integer the TV belongs to this
type, for example, the FID version for IPv6 the TV is always 6 it is an
integer. Do I miss something?

Ana

---

From: Laurent Toutain <[email-masked]>
Date: Tue, 27 Nov 2018 11:12:59 +0100

Hi,

I agree with Ana, FID helps to have the type of TV. A FID for a port number
implies an integer and for a CoAP.URI-path a string. Reading the draft, the
FID usage is not well described.
The position in the rule is not enough, the matching operation includes the
FID to take its decision. For instance, if you have a header A with fields
F1 and F2 and a header B with fields F3 and F4 and the values match the TV
for both protocols, the decision will be based on the type of the fields,
so FID.

We need to add in section 7.3 of draft -17

  *  The first step is to choose the Field Descriptions by their
         direction, using the Direction Indicator (DI).  A Field
         Description that does not correspond to the appropriate DI will
         be ignored.  If all the fields of the packet do not have a
         Field Description with the correct *FID and* DI, the Rule is discarded
         and SCHC C/D proceeds to consider the next Rule.

Laurent

On Tue, Nov 27, 2018 at 10:33 AM ana minaburo <[email-masked]> wrote:

>> the data type of the Target Value (0 = byte, 1 = word, 2 = string, ....,
>> _______________________________________________
> lp-wan mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/lp-wan
>

---

From: Alexander Pelov <[email-masked]>
Date: Wed, 28 Nov 2018 10:19:50 +0100

Hello Charlie,

Thanks a lot for the detailed review!

A point on the Target Value (and by the way, the entire representation of the Rules).
This belongs to the Data Model, which we will focus on after the rechartering.

You are right to suggest that interoperable rule configuration is not defined in the SCHC document. That is not its goal.
The SCHC end-point can obtain its Context from many formats. A server-based implementation may rely on XML, a constrained device on binary representation. In the end, we need to work on the Architecture document to describe how the whole system works, and the Data Model to describe the Schema. Once we have that, the serialisation is trivial.

We've been heavily discussing the use of YANG as Data Model, with the possible serialisations being XML, JSON or CBOR. This also provides the configuration protocol : NETCONF, RESTCONF or CORECONF.

There are of course things to discuss, but I don't think the SCHC IP/UDP document is the right place to do. 

Does that seem reasonable to you?

Cheers
Alexander

> On 27 Nov 2018, at 05:45, Charlie Perkins <[email-masked]> wrote:
> 
> Hello folks,
> 
> The current draft does not provide details about how a Target Value is represented in a rule (e.g., as depicted in Figure 6). Without some specification for the data format, it is difficult to see how different vendors could enable interoperable Rule configuration for senders and receivers.
> 
> During discussion with some of the authors, that much was agreed.  It was suggested that the matter belongs in a separate document.  I like that suggestion, but it just moves the problem elsewhere.  And, if the goal is to enable interoperable implementations, then the current SCHC draft would require a normative dependency on the new document.
> 
> Another possibility is to restrict the kinds of Target Values that can be used in the Rules as specified by the current document.  For example, there could be a 3-bit "TV_type prefix" for Target Value that determines the data type of the Target Value (0 = byte, 1 = word, 2 = string, ..., 6=JSON, 7=CBOR).  I just made this up and don't pretend to say that 3 bits are enough, or that the array of values needs to start with those three types. Since the type values are not transmitted over the air in SCHC packets, I suppose the number of bits in the type descriptor doesn't need to be restricted.
> 
> A last possibility is to somehow state that the current document is not intended to fully enable interoperable implementations. I'm not sure about that sort of thing, but to me it seems wrong to require a Hackathon to decide what's what before interoperability can occur.
> 
> Regards,
> Charlie P.
> 
> 
>

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 28 Nov 2018 18:53:06 +0000

Hello Alex and Charlie:

People can do what they like in their code. Devices will probably be hardcoded to ship an exact binary without the realization in the code that there are rules to turn the bitstrings into IPv6. 
The TV field in this draft is an abstract representation. It is not visible over the air. What is visible is the compressed and uncompressed IP. These are the formats that people must use to agree on what a rule does, e.g., turn bit X to bit Y  in the compressed format into an IPv6 address, and the other way around. I see from Charlie's comment that the spec has extraneous text that induces a misunderstanding and I suggest to remove it, less is more. 

What about:

Before:

The context contains a list of Rules (see Figure 6).  

After
The context represented in Figure 6 is an abstract data structure that contains a list of Rules. The exact format is implementation-dependant. 

-------

Before:
   o  Target Value (TV) is the value used to match against the packet
      header field.  The Target Value can be of any type (integer,
      strings, etc.).  It can be a single value or a more complex
      structure (array, list, etc.), such as a JSON or a CBOR structure.

After
   o  Target Value (TV) is the value used to match against the packet
      header field.  

--------

Take care;

Pascal

> -----Original Message-----
> From: lp-wan <[email-masked]> On Behalf Of Alexander Pelov
> Sent: mercredi 28 novembre 2018 10:20
> To: Charlie Perkins <[email-masked]>
> Cc: lp-wan <[email-masked]>; [email-masked]
> Subject: Re: [lp-wan] Discussion about Target Value data structure
> 
> Hello Charlie,
> 
> The SCHC end-point can obtain its Context from many formats. A server-based
> implementation may rely on XML, a constrained device on binary
> representation. In the end, we need to work on the Architecture document to
> describe how the whole system works, and the Data Model to describe the
> Schema. Once we have that, the serialisation is trivial.
> 
> We've been heavily discussing the use of YANG as Data Model, with the
> possible serialisations being XML, JSON or CBOR. This also provides the
> configuration protocol : NETCONF, RESTCONF or CORECONF.
> 
> represented in a rule (e.g., as depicted in Figure 6). Without some specification
> for the data format, it is difficult to see how different vendors could enable
> interoperable Rule configuration for senders and receivers.
> >
> > During discussion with some of the authors, that much was agreed.  It was
> suggested that the matter belongs in a separate document.  I like that
> suggestion, but it just moves the problem elsewhere.  And, if the goal is to
> enable interoperable implementations, then the current SCHC draft would
> require a normative dependency on the new document.
> >
> > Another possibility is to restrict the kinds of Target Values that can be used in
> the Rules as specified by the current document.  For example, there could be a
> 3-bit "TV_type prefix" for Target Value that determines the data type of the
> Target Value (0 = byte, 1 = word, 2 = string, ..., 6=JSON, 7=CBOR).  I just made
> this up and don't pretend to say that 3 bits are enough, or that the array of
> values needs to start with those three types. Since the type values are not
> transmitted over the air in SCHC packets, I suppose the number of bits in the
> type descriptor doesn't need to be restricted.
> >
> intended to fully enable interoperable implementations. I'm not sure about
> that sort of thing, but to me it seems wrong to require a Hackathon to decide
> what's what before interoperability can occur.
> >
> > Regards,
> > Charlie P.
> >
> >
> >

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 28 Nov 2018 19:09:07 +0000

In addition we could clarify the scope of this document in the intro:

Before

   This document defines generic functionality and offers flexibility
   with regard to parameters settings and mechanism choices.
   Technology-specific settings and product-specific and choices are
   expected to be grouped into Profiles specified in other documents.

After 

   This document defines generic functionality and offers flexibility
   with regard to parameters settings and mechanism choices. Data
   models for the context and profiles are out of scope; they will be
   required to program the network side of the SCHC protocol.
   Technology-specific settings and product-specific and choices are
   expected to be grouped into Profiles specified in other documents.

Works?

Pascal

> From: lp-wan <[email-masked]> On Behalf Of Pascal Thubert
> (pthubert)
> Sent: mercredi 28 novembre 2018 19:53
> To: Alexander Pelov <[email-masked]>; Charlie Perkins
> <[email-masked]>
> People can do what they like in their code. Devices will probably be hardcoded
> to ship an exact binary without the realization in the code that there are rules
> to turn the bitstrings into IPv6.
> The TV field in this draft is an abstract representation. It is not visible over the
> air. What is visible is the compressed and uncompressed IP. These are the
> formats that people must use to agree on what a rule does, e.g., turn bit X to
> bit Y  in the compressed format into an IPv6 address, and the other way
> around. I see from Charlie's comment that the spec has extraneous text that
> induces a misunderstanding and I suggest to remove it, less is more.
> 
> What about:
> 
> Before:
> 
> The context represented in Figure 6 is an abstract data structure that contains a
> list of Rules. The exact format is implementation-dependant.
> 
> -------
> 
> Before:
> > Cc: lp-wan <[email-masked]>;
> > [email-masked]
> > You are right to suggest that interoperable rule configuration is not
> > defined in the SCHC document. That is not its goal.
> > The SCHC end-point can obtain its Context from many formats. A
> > server-based implementation may rely on XML, a constrained device on
> > binary representation. In the end, we need to work on the Architecture
> > document to describe how the whole system works, and the Data Model to
> > describe the Schema. Once we have that, the serialisation is trivial.
> >
> > possible serialisations being XML, JSON or CBOR. This also provides
> > the configuration protocol : NETCONF, RESTCONF or CORECONF.
> >
> > There are of course things to discuss, but I don't think the SCHC
> > IP/UDP document is the right place to do.
> >
> > > On 27 Nov 2018, at 05:45, Charlie Perkins
> > > <[email-masked]>
> > wrote:
> > >
> > > Hello folks,
> > >
> > specification for the data format, it is difficult to see how
> > different vendors could enable interoperable Rule configuration for senders
> and receivers.
> > >
> > suggestion, but it just moves the problem elsewhere.  And, if the goal
> > is to enable interoperable implementations, then the current SCHC
> > the Rules as specified by the current document.  For example, there
> > could be a 3-bit "TV_type prefix" for Target Value that determines the
> > data type of the Target Value (0 = byte, 1 = word, 2 = string, ...,
> > SCHC packets, I suppose the number of bits in the type descriptor doesn't
> need to be restricted.
> > >

---

From: Charlie Perkins <[email-masked]>
Date: Wed, 28 Nov 2018 11:54:54 -0800

Hello Pascal and all,

There is one kind of Profile document that has to do with the underlying 
physical medium.

There is another kind of Profile document that identifies proper choices 
for a particular application.

If these two kinds of documents are mixed together, I reckon the results 
will be problematic.

Currently, it seems to me that Appendix D contains normative text that 
is needed for the main body of the document.  I didn't see Appendix D 
before sending my earlier comments to the authors, and so this point 
escaped my notice.

Regarding your text below, I think it is very good. Nevertheless, I 
don't see how you can expect to configure interoperable implementations 
from different vendors for transport or application protocols that are 
not specified by the IETF. Since a lot (or, numerically, maybe even 
most) of the intended applications might be specified elsewhere, it 
seems to me that this is an issue.  I am going to assume for the moment 
that you do not intend to restrict this document to only apply to 
devices that are hardcoded and cannot be configured.

Nevertheless, the discussion on this point has suggested a pretty good 
answer.  Namely, for each Target Value, there should be a Profile entry 
that identifies the protocol specification and the name of the protocol 
element in the application (or other!) header that defines the data type 
of the Target Value.  This kind of Profile entry should be in the second 
type of Profile document mentioned above.  As a matter of convenience, a 
Profile document could simply state that all Target Value names are as 
specified in a particular protocol document.

In this way, the SCHC document can keep the data typing out of scope, 
and yet enable a straightforward mechanism for promoting interoperability.

Regards,
Charlie P.