Subject: [6tisch] CoMI remote server clash file and YANG hash in URL

From: Alexander Pelov <[email-masked]>
Date: Wed, 3 Jun 2015 17:00:57 +0200

Hello everyone,

I would like to discuss with you to ideas/proposals in this mail, namely:
1) Caching the yang clash file on a remote server
2) Compressing the URL representation of YANG hashes

1) Caching the yang clash file on a remote server
Even though hash collisions should happen rarely (if ever), there may be cases when it could be inevitable, and as such, it may be interesting to cache the clash_file somewhere on a remote server to avoid sending it over the constrained wireless interface. It could be used for other cases, where some frequently used resources are re-hashed for performance purposes. Maybe it would be interesting to allow the server to specify a remote location for the clash file? For static collisions or optimizations it could be that the YANG definition includes alternative hash values, which could be used in addition to the normal ones.

2) Compressing the URL representation of YANG hashes
Another point (which is unrelated), is to elide the leading "A"s from the base64 representation of the hash in the URL (which corresponds to eliding the leading zeroes from the hash). As specified in section 5.4. a hash value of 0x0000007e will be encoded as AAAA_, but if the leading A’s are elided it will be simply _

This could pose problems only if in the future the size of the hash values is increased from 30 bits to some other (higher) value.

This, however, could be very useful when having to frequently access the same resource. In such case, the server/network operator may decide to re-hash the URI to a shorter version, e.g. the /sys:system-state/sys:clock/sys:current-datetime could be remapped from 0x15370408 and VNwQI to 0x1 and _ 

Best,
Alex

PS.
A typo (?), which may be already corrected, in the example of mod.uri on Page 26: the two responses have different scheme "mod.uri" and "moduri"

---

From: Michel Veillette <[email-masked]>
Date: Wed, 3 Jun 2015 16:05:40 +0000

Hi Alexander

At the last 6TiSCH bi-weekly call, 4 solutions have been presented to the hash clashes problem. Two based on a rehash table and two based on a new CoAP query parameter carrying the module name or registered module ID. This query parameter is used to select unambiguously the targeted data node(s) assuming that any hash collision within a module have been resolved at design time. You can find this presentation in attachment.

Both of your proposals make sense to me but I still hope that the clash file can be avoided or be optional.

Michel Veillette
System Architecture Director
Trilliant Inc.
Tel: 450-375-0556 ext. 237
[email-masked]
www.trilliantinc.com   

-----Original Message-----
From: core [mailto:[email-masked]] On Behalf Of Alexander Pelov
Sent: 3 juin 2015 11:01
To: [email-masked]; [email-masked]
Subject: [core] CoMI remote server clash file and YANG hash in URL

Hello everyone,

1) Caching the yang clash file on a remote server Even though hash collisions should happen rarely (if ever), there may be cases when it could be inevitable, and as such, it may be interesting to cache the clash_file somewhere on a remote server to avoid sending it over the constrained wireless interface. It could be used for other cases, where some frequently used resources are re-hashed for performance purposes. Maybe it would be interesting to allow the server to specify a remote location for the clash file? For static collisions or optimizations it could be that the YANG definition includes alternative hash values, which could be used in addition to the normal ones.

2) Compressing the URL representation of YANG hashes Another point (which is unrelated), is to elide the leading "A"s from the base64 representation of the hash in the URL (which corresponds to eliding the leading zeroes from the hash). As specified in section 5.4. a hash value of 0x0000007e will be encoded as AAAA_, but if the leading A’s are elided it will be simply _

Best,
Alex

PS.

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: Alexander Pelov <[email-masked]>
Date: Wed, 3 Jun 2015 18:52:17 +0200

Hi Michel,

Thank you for the presentation - I’ve missed it and it clarifies some questions I had on the discussions on the ML from the last days.

Actually, I’m not that concerned with the possible clashes, but mostly with the possibility to use this mechanism for performance optimization. The aspect that interests me the most is the re-mapping of the hashes, which combined with URL hash compression can save 4 bytes from the URL and 4 bytes from the CBOR representation in some cases (assuming only 1-23 resources are re-maped, 3 bytes saved if 23-255, which should rarely be used). 

This is somehow similar to the third alternate approach, in which each YANG module can specify the data node IDs. 

To elaborate on this, if the YANG module can explicitly specify the hash value of some of its items, I think that you could solve the hash clashes with a deterministic algorithm working on the loaded modules. For example, if you have modules A, B, and C. 
1) Module A’s  calculated IDs and module-defined data node IDs are put in the table
2) For each additional module:
   2.a) for each item, if there is a clash:
      while there is a clashing hash value, item’s hash value ++
 
This way, for a fixed set of modules and node IDs you’ll have a collision-free hash table. The benefit of this is, as it is deterministic, you can pre-calculate the hash remappings offline / statically.

Another point is, in case someone wants to do something optimized, and remaps hash value 1 in module A and B, module A’s item will remain with hash 1, while module B’s item will become 2, both of which can be compressed efficiently in the URL and the CBOR representation (and both of which will always be 1 and 2 if modules A and B are loaded in this order, with no preceding modules).

Unfortunately, my knowledge of YANG / NETCONF is still basic, so maybe I’m missing something obvious which could render this impractical.

Best,
Alexander

> Le 3 juin 2015 à 18:05, Michel Veillette <[email-masked]> a écrit :
> 
> Hi Alexander
> 
> <6TiSCH presentation on CoMI hash clashes r03.pptx>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 3 Jun 2015 10:08:30 -0700

> Both of your proposals make sense to me but I still hope that the clash file can be avoided or be optional.

I agree we should not need the client to retrieve the rehash info
before using a server.
The CoMI authors have discussed a solution which I will try to write up today
where a special "hash-clash" error is returned by CoMI if a rehash is needed.

   1) client requests module-A object with hash  "1234";
        there is no extended-name/local-name,
        just the YANG hash, used in any request

   2) if '1234' has a collision in this server then a special error
code is returned;
       The return payload has the rehash info [old-hash, {
module-name, new-hash }+]
           { "1234": [
                  "module-A":"4567",
                  "module-X":"6523"
              ]
           }

    3) the client gets the error info, and knows which module its
"1234" is from.
         It finds the correct module name in the array and replaces its old-hash
         with the new-hash.

   4) client re-sends the request, using hash "4567" instead of "1234"

The rehash table goes away, and only this error info is used instead.

> [email-masked]

Andy

---

From: Michel Veillette <[email-masked]>
Date: Wed, 3 Jun 2015 18:16:34 +0000

Hi Andy

Couple of questions about your proposed solution.

=== Question #1, New YANG statement

Your solution works only if hash clashes within a module are resolved at design time. If not, the error message will return the following information which is not sufficient to resolve which data node is which.

   { "1234": [
          "module-A":"4567",
          "module-A":"6523"
      ]
   }

Is it possible/acceptable to request a new YANG statement to manually assign a ID to a data node?
Same question for a module ID?

For example (see "id 16" and "id 25" bellow) :

   module a {
      namespace a-ns;
      prefix a;
      id 16;
      revision 2015-01-01;

      leaf leafA1 { type string; id 25 }
      container containerA {
         leaf leafA2 { type string}
      }
   }

=== Question #2, Introduction of an optional query parameter used to target a specific module

For nodes / networks which can afford a slitely larger payload (IEEE 802.15.4g for example) but want to avoid the processing and storage associated with the porposed error message, are you opposed to the support of an optional CoAP query parameter use carry the module name or module ID ?

For example:

GET example.com/Y8d7s?select= ietf-6tisch-mac

=== Question #3, Use of the select query parameter instead of the URI

Are you oposed to the use of the select query parameter instead of the URI to identify data node(s)

For example:

GET example.com/mg?select=Y8d7s
GET example.com/mg?select=Y8d7s,NKHaA
GET example.com/mg?select=ietf-6tisch-mac(Y8d7s,NKHaA)

=== Question #4, Compressing YANG hash base64 representation

Are you oposed to the compression of the base64 representation as propsed by Alexander.

For example:

GET example.com/mg?select=7s
Instead of
GET example.com/mg?select= AAA7s

Michel Veillette

-----Original Message-----
From: Andy Bierman [mailto:[email-masked]] 
Sent: 3 juin 2015 13:09
To: Michel Veillette
Cc: Alexander Pelov; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

> Both of your proposals make sense to me but I still hope that the clash file can be avoided or be optional.

I agree we should not need the client to retrieve the rehash info before using a server.
The CoMI authors have discussed a solution which I will try to write up today where a special "hash-clash" error is returned by CoMI if a rehash is needed.

   2) if '1234' has a collision in this server then a special error code is returned;
       The return payload has the rehash info [old-hash, { module-name, new-hash }+]
           { "1234": [
                  "module-A":"4567",
                  "module-X":"6523"
              ]
           }

    3) the client gets the error info, and knows which module its "1234" is from.
         It finds the correct module name in the array and replaces its old-hash
         with the new-hash.

Andy

> 2) Compressing the URL representation of YANG hashes Another point 
> (which is unrelated), is to elide the leading "A"s from the base64 
> representation of the hash in the URL (which corresponds to eliding 
> the leading zeroes from the hash). As specified in section 5.4. a hash 
> value of 0x0000007e will be encoded as AAAA_, but if the leading A’s 
> are elided it will be simply _
>
> This, however, could be very useful when having to frequently access 
> the same resource. In such case, the server/network operator may 
> decide to re-hash the URI to a shorter version, e.g. the 
> /sys:system-state/sys:clock/sys:current-datetime could be remapped 
> from 0x15370408 and VNwQI to 0x1 and _
>
> Best,
> Alex
>
> PS.

---

From: Andy Bierman <[email-masked]>
Date: Wed, 3 Jun 2015 11:53:38 -0700

> Your solution works only if hash clashes within a module are resolved at design time. If not, the error message will return the following information which is not sufficient to resolve which data n…

The original proposal had a 3-tuple { module-name, path-expr, new-hash }
We discussed some optimizations in the 6TISCH call to reduce the size of
the path-expr and still not require the client to store all the path strings,
such as store a few bytes from the path strings like 6th and 11th bytes
(Avoid the common ietf- prefix for many module names).

This is not deterministic. Only the full path string is fool-proof.
I would prefer if hash collisions in a single module were not allowed.
They need to be resolved at design time.  If this is not possible, then
a standard rehash algorithm should be used so the collision is resolved
in a predictable way.  e.g., try appending "_", then try "__", etc.
The result would be that hash collisions in a single module do not
occur in CoMI.

>          leaf leafA2 { type string}

You could have an extension statement do anything you want:

    comi:id  16;

But these numbers do not exist any any modules and so they
cannot be relied on in CoMI.

> Are you oposed to the use of the select query parameter instead of the URI to identify data node(s)

Not really.

I don't know how mixed the modules will get in CoMI.
In NETCONF, modules tend to use augment a lot,
so the nodes from 'module-A' may be nested nodes.
Retrieving just the augmenting nodes and ignoring the
augmented nodes would likely be operationally useless.
e.g., when augmenting a list, the key leafs are needed
to identify the instance.

I would like to keep CoMI extremely simple.
More bells and whistles means increased
memory and CPU requirements. I am glad CORE and 6TISCH
WGs are so concerned about constrained resources.
Only the most important bells and whistles will get standardized.

> Are you oposed to the compression of the base64 representation as propsed by Alexander.

No.  Optimizations that reduce the encoding size are important.
I want to use standard CBOR.  No special hacks just for CoMI.

Andy

---

From: Michel Veillette <[email-masked]>
Date: Wed, 3 Jun 2015 19:48:10 +0000

About "I would prefer if hash collisions in a single module were not allowed."
Agreed but how this is implemented.

About "But these numbers do not exist in any modules and so they cannot be relied on in CoMI.
I understand but I assume that a module can be re-published with the added statement when require. If a CoMI implementer try to implement a module containing a clash, he can look for a version of this module containing the special YANG statement. The likelihood of such clash is extremely low and such fallback solution won’t be required often. However, the RFC will specify how to resolve such situation if this ever happen. This statement can also be used to address the need for data node identifier compactness as mentioned by Alexander in its original email.

About "In NETCONF, modules tend to use augment a lot"
In my document " draft-vanderstok-core-comi-06 - MV.docx ", I propose a new definition to address this.

Module Context: A module context is composed of all data nodes, notifications and RPCs defined in a YANG module and included sub-modules or added to them using the augment YANG statement.

Section 4.1.3.4 show an example of include, import and augment.

Michel Veillette

-----Original Message-----
From: Andy Bierman [mailto:[email-masked]] 
Sent: 3 juin 2015 14:54
To: Michel Veillette
Cc: Alexander Pelov; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

> Your solution works only if hash clashes within a module are resolved at design time. If not, the error message will return the following information which is not sufficient to resolve which data n…

The original proposal had a 3-tuple { module-name, path-expr, new-hash } We discussed some optimizations in the 6TISCH call to reduce the size of the path-expr and still not require the client to store all the path strings, such as store a few bytes from the path strings like 6th and 11th bytes (Avoid the common ietf- prefix for many module names).

This is not deterministic. Only the full path string is fool-proof.
I would prefer if hash collisions in a single module were not allowed.
They need to be resolved at design time.  If this is not possible, then a standard rehash algorithm should be used so the collision is resolved in a predictable way.  e.g., try appending "_", then try "__", etc.
The result would be that hash collisions in a single module do not occur in CoMI.

    comi:id  16;

But these numbers do not exist any any modules and so they cannot be relied on in CoMI.

Not really.

I don't know how mixed the modules will get in CoMI.
In NETCONF, modules tend to use augment a lot, so the nodes from 'module-A' may be nested nodes.
Retrieving just the augmenting nodes and ignoring the augmented nodes would likely be operationally useless.
e.g., when augmenting a list, the key leafs are needed to identify the instance.

I would like to keep CoMI extremely simple.
More bells and whistles means increased
memory and CPU requirements. I am glad CORE and 6TISCH WGs are so concerned about constrained resources.
Only the most important bells and whistles will get standardized.

Andy

>> the leading zeroes from the hash). As specified in section 5.4. a 
>> hash value of 0x0000007e will be encoded as AAAA_, but if the leading 
>> A’s are elided it will be simply _
>>

---

From: Andy Bierman <[email-masked]>
Date: Wed, 3 Jun 2015 14:37:30 -0700

> Agreed but how this is implemented.

By checking the YANG hashes before it is published,
similar to checking the YANG syntax with a YANG compiler.

> I understand but I assume that a module can be re-published with the added statement when require. If a CoMI implementer try to implement a module containing a clash, he can look for a version of t…

I think the ad-hoc id assignments are pointless because they are not
universal.  The only thing that can be counted on is the module name
and the XPath absolute path expression of each object.

> In my document " draft-vanderstok-core-comi-06 - MV.docx ", I propose a new definition to address this.

What is it?
IMO you should post comments to the mailing list not markup
to a word document.  It is too hard to follow.

> Module Context: A module context is composed of all data nodes, notifications and RPCs defined in a YANG module and included sub-modules or added to them using the augment YANG statement.

Almost -- this is defined in RFC 6020 -- the augmenting nodes are
not part of the augmented module.  They will be returned in
NETCONF and RESTCONF retrieval operations for an ancestor node
but they are not part of the augmented module.  Only objects
defined in the module namespace are part of the module.

Andy

---

From: Alexander Pelov <[email-masked]>
Date: Thu, 4 Jun 2015 09:02:09 +0200

Hi Andy,

> Le 3 juin 2015 à 23:37, Andy Bierman <[email-masked]> a écrit :
> 

> and the XPath absolute path expression of each object.

Having a way of re-mapping (or aliasing) an ID to a locally-understood/negotiated ID is really important in some networks where the radio resources are extremely limited. If you are able to use IDs 1-23 and you have URL hash compression, you save 8 bytes (using standard CBOR, no changes here whatsoever). For IDs 24-63 you save 7 bytes, 63-255 - 6 bytes (without doing any particular treatment). 

When using a general hash function, the probability of getting a hash value < 255 is 10E-7, and choosing an URI that statically gives you such ID simply sounds wrong (and unrealistic).

Two options are then possible here:
1) Leave this out of CoMI
2) Have a minor accommodation, and require no additional resources on the devices

Both require defining hash clash handling mechanism, so here the entire discussion on how to handle this is pertinent.

For me, 2) can be implemented in two ways:
2.a) Explicit ID re-mapping (or aliasing) through a (external) file. Here, it would be nice to have a way to specify the URI of the file, e.g. on the server or on a remote location. Something like the mod.uri . 
2.b) Explicit ID re-mapping (or aliasing) in the YANG module (e.g. through the "id" option mentioned by Michel or maybe with an extension, e.g. comi:id)

Note that if this is considered hash aliasing (we forget the re-mapping), then the behavior can be very simple, e.g.:
It is up to the implementer to define the hash aliases, if any. In case of clash with a YANG hash value, the alias must be ignored. In case of clash with another alias, both aliases are ignored. 

=======================
CoMI can decide on a small concession for aliasing uses - say that YANG string hash values MUST be >= 64 (256?). The compiler can check that all hashes fulfill this before publishing. This way, there is even NO need to add any statement about alias clashes. It would be up to the implementer to chose the appropriate way of aliasing (if any). Aliases would be local in scope.

What do you think about it? 
=======================

Best,
Alexander

---

From: peter van der Stok <[email-masked]>
Date: Thu, 04 Jun 2015 11:04:10 +0200

Hi Alexander,

Thanks for your comments; it is nice to see a growing interest in CoMI.
I like to go back to the start of this thread, as you formulate it 
below.
Thanks for the typo.

If I understand correctly, you want the server to point to URI where 
possible hash clash information of even hash optimization is stored at a 
server.
Personally, I don't mind that the /mg/yang.hash resource is used for 
that.

The syntax of the file contents will become more complex than it is now. 
Good news: it needs to be changed anyway, given the hash clash solution 
discussion with 6tisch people.
Contents and syntax (JSON format?) of the file may turn out to be a 
subject without limits.

For the moment I like to write down a new version 7 of the CoMI draft 
containing the rehash approach for hash clashes discussed with 6tisch 
and described in this thread by Andy.
I propose to continue the file contents discussion independent of the 
current clash rehash solution.

Greetings,

Peter

Alexander Pelov schreef op 2015-06-03 17:00:
> Even though hash collisions should happen rarely (if ever), there may
> be cases when it could be inevitable, and as such, it may be
> interesting to cache the clash_file somewhere on a remote server to
> avoid sending it over the constrained wireless interface. It could be
> used for other cases, where some frequently used resources are
> re-hashed for performance purposes. Maybe it would be interesting to
> allow the server to specify a remote location for the clash file? For
> static collisions or optimizations it could be that the YANG
> definition includes alternative hash values, which could be used in
> addition to the normal ones.
> 
> Another point (which is unrelated), is to elide the leading "A"s from
> the base64 representation of the hash in the URL (which corresponds to
> eliding the leading zeroes from the hash). As specified in section
> 5.4. a hash value of 0x0000007e will be encoded as AAAA_, but if the
> leading A’s are elided it will be simply _
> 
> This could pose problems only if in the future the size of the hash
> values is increased from 30 bits to some other (higher) value.
> 
> A typo (?), which may be already corrected, in the example of mod.uri
> on Page 26: the two responses have different scheme "mod.uri" and
> "moduri"
>

---

From: peter van der Stok <[email-masked]>
Date: Thu, 04 Jun 2015 11:28:43 +0200

Hi Alexander,

Your suggestion to standardize a handle to a mechanism to do hash 
aliasing but keep the algorithms out of CoMI seems reasonable to me in a 
first approach.
It is then a matter of interest whether the mechanisms of hash aliasing 
are standardized within CoRE wg or even within CoMI I-D.

However, I think the idea is interesting and like to investigate it 
further; and understand what extra standardization is needed.

Peter

Alexander Pelov schreef op 2015-06-04 09:02:
>>> I understand but I assume that a module can be re-published with the 
>>> added statement when require. If a CoMI implementer try to implement 
>>> a module containing a clash, he can look for a version of this module 
>>> containing the special YANG statement. The likelihood of such clash 
>>> is extremely low and such fallback solution won’t be required often. 
>>> However, the RFC will specify how to resolve such situation if this 
>>> ever happen. This statement can also be used to address the need for 
>>> data node identifier compactness as mentioned by Alexander in its 
>>> original email.
>>> 
>> 
> Having a way of re-mapping (or aliasing) an ID to a
> locally-understood/negotiated ID is really important in some networks
> where the radio resources are extremely limited. If you are able to
> use IDs 1-23 and you have URL hash compression, you save 8 bytes
> (using standard CBOR, no changes here whatsoever). For IDs 24-63 you
> save 7 bytes, 63-255 - 6 bytes (without doing any particular
> treatment).
> 
> When using a general hash function, the probability of getting a hash
> value < 255 is 10E-7, and choosing an URI that statically gives you
> such ID simply sounds wrong (and unrealistic).
> 
> Both require defining hash clash handling mechanism, so here the
> entire discussion on how to handle this is pertinent.
> 
> 2.a) Explicit ID re-mapping (or aliasing) through a (external) file.
> Here, it would be nice to have a way to specify the URI of the file,
> e.g. on the server or on a remote location. Something like the mod.uri
> .
> 2.b) Explicit ID re-mapping (or aliasing) in the YANG module (e.g.
> through the "id" option mentioned by Michel or maybe with an
> extension, e.g. comi:id)
> 
> Note that if this is considered hash aliasing (we forget the
> re-mapping), then the behavior can be very simple, e.g.:
> It is up to the implementer to define the hash aliases, if any. In
> case of clash with a YANG hash value, the alias must be ignored. In
> case of clash with another alias, both aliases are ignored.
> 
> =======================
> CoMI can decide on a small concession for aliasing uses - say that
> YANG string hash values MUST be >= 64 (256?). The compiler can check
> that all hashes fulfill this before publishing. This way, there is
> even NO need to add any statement about alias clashes. It would be up
> to the implementer to chose the appropriate way of aliasing (if any).
> Aliases would be local in scope.
> 
>>> Module Context: A module context is composed of all data nodes, 
>>> notifications and RPCs defined in a YANG module and included 
>>> sub-modules or added to them using the augment YANG statement.
>>> 
>> 
>>>> Your solution works only if hash clashes within a module are 
>>>> resolved at design time. If not, the error message will return the 
>>>> following information which is not sufficient to resolve which data 
>>>> node is which.
>>>> 
>>>>   { "1234": [
>>>>          "module-A":"4567",
>>>>          "module-A":"6523"
>>>>      ]
>>>>   }
>>>> 
>>> 
>>> 
>>> The original proposal had a 3-tuple { module-name, path-expr, 
>>> new-hash } We discussed some optimizations in the 6TISCH call to 
>>> reduce the size of the path-expr and still not require the client to 
>>> store all the path strings, such as store a few bytes from the path 
>>> strings like 6th and 11th bytes (Avoid the common ietf- prefix for 
>>> many module names).
>>> 
>>> They need to be resolved at design time.  If this is not possible, 
>>> then a standard rehash algorithm should be used so the collision is 
>>> resolved in a predictable way.  e.g., try appending "_", then try 
>>> "__", etc.
>>>> For nodes / networks which can afford a slitely larger payload (IEEE 
>>>> 802.15.4g for example) but want to avoid the processing and storage 
>>>> associated with the porposed error message, are you opposed to the 
>>>> support of an optional CoAP query parameter use carry the module 
>>>> name or module ID ?
>>>> 
>>>> For example:
>>>> 
>>>> Are you oposed to the use of the select query parameter instead of 
>>>> the
>>>> URI to identify data node(s)
>>>> 
>>> 
>>> Not really.
>>> 
>>>>> At the last 6TiSCH bi-weekly call, 4 solutions have been presented 
>>>>> to the hash clashes problem. Two based on a rehash table and two 
>>>>> based on a new CoAP query parameter carrying the module name or 
>>>>> registered module ID. This query parameter is used to select 
>>>>> unambiguously the targeted data node(s) assuming that any hash 
>>>>> collision within a module have been resolved at design time. You 
>>>>> can find this presentation in attachment.
>>>>> 
>>>>> Both of your proposals make sense to me but I still hope that the 
>>>>> clash file can be avoided or be optional.
>>>>> 
>>>> 
>>>> 
>>>> The CoMI authors have discussed a solution which I will try to write 
>>>> up today where a special "hash-clash" error is returned by CoMI if a 
>>>> rehash is needed.
>>>> 
>>>>> 1) Caching the yang clash file on a remote server Even though hash 
>>>>> collisions should happen rarely (if ever), there may be cases when 
>>>>> it could be inevitable, and as such, it may be interesting to cache 
>>>>> the clash_file somewhere on a remote server to avoid sending it 
>>>>> over the constrained wireless interface. It could be used for other 
>>>>> cases, where some frequently used resources are re-hashed for 
>>>>> performance purposes. Maybe it would be interesting to allow the 
>>>>> server to specify a remote location for the clash file? For static 
>>>>> collisions or optimizations it could be that the YANG definition 
>>>>> includes alternative hash values, which could be used in addition 
>>>>> to the normal ones.
>>>>> 
>>>>> A typo (?), which may be already corrected, in the example of 
>>>>> mod.uri on Page 26: the two responses have different scheme 
>>>>> "mod.uri" and "moduri"
>>>>>

---

From: Andy Bierman <[email-masked]>
Date: Thu, 4 Jun 2015 05:56:36 -0700

> Having a way of re-mapping (or aliasing) an ID to a locally-understood/negotiated ID is really important in some networks where the radio resources are extremely limited. If you are able to use IDs…

Numbering objects in a schema and numbering radios in a network are different.
There is no way to assign unique numbers to all objects.
This is not something IANA should have to do, and not something
vendors will do.  It was already too difficult to simply number the modules,
let alone all schema nodes in all modules.

> 2) Have a minor accommodation, and require no additional resources on the devices

The 6TISCH argument for small clients is important.
If there is an alterate-ID mapping for every server, then the
client has a different number for every object (different on
every server).  This seems like a lot of additional memory
if there are 1000s of servers.

I don't agree that 1 byte object identifiers are needed,
The 30 bit hash seems small enough.

Andy

---

From: Alexander Pelov <[email-masked]>
Date: Thu, 4 Jun 2015 16:15:05 +0200

Hi Peter,

> Le 4 juin 2015 à 11:28, peter van der Stok <[email-masked]> a écrit :
> 
> Hi Alexander,
> 
> Your suggestion to standardize a handle to a mechanism to do hash aliasing but keep the algorithms out of CoMI seems reasonable to me in a first approach.
> It is then a matter of interest whether the mechanisms of hash aliasing are standardized within CoRE wg or even within CoMI I-D.
> 
> However, I think the idea is interesting and like to investigate it further; and understand what extra standardization is needed.

If the YANG hash compression + reservation of < 256 hash values is accepted to CoMI, which I believe can be achieved with minimal modification and at no cost in terms of implementation or resource requirements, there are two main approaches to implementing the hash aliases:
1) Independent of the YANG schema
  1.a) From a URI, which can be local (e.g /mg/yang.hash-alias or including the syntax in the future /mg/yang.hash with special provisioning of having aliased) or remote
  1.b) Reserve one of the alias values for the alias mapping resource (e.g. /mg/0)

2) Dependent on the YANG schema
  2.a) By using a leaf in the schema (e.g. alternate_hash = 1, or comi:alternate_hash = 3)
  2.b) By using an URI specified in the schema (not sure if this would be very useful though)

I would say that it should be possible to have the aliasing independent on the schema, as a given operator may decide on a different operation modes for the same YANG schema. 

==========================
1.a)
If there is a dedicated file (e.g. /mg/yang.hash-alias), it could be updated by the client if necessary. It would be nice to have a way to make sure that the server and the client are using the same aliases, which can be implemented with the help of a hash over the file + a token which is set upon the creation of the file (by the client or upon deploying the object). This way, a client can query the server if needed (e.g. GET /mg/yang.hash-alias?keys=hash,token ).

So, to sum up, the simplest solution would be to have a new resource /mg/yang.hash-alias which is empty. A client may create the resource with a POST operation, where the format is (YANG hash : alias).

Afterwards, the client can use the aliases to issue requests (example at the end). This is the bare minimum, but when you have a device (client) which manages many servers it is sufficient. 

==========================
1.b) 
In CoMI reserve all IDs < 256. In a different document specify, that IF you want to use hash aliasing, then if works as following:

The value « 0 » can be used for information on the aliases:
1) GET /mg/0 -> Resource not found -> aliases are unsupported feature, don’t even try (*standard behavior* - nothing to change to a normal CoMI object, as the resource ID 0 is reserved and is inexistent in the YANG hash table).
2) GET /mg/0 -> {} -> aliases are supported, but no alias configuration in place (alternatively, this could return some other info, but the source_uri key is empty
3) GET /mg/0 ->
  3.1) {"source_uri" : "/mg/yang.hash-alias", "hash": 0x127804bc}
  3.2) {"source_uri" : "example.com/mg/yang.hash-alias", "hash": 0x127804bc}

This can actually be achieved in the following way:

In CoMI:
  1) add YANG hash compression to section 5.4. (minor change)
  2) add a statement that hash values 0-255 inclusive are reserved
  3) eventually, mention as a guidance that these could be used for hash aliasing

Afterwards, either add a new document describing hash aliasing, or add a small section, which defines:
1) Data ID = 0 is reserved for the alias file. 
2) IDs 1-255 inclusive are mapped to YANG hashes according to the alias 0

As an alternative, the /mg/0 could directly be the alias file, but I like this to a smaller extent, as a GET /mg/0 will fetch the whole mapping, and not just the ‘yes, aliases are in place, and we’re talking the same thing’.

==========================

I’d go with 1.b) but have no strong objections to the rest.

Best,
Alexander

REQ: POST /mg/0 <http://example.com/mg/0>    (<- this one actually creates a file on the server, e.g. /mg/yang.hash-aliases)
     (Content-Format: application/cbor)

{
   0x387804eb: 1,
   0x15370408: 2
}

RES: 2.01 Created 

REQ: GET example.com/mg/C <http://example.com/mg/C>
       (Content-Format: application/cbor)
 
RES: 2.05 Content (Content-Format: application/cbor)
{
         0x2 : "2014-10-26T12:16:51Z",
}

---

From: Alexander Pelov <[email-masked]>
Date: Thu, 4 Jun 2015 16:35:24 +0200

Hi Andy,

> Le 4 juin 2015 à 14:56, Andy Bierman <[email-masked]> a écrit :
> 

Completely agree.

> if there are 1000s of servers.

This is supposing that the 1000s of servers will require unique mapping.

Imagine a network where all radios are of the same type, and you would like to manage their parameters (e.g. channel, slot, offset). You could alias the channel to ID 1, the slot to ID 2, and operate with these IDs, instead of the full hashes. The point is, the way you do the actual mapping does not have to do anything with CoMI, where only the first 256 values will be considered ‘occupied’. 

If you want to do more advanced things, like having several radios and mapping 1-10 to one of the radios, 11-20 to the second, and so forth - it would be up to you to make sure your aliasing scheme is consistent. A hash could help you identify to who are you talking to (e.g. /mg/0), but I would suppose that if you have a client powerful enough to manage 1000s of devices, it should be able to note somewhere - nodes 1-500 use aliasing A, nodes 501-900 use aliasing B, and the rest don’t support aliasing, so we’ll go the YANG hash road.

And if the client is too limited, then you just don’t use aliases, no harm done.

> The 30 bit hash seems small enough.

This of course depends on the case. I agree with you that in many cases it may be an overkill to try to save a couple of bytes. If, however, your network has long-lived objects (days-months-years), where 90% of your management queries are with 20-30 items, you could significantly reduce the trafic by aliasing these once to more convenient values. 

The aliases could save in a GET 8 bytes - 4 from the request hash, and 4 from the CBOR representation (assuming IDs < 24). 

The reasoning goes the following way: hashes and CBOR have a really nice, short representations for the first hash values (<256). Reserve them, just so that we don’t have to do clash management with the YANG schemes, and leave to the operator/network manager the way to handle them.

Best,
Alexander

---

From: Michel Veillette <[email-masked]>
Date: Thu, 4 Jun 2015 15:21:41 +0000

Hi Andy

About "Almost -- this is defined in RFC 6020 -- the augmenting nodes are not part of the augmented module.  They will be returned in NETCONF and RESTCONF retrieval operations for an ancestor node but they are not part of the augmented module.  Only objects defined in the module namespace are part of the module."

Understood, this is why my contribution introduce the new term, "Module context". Augmenting nodes are not part of the augmented module but, as defined, are part of the "Module context". This new concept was require to minimize the overhead when retrieving the complete datastore containing multiple module contexts with some having augmenting nodes.

   RES: 2.05 Content (Content-Format: application/cbor)
   {
      "a" : {
         365257235 : "leafA1 value",
         702149626 : {
            215993329 : "leafA2 value",
            962191682 : "leafC1 value"                    # Augmenting data node
         },
         829222983 : "leafS value"                         # Sub-module data node
      },
      "b" : {
         289564696 : "leafB value"
      },
      "c" : {
         993533527 : "leafC2 value"
      }
   }

Michel Veillette

-----Original Message-----
From: Andy Bierman [mailto:[email-masked]] 
Sent: 3 juin 2015 17:38
To: Michel Veillette
Cc: Alexander Pelov; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

> Agreed but how this is implemented.

By checking the YANG hashes before it is published, similar to checking the YANG syntax with a YANG compiler.

> I understand but I assume that a module can be re-published with the added statement when require. If a CoMI implementer try to implement a module containing a clash, he can look for a version of t…

I think the ad-hoc id assignments are pointless because they are not universal.  The only thing that can be counted on is the module name and the XPath absolute path expression of each object.

> In my document " draft-vanderstok-core-comi-06 - MV.docx ", I propose a new definition to address this.

What is it?
IMO you should post comments to the mailing list not markup to a word document.  It is too hard to follow.

> Module Context: A module context is composed of all data nodes, notifications and RPCs defined in a YANG module and included sub-modules or added to them using the augment YANG statement.

Almost -- this is defined in RFC 6020 -- the augmenting nodes are not part of the augmented module.  They will be returned in NETCONF and RESTCONF retrieval operations for an ancestor node but they are not part of the augmented module.  Only objects defined in the module namespace are part of the module.

Andy

>> the URI to identify data node(s)
>>
>
> Not really.
>
> More bells and whistles means increased memory and CPU requirements. I 
> am glad CORE and 6TISCH WGs are so concerned about constrained resources.
>> GET example.com/mg?select=Y8d7s,NKHaA GET 
>> example.com/mg?select=ietf-6tisch-mac(Y8d7s,NKHaA)
>>
>>> hash value of 0x0000007e will be encoded as AAAA_, but if the

---

From: Andy Bierman <[email-masked]>
Date: Thu, 4 Jun 2015 08:36:34 -0700

Hi,

I do not agree that the module name or a module ID needs to be
used at all in the object identifier.  Each object has a YANG Hash.
Retrieving a node, whether by hash ID or module-ID.hash will
cause all descendant nodes to be included in the retrieval,
even if they are augmenting nodes from another module.

YANG data is hierarchical.
The modules are just packaging for definitions.
They do not define any containment or structure
within the conceptual schema tree.

Andy

---

From: Michel Veillette <[email-masked]>
Date: Thu, 4 Jun 2015 16:04:26 +0000

Hi Andy,

Just to be sure about the scope of your comment.

You say " Each object has a YANG Hash", yes but this is not a uniquely object identify.
Did we reach a consensus about using the  " YANG hash" + "module name or a module ID" as a unique data node (object) identifier?

This can take the form of a rehash table available on the CoMI server or elsewhere (" YANG hash", "module name or module ID", "rehash or alias")
This can take the form of an error message containing (" YANG hash", "module name or module ID", "rehash or alias")
This can take the form of a fully qualified identifier within the request or payload (REQ: GET example.com/mg?select(<module name>.<YANG hash>)

Michel Veillette

-----Original Message-----
From: Andy Bierman [mailto:[email-masked]] 
Sent: 4 juin 2015 11:37
To: Michel Veillette
Cc: Alexander Pelov; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

Hi,

I do not agree that the module name or a module ID needs to be used at all in the object identifier.  Each object has a YANG Hash.
Retrieving a node, whether by hash ID or module-ID.hash will cause all descendant nodes to be included in the retrieval, even if they are augmenting nodes from another module.

YANG data is hierarchical.
The modules are just packaging for definitions.
They do not define any containment or structure within the conceptual schema tree.

Andy

On Thu, Jun 4, 2015 at 8:21 AM, Michel Veillette <[email-masked]> wrote:
>> More bells and whistles means increased memory and CPU requirements. 
>> I am glad CORE and 6TISCH WGs are so concerned about constrained resources.
>>>> This, however, could be very useful when having to frequently 
>>>> access the same resource. In such case, the server/network operator 
>>>> may decide to re-hash the URI to a shorter version, e.g. the

---

From: Andy Bierman <[email-masked]>
Date: Thu, 4 Jun 2015 09:30:19 -0700

> Did we reach a consensus about using the  " YANG hash" + "module name or a module ID" as a unique data node (object) identifier?

I don't know if any consensus has been reached.
IMO the YANG hash is all that is needed.
There is a small probability that the hash is not unique.
If this happens, the rehash is needed.

If a specific value maps to more than 1 object on a server,
then that value triggers a rehash if the client uses it.
I prefer to send the rehash info as an error response only
when needed.

I prefer to limit optional features so the server code can be
as small as possible.

There is a problem with GET.
Let's say the data model is a simple container with 2 leafs:

mod1:

   container A {
       leaf A1 { type string; }
   }

mod2:

    augment /mod1:A {
       leaf A2 { type string; }
    }

What if the server does a GET on /A but A1 and A2 have a hash collision?
The server is supposed to return

    {"A": {
       "A1":"foo",
       "A2":"bar"
       }
     }

But the original hashes will not work:

   {"2345": {
      "1234":"foo",
      "1234":"bar"
      }
   }

The client never requested A1 or A2 directly but will
still need to know the rehash info for these nodes.

The server could return the rehash info in the GET response,
or return some flag that the client needs to retrieve the rehash info.
Even if A1 and A2 are not siblings, the rehash info needs to be
sent because the server will remap the collided hash value "1234".

In this case it might be useful for the server to return a "long form"
of the node, which includes the requested value and the rehash info.
This part of your plan I like (if I understand it).  But requiring
the server to accept the long-form as input is not needed.

Andy

---

From: Alexander Pelov <[email-masked]>
Date: Thu, 4 Jun 2015 18:32:05 +0200

Hi all,

Just a short idea for the hash clash issue. 

The 30 bits, which are reserved for the YANG hash can have a structure :
1 bit = structured YANG hash (0) or pure YANG hash (1)
   If 0 (structured), then :
       1 bit = managed (0) or non-managed (1)
       18 bits = module ID
       10 bits = item ID
   If 1 (non-structured), then:
       29 bits = least significant 29 bits of the YANG hash.
     
In case of clash, send error, the full URI must be used (which should be with extremely low probability)
This way, in the worst case you are limiting your hash value from 30 to 29 bits (and thus slightly increasing the clash probability).

If structured YANG hash, then:
   0 = managed (e.g. IETF-assigned module ID)
      18 bits = the ID of the module (1, 2, 3, …., 262143) assigned incrementally by IANA (0 reserved for special usage)
      10 bits = the ID of the item in the module, incremented at each item (1..1023 - hope this is sufficient for all cases).
   1 = unmanaged
       18 bits = ID of module, by default the 18 less significant bits of the module. It is up to the user to assure that there are no clashes with the names of the other modules on the device
       10 bits = ID of the item. It is up to the implementer to ensure that there are no clashes within the module. 

Two possible drawbacks of this approach, however:
  - For managed IDs you will probably have to keep both the YANG hash and the managed full ID.
  - In case of unmanaged IDs you increase the probability of collision of 1E-9 to 2E-9.

Best,
Alexander

> Le 4 juin 2015 à 18:04, Michel Veillette <[email-masked]> a écrit :
> 
> Hi Andy,
>

---

From: Michel Veillette <[email-masked]>
Date: Thu, 4 Jun 2015 17:09:30 +0000

Hi Andy

I was assuming that YANG hash values was unique within a module context (Module data nodes + augmenting data nodes).
If this is not the case, we might have a more important issue than what you are describing.

Let assume that we have two modules augmenting the same container.
Let also assume that the YANG hash of data node A2 and A3 are the same.

mod1:
   container A {                                 # YANG hash 2345
       leaf A1 { type string; }               # YANG hash  1234
   }

mod2:
    augment /mod1:A {
       leaf A2 { type string; }              # YANG hash  5678
    }

mod3:
    augment /mod1:A {
       leaf A3 { type string; }              # YANG hash  5678
    }

Finally, let assume that the CoMI server implements mod1 and mod2 and the CoMI client implements all three modules.

   { 2345: {
       1234 : "foo",
       5678 : "bar"
      }
   }

How the CoMI client known the meaning of "bar" ?

Michel Veillette

-----Original Message-----
From: Andy Bierman [mailto:[email-masked]] 
Sent: 4 juin 2015 12:30
To: Michel Veillette
Cc: Alexander Pelov; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

If a specific value maps to more than 1 object on a server, then that value triggers a rehash if the client uses it.
I prefer to send the rehash info as an error response only when needed.

I prefer to limit optional features so the server code can be as small as possible.

mod1:

   container A {
       leaf A1 { type string; }
   }

mod2:

    augment /mod1:A {
       leaf A2 { type string; }
    }

    {"A": {
       "A1":"foo",
       "A2":"bar"
       }
     }

   {"2345": {
      "1234":"foo",
      "1234":"bar"
      }
   }

The client never requested A1 or A2 directly but will still need to know the rehash info for these nodes.

The server could return the rehash info in the GET response, or return some flag that the client needs to retrieve the rehash info.
Even if A1 and A2 are not siblings, the rehash info needs to be sent because the server will remap the collided hash value "1234".

In this case it might be useful for the server to return a "long form"
of the node, which includes the requested value and the rehash info.
This part of your plan I like (if I understand it).  But requiring the server to accept the long-form as input is not needed.

---

From: Michel Veillette <[email-masked]>
Date: Thu, 4 Jun 2015 17:17:43 +0000

Hi Alexander

In this proposal, the "Managed vs unmanaged" flag is the LSB or MSB?
If it's the LBB, we can use your compression proposal to save URI and payload space.

I like this approach.

Michel Veillette

-----Original Message-----
From: Alexander Pelov [mailto:[email-masked]] 
Sent: 4 juin 2015 12:32
To: Michel Veillette
Cc: Andy Bierman; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

Hi all,

     
In case of clash, send error, the full URI must be used (which should be with extremely low probability) This way, in the worst case you are limiting your hash value from 30 to 29 bits (and thus slightly increasing the clash probability).

Best,
Alexander

> This can take the form of a rehash table available on the CoMI server 
> or elsewhere (" YANG hash", "module name or module ID", "rehash or 
> alias") This can take the form of an error message containing (" YANG 
> hash", "module name or module ID", "rehash or alias") This can take 
> the form of a fully qualified identifier within the request or payload 
> (REQ: GET example.com/mg?select(<module name>.<YANG hash>)
> 
> Michel Veillette
>>>>> representation of the hash in the URL (which corresponds to 
>>>>> 5.4. a hash value of 0x0000007e will be encoded as AAAA_, but if 
>>>>> the leading A’s are elided it will be simply _
>>>>> 
>>>>> access the same resource. In such case, the server/network 
>>>>> operator may decide to re-hash the URI to a shorter version, e.g. 
>>>>> the /sys:system-state/sys:clock/sys:current-datetime could be 
>>>>> remapped from 0x15370408 and VNwQI to 0x1 and _
>>>>> 
>>>>> Best,
>>>>> Alex
>>>>> 
>>>>> PS.

---

From: Andy Bierman <[email-masked]>
Date: Thu, 4 Jun 2015 10:31:14 -0700

> How the CoMI client known the meaning of "bar" ?

Because this response message that you are describing is not
allowed to happen.  The server knows that hash "5678"
is a clash and does not send this hash out as normal.
The value "5678" is no longer valid on this server.
Both nodes that hashed to "5678" need to be re-hashed.

NETCONF allows in-line <rpc-error> to be returned for get*.
Something similar could be done for CoMI.

Andy

---

From: Michel Veillette <[email-masked]>
Date: Thu, 4 Jun 2015 17:45:15 +0000

Hi Andy

About " The server knows that hash 5678 is a clash"

In the described scenario, the CoMI server implement only mod1 and mod2 and in this context the hash value 5678 is not a clash. On the client side, the CoMI client implement mod1, mod2 and mod3 and is confused about the received data.

Michel Veillette

-----Original Message-----
From: Andy Bierman [mailto:[email-masked]] 
Sent: 4 juin 2015 13:31
To: Michel Veillette
Cc: Alexander Pelov; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

> How the CoMI client known the meaning of "bar" ?

Because this response message that you are describing is not allowed to happen.  The server knows that hash "5678"
is a clash and does not send this hash out as normal.
The value "5678" is no longer valid on this server.
Both nodes that hashed to "5678" need to be re-hashed.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 4 Jun 2015 12:02:43 -0700

> In the described scenario, the CoMI server implement only mod1 and mod2 and in this context the hash value 5678 is not a clash. On the client side, the CoMI client implement mod1, mod2 and mod3 and…

But hash collisions are only within a server.
The client will get the ietf-yang-library inventory for the
server and see that mod3 is not supported by that server.

The client cannot treat rehash info as global.
It has to remember the rehash info from each server.

Andy

---

From: Alexander Pelov <[email-masked]>
Date: Thu, 4 Jun 2015 21:26:51 +0200

Hi Michel,

> Le 4 juin 2015 à 19:17, Michel Veillette <[email-masked]> a écrit :
> 
> Hi Alexander
> 

Yes, it is the LSB, so managed IDs should compress well. Not sure if the 10 bits for user item ID are sufficient / too much compared to the typical/max size of modules.

Alexander

---

From: Michel Veillette <[email-masked]>
Date: Thu, 4 Jun 2015 19:33:17 +0000

I don't remember in the CoMI draft a method which can be used by CoMI clients to get the list of YANG modules supported by CoMI servers.
Do you means that CoMI clients need be pre-loaded with the list of YANG modules implemented by each CoMI server they might need to interact with?
How we will deal with dynamic change in these list of YANG modules?

Instead, should we require that augmented data nodes are always identified using a fully qualified identifier?
(e.g. " ietf-interfaces.jVLxJ" or the 18 bits  module ID + 10 bits = data node ID as proposed by Alexander)

Michel Veillette

-----Original Message-----
From: Andy Bierman [mailto:[email-masked]] 
Sent: 4 juin 2015 15:03
To: Michel Veillette
Cc: Alexander Pelov; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

But hash collisions are only within a server.
The client will get the ietf-yang-library inventory for the server and see that mod3 is not supported by that server.

Andy

---

From: Michel Veillette <[email-masked]>
Date: Thu, 4 Jun 2015 19:36:08 +0000

1024 data nodes within a single YANG module seem plenty.
I'm find with this proposed allocation of bits.

Michel Veillette

-----Original Message-----
From: Alexander Pelov [mailto:[email-masked]] 
Sent: 4 juin 2015 15:27
To: Michel Veillette
Cc: Andy Bierman; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

Hi Michel,

Alexander

>>>  RES: 2.05 Content (Content-Format: application/cbor)  {
>>>     "a" : {
>>>>>> [email-masked] www.trilliantinc.com
>>>>>> 
>>>>> 
>>>>> 
>>>>> Andy
>>>>> 
>>>>>>

---

From: Andy Bierman <[email-masked]>
Date: Thu, 4 Jun 2015 12:39:39 -0700

> How we will deal with dynamic change in these list of YANG modules?

comi-06, pg 26:

   Therefore, it is recommended that details about management objects
   are discovered following the RESTCONF protocol.  The YANG module
   information is stored in the "ietf-yang-library" module
   [I-D.ietf-netconf-restconf].  The resource "/mg/mod.uri" is used to
   retrieve the location of the YANG module library.

I think Carsten suggested that an additional attribute
(last-changed-time or an entity-tag) be maintained by the server
so it can be observed by the client, and get notified if the YANG
module list changes for the specific server.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 4 Jun 2015 15:19:18 -0700

>        29 bits = least significant 29 bits of the YANG hash.

I do not really want to change the YANG Hash from 30 bits to 29 bits.

Having a different numbering scheme for every object seems too complicated.
I would add the constraint that the server must use the same numbering scheme
for all objects, so this can be discovered via .well-known capabilities, and
all 30 bits can be used in a YANG Hash.

>   - In case of unmanaged IDs you increase the probability of collision of 1E-9 to 2E-9.

IMO only managed mode is needed. The IANA registry (that will
be created) should be open to vendors and other SDOs.
This registry just assigns a module number to a module name.

The 10-bit mystery encoding does not seem very interoperable.
It should be standard.  I think the YANG "id" extension to
number each object manually will work.

Andy

---

From: Michel Veillette <[email-masked]>
Date: Thu, 4 Jun 2015 22:36:41 +0000

=== About "IMO only managed mode is needed."

Totally agree

=== About "The 10-bit mystery encoding does not seem very interoperable."

If we " just assigns a module number to a module name", a unique data node identifier need to be formed by this module number + some sort of data node number. The 30 bits data node identifier similar to scheme proposed by Alexander seen to work fine. (e.g. 20 bits module ID, 10 bits data node ID)

Michel Veillette

-----Original Message-----
From: Andy Bierman [mailto:[email-masked]] 
Sent: 4 juin 2015 18:19
To: Alexander Pelov
Cc: Michel Veillette; [email-masked]; [email-masked]
Subject: Re: [core] CoMI remote server clash file and YANG hash in URL

Having a different numbering scheme for every object seems too complicated.
I would add the constraint that the server must use the same numbering scheme for all objects, so this can be discovered via .well-known capabilities, and all 30 bits can be used in a YANG Hash.

> In case of clash, send error, the full URI must be used (which should 
> be with extremely low probability) This way, in the worst case you are limiting your hash value from 30 to 29 bits (and thus slightly increasing the clash probability).
>

IMO only managed mode is needed. The IANA registry (that will be created) should be open to vendors and other SDOs.
This registry just assigns a module number to a module name.

The 10-bit mystery encoding does not seem very interoperable.
It should be standard.  I think the YANG "id" extension to number each object manually will work.

Andy

>> the form of a fully qualified identifier within the request or 
>> payload (REQ: GET example.com/mg?select(<module name>.<YANG hash>)
>>
>> Michel Veillette
>>>>>> 2) Compressing the URL representation of YANG hashes Another 
>>>>>> point (which is unrelated), is to elide the leading "A"s from the 
>>>>>> base64 representation of the hash in the URL (which corresponds 
>>>>>> to eliding the leading zeroes from the hash). As specified in 
>>>>>> section 5.4. a hash value of 0x0000007e will be encoded as AAAA_, 
>>>>>> but if the leading A’s are elided it will be simply _
>>>>>>

---

From: Alexander Pelov <[email-masked]>
Date: Fri, 5 Jun 2015 08:38:11 +0200

Hi Michel, Andy,

> 
> Le 5 juin 2015 à 00:36, Michel Veillette <[email-masked]> a écrit :
> 

If the managed mode is all that is necessary (which also seems reasonable to me), then there could be a deterministic assignment of the IDs for the data nodes in a module, for example for each module, start from 1 and increase at each data node in a breadth-first enumeration of the nodes. At the end of this mail I’ve tried to put a short example based on the YANG scheme of 6top.

Also, if the data node ID is 0, then the YANG hash ID will be simply the ID of the module, which could then be addressed (maybe for some meta-info?).

Alexander

Example:
id = data node ID
id=1  typedef nodeaddresstype {}

id=2  list CellList {
id=3  leaf CellID {}
id=4     leaf SlotframeID {}
id=5     leaf SlotOffset {}
id=6     leaf ChannelOffset {}
id=7     leaf LinkOption {}
id=8     leaf LinkType {}
id=9     leaf CellType {}
id=10    leaf NodeAddress {}
id=11    leaf TrackID {}

id=12    container Statistic {
id=13        leaf NumOfStatistic {}
id=14            list MeasureList {
id=15                leaf StatisticsMetricsID{}
id=16                leaf StatisticsValue{}
                  }
              }
          }
          …
      }