Subject: [netmod] iana-if-type.yang has multiple revisions with the same date

From: William Lupton <[email-masked]>
Date: Tue, 1 Mar 2022 12:53:45 +0000

All,

Sorry if (as is quite likely) this is a duplicate.

I noticed from
https://yangcatalog.org/private-page/BBFYANGPageCompilation.html that
there's a (long-standing?) problem in iana-if-type.yang
<https://www.iana.org/assignments/yang-parameters/[email-masked]>:
it has multiple revision statements with the same date:

  revision 2018-06-28 {
    description
      "Registered ifType 294.";
  }

  revision 2018-06-28 {
    description
      "Registered ifType 293.";
  }

This has presumably happened as a result of an automated update script that
doesn't check for this case (*)? From a quick scan, I didn't see anything
in RFC 7950 banning duplicate revision dates, but RFC 8407 section 4.8 says
"*If the module contents have changed, then the revision date of that new
module version MUST be updated to a date later than that of the previous
version*" and of course yangdump-pro is checking this.

I think that this should be fixed. What's the best way to achieve this?

Thanks,
William

(*) In the rare event that multiple changes are made in the same day,
perhaps the second change should be (strictly wrongly) assigned to the
following day. In theory this could cause revision dates to run far into
the future but in practice I don't think this will happen :).

---

From: Andy Bierman <[email-masked]>
Date: Tue, 1 Mar 2022 06:49:10 -0800

On Tue, Mar 1, 2022 at 4:54 AM William Lupton <[email-masked]>
wrote:

> This has presumably happened as a result of an automated update script
> that doesn't check for this case (*)? From a quick scan, I didn't see
> anything in RFC 7950 banning duplicate revision dates, but RFC 8407 section
> 4.8 says "*If the module contents have changed, then the revision date of
> that new module version MUST be updated to a date later than that of the
> previous version*" and of course yangdump-pro is checking this.
>

I think this issue should be resolved as well.
The YANG library identifies each module by a [name, date] tuple.
The <get-schema> operation uses this tuple to identify a specific revision
to retrieve.
The import-by-revision mechanism uses this tuple to identify a specific
revision to import.

If this [name, date] tuple is not unique, then it cannot be mapped to a
single module revision.

Note that with multiple release trains and the new SERMVER, it is likely
that multiple [name, date, label] tuples resolve to the same [name, date]
pair,
making the uniqueness problem even worse.

This is quite significant if a client reads the YANG library from a server
and decides it already has the module cached (based on the [name, date]
tuple,
as defined in the standard.  Then it will not use the <get-schema> operation
to retrieve the module from the server.

YANG artifacts and SID files also rely on this [name, date] tuple
uniqueness.

Even with the new versioning drafts, it is impossible for the client to know
"Do you mean the REAL module foo, version xxxx-xx-xx, or your private
version?"

> William

 Andy

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: William Lupton <[email-masked]>
Date: Thu, 3 Mar 2022 09:24:52 +0000

Thanks Andy. What is the next step? Should I (or someone else) email
[email-masked], or can we assume that the relevant IANA person will already
have seen this discussion?

---

From: Andy Bierman <[email-masked]>
Date: Thu, 3 Mar 2022 08:56:05 -0800

On Thu, Mar 3, 2022 at 1:25 AM William Lupton <[email-masked]>
wrote:

> [email-masked], or can we assume that the relevant IANA person will
> already have seen this discussion?
>
>
It seems that RFC 8407 already says what to do (use a different
revision-date).
We combine the revision-stmt so there is only 1 revision entry instead of 2.

It is too late to do anything about this module.

I am interested in the OPS issues:
The client MUST be able to produce the same[*] schema tree as the server
in order to have an accurate model of the server's YANG API.

 1) server uses implementation-specific mechanisms (e.g. search path)
     to select the modules it will advertise in its yang-library
 2) client reads the yang-library, which provides all the [name,date] tuples
     and other info needed
 3a) client can use cached yang-library data and locally obtained YANG files
 3b) client can use <get-schema> (IFF supported by the server) to retrieve
the YANG files

[*] same can mean a later revision if specific schema definitions have not
changed

Issues:

1) Is there ANY uniqueness guarantee that [name, date] is GLOBALLY unique.
A: Yes according to RFC 7950, but not really in implementations.
A: No, if revision-label is added and the same revision-date is used in
multiple release trains.

So if a client cannot rely on [name, date] uniqueness, then it does not
really know if
step 3a or step 3b is required.

This is currently a solved problem using proprietary means
(e.g., client hacked to know which one, based on server testing).

But now there are more system components, not just a server,
such as YANG Data Instance Files and YANG SID Files.
If the [name, date] tuples are not globally unique here,
then these standards do not work.

Andy

---

From: William Lupton <[email-masked]>
Date: Thu, 3 Mar 2022 17:45:25 +0000

> It is too late to do anything about this module.

This module is republished every time a new ifType is added. Are you saying
that it would be unacceptable to collapse the duplicate revisions next time
it's updated? If so then we will live with this FOR EVER!

---

From: Jürgen Schönwälder <[email-masked]>
Date: Thu, 3 Mar 2022 20:00:08 +0100

The obvious thing to do in this particular case (where there are only
allocations of new values) is to collapse the revisions and to move
on. Slightly better would be to ensure this does not happen again.

/js

-- 
Jürgen Schönwälder              Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>

---

From: Benoit Claise <[email-masked]>
Date: Thu, 3 Mar 2022 19:24:13 +0000

+1  to Jürgen point of view.

Regards, Benoit

From:Jürgen Schönwälder <[email-masked]>
To:William Lupton <[email-masked]>
Cc:NetMod WG <[email-masked]>
Date:2022-03-03 20:01:06
Subject:Re: [netmod] iana-if-type.yang has multiple revisions with the same date

/js

---

From: William Lupton <[email-masked]>
Date: Fri, 4 Mar 2022 10:14:30 +0000

+1 (not surprisingly). What action? And whose action?

On Thu, 3 Mar 2022 at 19:24, Benoit Claise <[email-masked]> wrote:

> *From:*Jürgen Schönwälder <[email-masked]>
> *To:*William Lupton <[email-masked]>
> *Cc:*NetMod WG <[email-masked]>
> *Date:*2022-03-03 20:01:06
> *Subject:*Re: [netmod] iana-if-type.yang has multiple revisions with the
> same date
>

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Fri, 4 Mar 2022 14:37:18 +0000

Hi William,

I have asked Sabrina in IANA to please publish a new revision with the history fixed.

Regards,
Rob

From: netmod <[email-masked]> On Behalf Of William Lupton
Sent: 04 March 2022 10:15
To: Benoit Claise <[email-masked]>
Cc: NetMod WG <[email-masked]>
Subject: Re: [netmod] iana-if-type.yang has multiple revisions with the same date

On Thu, 3 Mar 2022 at 19:24, Benoit Claise <[email-masked]<mailto:[email-masked]>> wrote:
+1  to Jürgen point of view.

Regards, Benoit
From:Jürgen Schönwälder <[email-masked]<mailto:[email-masked]>>
To:William Lupton <[email-masked]<mailto:[email-masked]>>
Cc:NetMod WG <[email-masked]<mailto:[email-masked]>>
Date:2022-03-03 20:01:06
Subject:Re: [netmod] iana-if-type.yang has multiple revisions with the same date

/js

> [email-masked]<mailto:[email-masked]>

_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Fri, 4 Mar 2022 14:53:04 +0000

Hi Andy,

On this specific point:

Note that with multiple release trains and the new SERMVER, it is likely
that multiple [name, date, label] tuples resolve to the same [name, date] pair,
making the uniqueness problem even worse.

The module versioning draft explicitly disallows this.  Even when using revision-labels/Semver there is a requirement that they are separate revision modules with unique revision dates.

Specifically, it contains this text:

   In addition, this document uses the following terminology:

   o  YANG module revision: An instance of a YANG module, uniquely
      identified with a revision date, with no implied ordering or
      backwards compatibility between different revisions of the same
      module.

   This document clarifies [RFC7950] and [RFC6020] to explicitly allow

   non-linear development of YANG module and submodule revisions, so

   that they MAY have multiple revisions that directly derive from the

   same parent revision.  As per [RFC7950] and [RFC6020], YANG module

   and submodule revisions continue to be uniquely identified by their

   revision date, and hence all revisions of a given module or submodule

   MUST have unique revision dates.

And for revision labels:

      Revision labels MUST be unique amongst all revisions of a

      module or submodule.

Regards,
Rob
// As an author/contributor.

From: netmod <[email-masked]> On Behalf Of Andy Bierman
Sent: 01 March 2022 14:49
To: William Lupton <[email-masked]>
Cc: NetMod WG <[email-masked]>
Subject: Re: [netmod] iana-if-type.yang has multiple revisions with the same date

On Tue, Mar 1, 2022 at 4:54 AM William Lupton <[email-masked]<mailto:[email-masked]>> wrote:
All,

I noticed from https://yangcatalog.org/private-page/BBFYANGPageCompilation.html that there's a (long-standing?) problem in iana-if-type.yang<https://www.iana.org/assignments/yang-parameters/[email-masked]>: it has multiple revision statements with the same date:

  revision 2018-06-28 {
    description

  revision 2018-06-28 {
    description
      "Registered ifType 293.";
  }
This has presumably happened as a result of an automated update script that doesn't check for this case (*)? From a quick scan, I didn't see anything in RFC 7950 banning duplicate revision dates, but RFC 8407 section 4.8 says "If the module contents have changed, then the revision date of that new module version MUST be updated to a date later than that of the previous version" and of course yangdump-pro is checking this.

I think this issue should be resolved as well.
The YANG library identifies each module by a [name, date] tuple.
The <get-schema> operation uses this tuple to identify a specific revision to retrieve.
The import-by-revision mechanism uses this tuple to identify a specific revision to import.

If this [name, date] tuple is not unique, then it cannot be mapped to a single module revision.

Note that with multiple release trains and the new SERMVER, it is likely
that multiple [name, date, label] tuples resolve to the same [name, date] pair,
making the uniqueness problem even worse.

This is quite significant if a client reads the YANG library from a server
and decides it already has the module cached (based on the [name, date] tuple,
as defined in the standard.  Then it will not use the <get-schema> operation
to retrieve the module from the server.

YANG artifacts and SID files also rely on this [name, date] tuple uniqueness.

Even with the new versioning drafts, it is impossible for the client to know
"Do you mean the REAL module foo, version xxxx-xx-xx, or your private version?"

Thanks,
William

 Andy

(*) In the rare event that multiple changes are made in the same day, perhaps the second change should be (strictly wrongly) assigned to the following day. In theory this could cause revision dates to run far into the future but in practice I don't think this will happen :).
_______________________________________________
netmod mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/netmod

---

From: Andy Bierman <[email-masked]>
Date: Fri, 4 Mar 2022 09:58:00 -0800

Hi,

Vendors (including YumaWorks) have been violating the [name, date]
uniqueness rule for about a decade.
This is because vendors do not release YANG modules independently from
software.
Only SDOs do that.

A common software release train model (obsolete, oldstable, stable, latest,
testing)
shows how a YANG module with the same date ends up in multiple release
trains.

- Obsolete release trains do not get any module updates at all
- Oldstable modules usually only get changed for serious bugfixes.
- Stable modules usually only get changed for bugfixes and minor
enhancements
- Latest modules get all bugfixes and all new features
- Testing is work-in-progress and the version tracking is not useful during
this phase
   since the modules and code are unstable and subject to many changes

Consider a simple use-case: a bugfix in a leaf is added to the oldstable
release train

This is then merged downstream into each subsequent release train.
It is impractical and misleading to give each updated release train a
different
revision date.  This may break client applications and break
import-by-revision in
other modules.

IMO a practical solution is to add a leaf to the yang-library list entries
called "need-get-schema" (or whatever) that tells the client that the
[name, date] is not globally unique and the <get-schema> operation
should be used to retrieve the module used on this server with that tuple.

Andy

On Fri, Mar 4, 2022 at 6:53 AM Rob Wilton (rwilton) <[email-masked]>
wrote:

> The module versioning draft explicitly disallows this.  Even when using
> revision-labels/Semver there is a requirement that they are separate
> revision modules with unique revision dates.
>
>
>
>    o  YANG module revision: An instance of a YANG module, *uniquely*
>
> *      identified with a revision date*, with no implied ordering or
>
>    same parent revision.  *As per [RFC7950] and [RFC6020], YANG module*
>
> *   and submodule revisions continue to be uniquely identified by their*
>
> *   revision date, and hence all revisions of a given module or submodule*
>
> *   MUST have unique revision dates.*
>
>
>
> And for revision labels:
>
>
>
> *From:* netmod <[email-masked]> *On Behalf Of *Andy Bierman
> *Sent:* 01 March 2022 14:49
> *To:* William Lupton <[email-masked]>
> *Cc:* NetMod WG <[email-masked]>
> *Subject:* Re: [netmod] iana-if-type.yang has multiple revisions with the
> same date
>
>
>
>
>
>
>

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Fri, 4 Mar 2022 18:00:16 +0000

Hi Andy,

YANG packages are aimed at partly solving the problem you describe below.

In the -00 revision of the packages draft, it included SHA hashes* (YANG leaf “checksum”) of the included modules and packages so that a client can determine that its local copy of [email-masked]<mailto:[email-masked]> (or revision-date) exactly matches the one that the server is declaring that it is using in the YANG package definition.  You can see this in the -00 version (YANG Packages (ietf.org)<https://tools.ietf.org/id/draft-ietf-netmod-yang-packages-00.html>).

The checksum/hashes have been taken out of later revisions of the draft for two reasons:

  1.  At the time, we were not sure that we had a canonical representation of a YANG module, although possibly this has been mitigated by the YANG module versioning draft that effectively defines the file text of the published YANG module to be that canonical representation.
  2.  It was thought that it was too complex, and hence better deferred to an extension of future version of the YANG packages draft.  We also thought that we would need to get real security folks involved to check that we are really getting this right.

But either way, longer term, I think that a scheme along these lines this will probably be helpful.

Rob
// As an individual contributor

Sent: 03 March 2022 16:56

On Thu, Mar 3, 2022 at 1:25 AM William Lupton <[email-masked]<mailto:[email-masked]>> wrote:
Thanks Andy. What is the next step? Should I (or someone else) email [email-masked]<mailto:[email-masked]>, or can we assume that the relevant IANA person will already have seen this discussion?

It seems that RFC 8407 already says what to do (use a different revision-date).
We combine the revision-stmt so there is only 1 revision entry instead of 2.

 3b) client can use <get-schema> (IFF supported by the server) to retrieve the YANG files

[*] same can mean a later revision if specific schema definitions have not changed

Issues:

1) Is there ANY uniqueness guarantee that [name, date] is GLOBALLY unique.
A: Yes according to RFC 7950, but not really in implementations.
A: No, if revision-label is added and the same revision-date is used in multiple release trains.

So if a client cannot rely on [name, date] uniqueness, then it does not really know if
step 3a or step 3b is required.

Andy

On Tue, 1 Mar 2022 at 14:49, Andy Bierman <[email-masked]<mailto:[email-masked]>> wrote:

---

From: Andy Bierman <[email-masked]>
Date: Fri, 4 Mar 2022 10:25:02 -0800

On Fri, Mar 4, 2022 at 10:00 AM Rob Wilton (rwilton) <[email-masked]>
wrote:

> In the -00 revision of the packages draft, it included SHA hashes* (YANG
> leaf “checksum”) of the included modules and packages so that a client can
> determine that its local copy of [email-masked] (or revision-date) exactly
> matches the one that the server is declaring that it is using in the YANG
> package definition.  You can see this in the -00 version (YANG Packages
> (ietf.org)
> <https://tools.ietf.org/id/draft-ietf-netmod-yang-packages-00.html>).
>
>
>
>    1. At the time, we were not sure that we had a canonical
>    representation of a YANG module, although possibly this has been mitigated
>    by the YANG module versioning draft that effectively defines the file text
>    of the published YANG module to be that canonical representation.
>    2. It was thought that it was too complex, and hence better deferred
>    to an extension of future version of the YANG packages draft.  We also
>    thought that we would need to get real security folks involved to check
>    that we are really getting this right.
>
>
>

I was going to mention the checksums. I am aware they were taken out.

I have understood for years that YANG conformance at the module-level,
which is baked into the module itself, is a broken architecture.

Conformance should be based on use-cases, for which YANG module boundaries
are irrelevant.  The key to package management is that it replaces
individual file management.
How do YANG 1.1 compliant systems which only know about YANG modules work
with newer systems that know about packages.  Managing the interaction
details
at both the module and package level seems complicated.

IMO a new YANG version number is needed to make the sort of changes proposed
in the versioning work. There are existing issues, e.g.:

- the behavior for 'deprecated' and 'obsolete' has been broken from the
start
   and only a new YANG version can really fix it
- a standard module (or package) should be able to use deviations to alter
  a module for its own use-cases, but this violates a MUST NOT in YANG 1.1
- All extensions are purely optional in YANG 1.1, which could be fixed in a
new YANG version
- YANG packages could be mandatory for the new YANG version, allowing
   conformance requirements to be properly expressed

Rob

Andy

> *Sent:* 03 March 2022 16:56

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Tue, 8 Mar 2022 13:07:36 +0000

Hi William,

IANA have published a new revision with the revision history fixed.

iana-if-type YANG Module<https://www.iana.org/assignments/iana-if-type/iana-if-type.xhtml>

Regards,
Rob

From: netmod <[email-masked]> On Behalf Of Rob Wilton (rwilton)
Sent: 04 March 2022 14:37
To: William Lupton <[email-masked]>; Benoit Claise <[email-masked]>
Cc: NetMod WG <[email-masked]>
Subject: Re: [netmod] iana-if-type.yang has multiple revisions with the same date

Hi William,

Regards,
Rob

From: netmod <[email-masked]<mailto:[email-masked]>> On Behalf Of William Lupton
Sent: 04 March 2022 10:15
To: Benoit Claise <[email-masked]<mailto:[email-masked]>>
Cc: NetMod WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: [netmod] iana-if-type.yang has multiple revisions with the same date

Regards, Benoit

/js