Subject: [netmod] Revision-labels within filenames

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Tue, 14 Sep 2021 14:55:45 +0000

Carsten raised a point at the mic at IETF111 that the chosen character
'#' to separate the module name and module revision-label in filenames
is problematic.  Currently, the YANG module versioning draft says that
if you want to use revision-label within the filename, you use
MODULE_NAME#REVISION_LABEL (e.g., my-module#3.2.3.yang).  The reason for
a new character is not to confuse existing tools that expect a
revision-date to follow '@'.

Carsten mentioned that '#' causes problem with shells and within URLs. 
While the position of the '#' in the name does not actually trigger
issues with shells (it would if the filename began with a '#'), it does
indeed cause issues in URLs if it's not encoded.  For example,
https://example.com/my-module#3.2.3.yang will result in a 404 since a
file "my-module" does not exist.  Jan also pointed out that '#' will
cause issues in Makefiles, causing developers to have to escape them.

On the last meeting, the authors and contributors spent some time
running down other characters options, but all have their issues and
require escaping or encoding.  The consensus on that call was that the #
--> %23 URL encoding won't be as burdensome as initially thought. 
Platforms like GitHub (or its ilk) -- which are popular for hosting
modules -- will do this encoding automatically and downloading the
module will produce a filename with the un-encoded '#'.  Additionally,
yang-library location will have the encoded version of the URL (likely
generated by tooling), which clients then download and typically get the
un-encoded '#'.

Ultimately, we do not foresee filesystems cluttered with "%23" in
filenames nor do we expect a great deal of manual typing of "%23".  Yes,
the '#' will cause some additional developer work for use in Makefiles,
but the authors feel this is acceptable given the overall impact of
choosing another character.

More details on this, including our run-down of other characters can be
found in GitHub Issue #104
(https://github.com/netmod-wg/yang-ver-dt/issues/104).

Joe on behalf of the YANG versioning authors and contributors

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Tue, 14 Sep 2021 16:00:22 +0000

Hi Joe,

Sorry, I was on PTO for the meeting where this was discussed.  I don't think that it is great that the character needs to be encoded in a URL.  It would seem to make references to YANG files in YANG packages a lot more awkward to read/write.

Perhaps reusing '@' is pragmatically the best choice.  We already have the constraint that a revision label cannot match the format of a revision-date anyway.

Or, alternatively, if we really want to differentiate, then could '@@' work?

Rob
// As an author/contributor.

> -----Original Message-----
> From: netmod <[email-masked]> On Behalf Of Joe Clarke (jclarke)
> Sent: 14 September 2021 15:56
> To: [email-masked]
> Subject: [netmod] Revision-labels within filenames
> 
> Carsten raised a point at the mic at IETF111 that the chosen character '#' to
> separate the module name and module revision-label in filenames is
> problematic.  Currently, the YANG module versioning draft says that if you
> want to use revision-label within the filename, you use
> MODULE_NAME#REVISION_LABEL (e.g., my-module#3.2.3.yang).  The reason
> for a new character is not to confuse existing tools that expect a revision-
> date to follow '@'.
> 
> While the position of the '#' in the name does not actually trigger issues with
> shells (it would if the filename began with a '#'), it does indeed cause issues
> in URLs if it's not encoded.  For example, https://example.com/my-
> module#3.2.3.yang will result in a 404 since a file "my-module" does not
> exist.  Jan also pointed out that '#' will cause issues in Makefiles, causing
> developers to have to escape them.
> 
> On the last meeting, the authors and contributors spent some time running
> down other characters options, but all have their issues and require escaping
> or encoding.  The consensus on that call was that the #
> Platforms like GitHub (or its ilk) -- which are popular for hosting modules --
> will do this encoding automatically and downloading the module will produce
> a filename with the un-encoded '#'.  Additionally, yang-library location will
> have the encoded version of the URL (likely generated by tooling), which
> clients then download and typically get the un-encoded '#'.
> 
> Ultimately, we do not foresee filesystems cluttered with "%23" in filenames
> nor do we expect a great deal of manual typing of "%23".  Yes, the '#' will
> cause some additional developer work for use in Makefiles, but the authors
> feel this is acceptable given the overall impact of choosing another character.
> 
> found in GitHub Issue #104 (https://github.com/netmod-wg/yang-ver-
> dt/issues/104).
> 
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Tue, 14 Sep 2021 16:56:38 +0000

> Sorry, I was on PTO for the meeting where this was discussed.  I don't think that it is great that the character needs to be encoded in a URL.  It would seem to make references to YANG files in YAN…

The writing here will likely still be programmatically driven, but I get
the reading argument.  While client automation won't care, a user
reading %233.2.3 does not as easily grok that as #3.2.3.  That said, I
wouldn't use the URL as a means to get the revision-label.  I'd use the
revision leaf in the package for a given module.

> Or, alternatively, if we really want to differentiate, then could '@@' work?

@@ was discussed, but would likely still trip up current tools that
expect '@' followed by a revision date.  But if we're good with
disrupting this, I'd prefer we overload the single '@'.

Joe

---

From: Kent Watsen <[email-masked]>
Date: Tue, 14 Sep 2021 17:18:51 +0000

>> Sorry, I was on PTO for the meeting where this was discussed.  I don't think that it is great that the character needs to be encoded in a URL. 

How often are YANG module names in URLs?

A website for downloading modules would reference modules by URL but, as Joe said, the encoding would be transparent to the end-user (i.e., the downloaded filename would NOT have the encoded character in the name).

K. // contrib

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Tue, 14 Sep 2021 17:42:57 +0000

> From: Joe Clarke (jclarke) <jclarke=[email-masked]>
> Sent: 14 September 2021 17:57
> To: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
> Subject: Re: Revision-labels within filenames
> 
> > Sorry, I was on PTO for the meeting where this was discussed.  I don't think
> that it is great that the character needs to be encoded in a URL.  It would
> seem to make references to YANG files in YANG packages a lot more
> awkward to read/write.
> 
> The writing here will likely still be programmatically driven, but I get the
> reading argument.  While client automation won't care, a user reading
> %233.2.3 does not as easily grok that as #3.2.3.  That said, I wouldn't use the
> URL as a means to get the revision-label.  I'd use the revision leaf in the
> package for a given module.

I wasn't thinking of a URL to get the revision-label, I was more thinking of a URL to identify the source YANG file for a particular revision.

E.g., in the YANG packages examples:

https://datatracker.ietf.org/doc/html/draft-ietf-netmod-yang-packages#appendix-A

Ideally, I would like the URLs pointing the source YANG files to be able to do so without requiring any encoding of the '@' or '#' symbol.

E.g., looking at some of the examples (which seem to have been converted somewhere), and hence the @ symbol has been converted to %40, which makes the date look like 40k years into the future.

              "location": [ "https://tiny.cc/ietf-yang/\
                                   iana-crypt-hash%402014-08-06.yang" ],

If this location can be something like https://tiny.cc/ietf-yang/[email-masked] or https://tiny.cc/ietf-yang/[email-masked] then I think that it would make these URLs more readable.

> 
> >
> > Perhaps reusing '@' is pragmatically the best choice.  We already have the
> constraint that a revision label cannot match the format of a revision-date
> anyway.
> >
> @@ was discussed, but would likely still trip up current tools that expect '@'
> followed by a revision date.  But if we're good with disrupting this, I'd prefer
> we overload the single '@'.

I'm pretty sure that the existing tools will need to be updated regardless to understand the new version labels.  If the tool can't cope then a script needs to be used to strip the revision labels from the files, or rename them to their equivalent revision dates.

Thanks,
Rob

> >> Carsten raised a point at the mic at IETF111 that the chosen
> >> character '#' to separate the module name and module revision-label
> >> in filenames is problematic.  Currently, the YANG module versioning
> >> draft says that if you want to use revision-label within the
> >> filename, you use MODULE_NAME#REVISION_LABEL (e.g.,
> >> my-module#3.2.3.yang).  The reason for a new character is not to
> >> confuse existing tools that expect a revision- date to follow '@'.
> >>
> >> issues with shells (it would if the filename began with a '#'), it
> >> does indeed cause issues in URLs if it's not encoded.  For example,
> >> https://example.com/my- module#3.2.3.yang will result in a 404 since
> >> a file "my-module" does not exist.  Jan also pointed out that '#'
> >> will cause issues in Makefiles, causing developers to have to escape them.
> >>
> >> module will produce a filename with the un-encoded '#'.
> >> Additionally, yang-library location will have the encoded version of
> >> the URL (likely generated by tooling), which clients then download and
> typically get the un-encoded '#'.
> >>
> >> filenames nor do we expect a great deal of manual typing of "%23".
> >> Yes, the '#' will cause some additional developer work for use in
> >> Makefiles, but the authors feel this is acceptable given the overall impact
> of choosing another character.
> >>
> >> More details on this, including our run-down of other characters can
> >> be found in GitHub Issue #104 (https://github.com/netmod-wg/yang-ver-
> >> dt/issues/104).
> >>

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Tue, 14 Sep 2021 17:43:52 +0000

Hi Kent,

It was the YANG package definitions that I had in mind.  More details in the response that I have just sent back to Joe.

Regards,
Rob

> From: Kent Watsen <[email-masked]>
> Sent: 14 September 2021 18:19
> To: Joe Clarke (jclarke) <[email-masked]>
> Cc: Rob Wilton (rwilton) <[email-masked]>; [email-masked]
> Subject: Re: [netmod] Revision-labels within filenames
> 
> 
> 
> >> Sorry, I was on PTO for the meeting where this was discussed.  I don't
> think that it is great that the character needs to be encoded in a URL.
> 
> A website for downloading modules would reference modules by URL but, as
> Joe said, the encoding would be transparent to the end-user (i.e., the
> downloaded filename would NOT have the encoded character in the name).
> 
> K. // contrib
>

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Tue, 14 Sep 2021 19:38:23 +0000

> Ideally, I would like the URLs pointing the source YANG files to be able to do so without requiring any encoding of the '@' or '#' symbol.

Wouldn't you think these locations would be generated via tooling (i.e.,
consumable by automation or just having people click)?  I would expect a
human to look at the other package metadata to get the list of modules
and their revisions.

> If this location can be something like https://tiny.cc/ietf-yang/[email-masked] or https://tiny.cc/ietf-yang/[email-masked] then I think that it would make these URLs more readable.

I do agree it's more readable, but I don't think it's critical to ensure.

Joe

---

From: "Rob Wilton (rwilton)" <[email-masked]>
Date: Wed, 15 Sep 2021 09:08:27 +0000

> Sent: 14 September 2021 20:38
> > I wasn't thinking of a URL to get the revision-label, I was more thinking of a
> URL to identify the source YANG file for a particular revision.
> >
> > Ideally, I would like the URLs pointing the source YANG files to be able to do
> so without requiring any encoding of the '@' or '#' symbol.
> 
> human to look at the other package metadata to get the list of modules and
> their revisions.

I agree that over time, most of these will be generated by tooling.  But I suspect that there will still be some cases where simple packages may be written and maintained by hand, where the risk is that incorrect URLs will be used, albeit, that it would be relatively easy for tooling to validate the URLs.

> 
> >
> > E.g., looking at some of the examples (which seem to have been converted
> somewhere), and hence the @ symbol has been converted to %40, which
> makes the date look like 40k years into the future.
> >
> > If this location can be something like https://tiny.cc/ietf-yang/iana-crypt-
> [email-masked] or https://tiny.cc/ietf-yang/iana-crypt-
> [email-masked] then I think that it would make these URLs more readable.
> 

I agree it is not critical, just nice to have.

Thanks,
Rob

---

From: "Sterne, Jason (Nokia - CA/Ottawa)" <[email-masked]>
Date: Tue, 28 Sep 2021 14:24:33 +0000

Hi guys,

I'm not sure how reusing '@' is preferrable for URLs than using '#'. They are both escaped right ?

Jason

> From: netmod <[email-masked]> On Behalf Of Rob Wilton
> (rwilton)
> Sent: Wednesday, September 15, 2021 5:08 AM
> To: Joe Clarke (jclarke) <jclarke=[email-masked]>;
> [email-masked]
> and
> I agree that over time, most of these will be generated by tooling.  But I
> suspect that there will still be some cases where simple packages may be
> written and maintained by hand, where the risk is that incorrect URLs will be
> used, albeit, that it would be relatively easy for tooling to validate the URLs.
> 
> >
> > >

---

From: "Joe Clarke (jclarke)" <[email-masked]>
Date: Tue, 28 Sep 2021 20:41:51 +0000

> I'm not sure how reusing '@' is preferrable for URLs than using '#'. They are both escaped right ?

The Makefile case would be the only thing I can think of where @ is more
advantageous over #.  But then you have the issue with tooling.

Joe