Subject: Re: [yang-doctors] [Rtg-dt-yang-arch] Open Config on ietf-routing-types

From: Benoit Claise <[email-masked]>
Date: Tue, 21 Feb 2017 09:58:14 +0100

Including the YANG doctors on this important topic.

Regards, Benoit
> As I wrote the initial email. I can probably add some comments at the 
> next meeting.
>
> We have examined multiple times rewriting of regexps from the W3C 
> standard to something that is more commonly supported. And to be 
> frank, it's not a useful task to spend time on. Worse, the existing 
> IETF types modules use elements such as \p{L} when AFAICS, there is no 
> reason not to use [a-zA-Z] for all practical use cases (I know of no 
> system that has an interface that needs to be zoned to 🍺  for example).
>
> r.
>
> On Mon, 20 Feb 2017 at 17:49 Lou Berger <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     [Adding Kent as netmod co-chair, netmod AD is already on the list.]
>
>     Jeff,
>
>         This is an important user perspective.  I personally am open to
>     changing the regexp reference in YANG (see [1]), *but* I think this
>     would required rev'ing YANG (to 1.2, for example) to do so.
>
>     Also, in poking around, I can only find posix definitions behind
>     pay-walls, which may be an issue - but as IAB is responsible for IETF
>     liaisons, perhaps you could run this part to ground ;-)
>
>     If Kent is agreeable, I'd be open to having an presentation and
>     discussion on this (even without a draft) at the next meeting.
>
>     Kent?
>
>     Thanks,
>
>     Lou
>
>     [1] https://tools.ietf.org/html/rfc7950#section-9.4.5
>     On 2/20/2017 7:55 PM, Jeff Tantsura wrote:
>     > HI,
>     >
>     > I have been having some discussion with OC folks (privately),
>     I’d really want to see convergence between OC and IETF.
>     >
>     > Below is the explanation why not to use IETF version.  Do you
>     think there’s anything we could do (Lou?)
>     >
>     > “For types modules, there's a pretty interesting problem
>     (AFAICS). We have found that the W3C standard that is being used
>     for regexp has very limited support. We've actually forked our
>     types modules to stop referencing IETF ones, since we're aiming to
>     use a regexp standard that is more commonly supported (POSIX).
>     >
>     > When I've raised this privately, there seems to be a bunch of
>     opposition to even discussing such implementation questions - such
>     that forking and moving forward seems a much more effective
>     solution. Whilst I can't speak for OC, personally, I'd oppose
>     trying to split our types across IETF and OC as it adds to the
>     implementation complexity.
>     >
>     > Perhaps with your new hat, it might be possible for us to
>     discuss further what IETF's interest is in usability vs.
>     long-winded NETMOD discussions :-) that seem to reach few answers.”
>     >
>     >
>     > Cheers,
>     > Jeff
>     >
>     >
>     >
>     > _______________________________________________
>     > Rtg-dt-yang-arch mailing list
>     > [email-masked] <mailto:[email-masked]>
>     > https://www.ietf.org/mailman/listinfo/rtg-dt-yang-arch
>
>     _______________________________________________
>     Rtg-dt-yang-arch mailing list
>     [email-masked] <mailto:[email-masked]>
>     https://www.ietf.org/mailman/listinfo/rtg-dt-yang-arch
>
>
>
> _______________________________________________
> Rtg-dt-yang-arch mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/rtg-dt-yang-arch

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 21 Feb 2017 12:21:07 +0100 (CET)

Hi,

First of all, I am not particularily attached to any specific regexp
flavor, so if we were designing something new I would be open to using
POSIX or something else.

I note that there are several YANG implementations out there that
successfully *are* implementing the XML regexp flavor.  My guess is
that most internally use libxml2.

I also note that while we can discuss individual datatypes, it seems
reasonable to pick a regexp dialect that supports unicode (which POSIX
doesn't do).

But is the problem lack of library support, or that the XSD regexp
dialect is too primitive?   FWIW, POSIX is also fairly simple.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 21 Feb 2017 18:08:14 +0100

>From the narrow viewpoint of the regex language design, the W3C XML
Schema flavour is probably the most appropriate as it is specifically
designed for use in schema languages. For example, it is a good thing
that the ^ and $ anchors are not needed. However, ...  

> that most internally use libxml2.

AFAICT, libxml2 is not available for Python 3, so I am currently using the
PyXB module, it works but it would certainly be easier to use a
built-in module.

I guess it is the same in most other languages: a native regex package
exists but it is specific to that language.

> reasonable to pick a regexp dialect that supports unicode (which POSIX

This is a showstopper.

> dialect is too primitive?   FWIW, POSIX is also fairly simple.

I wouldn't say that it is too primitive - we haven't much use
e.g. for groups.

I don't see any better alternative, so my preference is to stick to XSD.

Lada

> yang-doctors mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/yang-doctors

-- 
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Feb 2017 09:24:58 -0800

Hi,

Maybe the openconfig folks are just looking for reasons to ignore the IETF
modules.
Implementation details such as regexp are not the type of thing the IETF
discusses at length.  libxml2 is widely deployed, so "lack of
implementations"
does not seem true.

Google has over 900 opensource projects listed on github:
https://github.com/google

You would think they would have an implementation of this regexp in there
somewhere. ;-)

It would require a new version of YANG that breaks backward compatibility
with v1.0 and 1.1 to change this. Not sure it is really worth it.

Andy

On Tue, Feb 21, 2017 at 12:58 AM, Benoit Claise <[email-masked]> wrote:

> As I wrote the initial email. I can probably add some comments at the next
> meeting.
>
> We have examined multiple times rewriting of regexps from the W3C standard
> to something that is more commonly supported. And to be frank, it's not a
> useful task to spend time on. Worse, the existing IETF types modules use
> elements such as \p{L} when AFAICS, there is no reason not to use [a-zA-Z]
> for all practical use cases (I know of no system that has an interface that
> needs to be zoned to 🍺  for example).
>
> r.
>
> On Mon, 20 Feb 2017 at 17:49 Lou Berger <[email-masked]> wrote:
>
>> > I have been having some discussion with OC folks (privately), I’d
>> really want to see convergence between OC and IETF.
>> >
>> > Below is the explanation why not to use IETF version.  Do you think
>> there’s anything we could do (Lou?)
>> >
>> > “For types modules, there's a pretty interesting problem (AFAICS). We
>> have found that the W3C standard that is being used for regexp has very
>> limited support. We've actually forked our types modules to stop
>> referencing IETF ones, since we're aiming to use a regexp standard that is
>> more commonly supported (POSIX).
>> >
>> opposition to even discussing such implementation questions - such that
>> forking and moving forward seems a much more effective solution. Whilst I
>> can't speak for OC, personally, I'd oppose trying to split our types across
>> IETF and OC as it adds to the implementation complexity.
>> >
>> > Perhaps with your new hat, it might be possible for us to discuss
>> further what IETF's interest is in usability vs. long-winded NETMOD
>> discussions :-) that seem to reach few answers.”
>> >
>> >
>> > Cheers,
>> > Jeff
>> >
>> >
>> >
> Rtg-dt-yang-arch mailing [email-masked]://www.ietf.org/mailman/listinfo/rtg-dt-yang-arch
>
>
>

---

From: Christian Hopps <[email-masked]>
Date: Tue, 21 Feb 2017 12:27:02 -0500

>> reasonable to pick a regexp dialect that supports unicode (which POSIX

I'm sympathetic to supporting unicode, and while I don't wish to debate
the point personally, it clearly is not stopping actual users. They
appear to be voting with their feet.

---

From: Kent Watsen <[email-masked]>
Date: Tue, 21 Feb 2017 17:33:10 +0000

[+phil]

My chair response is that we'd need to see if there is sufficient WG interest to propose making a change.  Certainly, a solution that could augment the existing solution in a way that didn't break existing clients without warning would be good (e.g., YANG-next).

My contributor response is that JUNOS also struggles to support XSD regex.  In fact, I think that it has always used POSIX regex and continues to do so to this day.  Phil might have more to say on this point.

Thanks,
Kent

On 2/21/17, 12:24 PM, "Andy Bierman" <[email-masked]<mailto:[email-masked]>> wrote:

Hi,

Maybe the openconfig folks are just looking for reasons to ignore the IETF modules.
Implementation details such as regexp are not the type of thing the IETF
discusses at length.  libxml2 is widely deployed, so "lack of implementations"
does not seem true.

You would think they would have an implementation of this regexp in there somewhere. ;-)

Andy

On Tue, Feb 21, 2017 at 12:58 AM, Benoit Claise <[email-masked]<mailto:[email-masked]>> wrote:
Including the YANG doctors on this important topic.

Regards, Benoit
As I wrote the initial email. I can probably add some comments at the next meeting.

We have examined multiple times rewriting of regexps from the W3C standard to something that is more commonly supported. And to be frank, it's not a useful task to spend time on. Worse, the existing IETF types modules use elements such as \p{L} when AFAICS, there is no reason not to use [a-zA-Z] for all practical use cases (I know of no system that has an interface that needs to be zoned to 🍺  for example).

r.

On Mon, 20 Feb 2017 at 17:49 Lou Berger <[email-masked]<mailto:[email-masked]>> wrote:
[Adding Kent as netmod co-chair, netmod AD is already on the list.]

Jeff,

Kent?

Thanks,

Lou

[1] https://tools.ietf.org/html/rfc7950#section-9.4.5
On 2/20/2017 7:55 PM, Jeff Tantsura wrote:
> HI,
>
> I have been having some discussion with OC folks (privately), I’d really want to see convergence between OC and IETF.
>
> Below is the explanation why not to use IETF version.  Do you think there’s anything we could do (Lou?)
>
> “For types modules, there's a pretty interesting problem (AFAICS). We have found that the W3C standard that is being used for regexp has very limited support. We've actually forked our types modules to stop referencing IETF ones, since we're aiming to use a regexp standard that is more commonly supported (POSIX).
>
> When I've raised this privately, there seems to be a bunch of opposition to even discussing such implementation questions - such that forking and moving forward seems a much more effective solution. Whilst I can't speak for OC, personally, I'd oppose trying to split our types across IETF and OC as it adds to the implementation complexity.
>
> Perhaps with your new hat, it might be possible for us to discuss further what IETF's interest is in usability vs. long-winded NETMOD discussions :-) that seem to reach few answers.”
>
>
> Cheers,
> Jeff
>
>
>
> [email-masked]<mailto:[email-masked]>

_______________________________________________
Rtg-dt-yang-arch mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/rtg-dt-yang-arch

[email-masked]<mailto:[email-masked]>

_______________________________________________
yang-doctors mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/yang-doctors

---

From: Lou Berger <[email-masked]>
Date: Tue, 21 Feb 2017 12:38:39 -0500

Thanks kent - I read this as also being open to discussing in the WG.

Jeff,

    You do you want to take the lead on putting something together on
this to drive discussion in  Chicago?  If not, who is the right person?

Thanks,

Lou

On 2/21/2017 12:33 PM, Kent Watsen wrote:
>
> [+phil]
>
>  
>
> My chair response is that we'd need to see if there is sufficient WG
> interest to propose making a change.  Certainly, a solution that could
> augment the existing solution in a way that didn't break existing
> clients without warning would be good (e.g., YANG-next).
>
>  
>
> My contributor response is that JUNOS also struggles to support XSD
> regex.  In fact, I think that it has always used POSIX regex and
> continues to do so to this day.  Phil might have more to say on this
> point.
>
>  
>
> Thanks,
>
> Kent
>
>  
>
>  
>
>  
>
>         We have examined multiple times rewriting of regexps from the
>         W3C standard to something that is more commonly supported. And
>         to be frank, it's not a useful task to spend time on. Worse,
>         the existing IETF types modules use elements such as \p{L}
>         when AFAICS, there is no reason not to use [a-zA-Z] for all
>         practical use cases (I know of no system that has an interface
>         that needs to be zoned to 🍺  for example).
>
>          
>
>         r.
>
>          
>
>             > I have been having some discussion with OC folks
>             (privately), I’d really want to see convergence between OC
>             and IETF.
>             >
>             > Below is the explanation why not to use IETF version. 
>             Do you think there’s anything we could do (Lou?)
>             >
>             (AFAICS). We have found that the W3C standard that is
>             being used for regexp has very limited support. We've
>             actually forked our types modules to stop referencing IETF
>             ones, since we're aiming to use a regexp standard that is
>             > When I've raised this privately, there seems to be a
>             bunch of opposition to even discussing such implementation
>             questions - such that forking and moving forward seems a
>             much more effective solution. Whilst I can't speak for OC,
>             personally, I'd oppose trying to split our types across
>             > Perhaps with your new hat, it might be possible for us
>             to discuss further what IETF's interest is in usability
>             vs. long-winded NETMOD discussions :-) that seem to reach
>             few answers.”
>             >
>             >
>             > Cheers,
>             > Jeff
>             >
>             >
>             >
>     [email-masked] <mailto:[email-masked]>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 21 Feb 2017 19:03:13 +0100

> appear to be voting with their feet.

There are multiple POSIX regex variants. If we were to pick one, some
other people will say POSIX regex suck and we should at least do PCRE,
which then comes in different flavours (with or without unicode/UTF8
support). It may be useful if someone does some detailed background
reading. It might turn out that the XSD regex syntax YANG is using is
not really a bad one once you want to handle unicode. Here is something
for a start:

http://www.regular-expressions.info/unicode.html
https://en.wikipedia.org/wiki/Comparison_of_regular_expression_engines

/js

PS: I actually do not know how far PCRE is from the XSD regex we use
    in YANG if you select the right options.

PS: PCRE seems to do \p{L} like many of the more modern regular
    expression implementations seem to do. I guess you end up with
    this once you take unicode support serious.

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Rob Shakir <[email-masked]>
Date: Tue, 21 Feb 2017 18:52:18 +0000

Skipping some elements of this message that I'm not sure are constructive
to respond to.

Until very recently, OpenConfig models did use ietf imports. This would
appear to me to be exactly counter to the "looking for excuses" comment.

On Tue, 21 Feb 2017 at 09:24 Andy Bierman <[email-masked]> wrote:

> discusses at length.

That's fine. Other than the specification mandates an implementation of a
particular regexp set of functionality. Driving specific implementations
and code maintenance for a standard negatively impacts its adoption.
RFC5218 did a good job of talking about this -
https://tools.ietf.org/html/rfc5218#section-2.1.3

r.

---

From: Kent Watsen <[email-masked]>
Date: Tue, 21 Feb 2017 19:28:22 +0000

> Skipping some elements of this message that I'm not sure are constructive to respond to.
>
>Until very recently, OpenConfig models did use ietf imports. This would appear to me to be >exactly counter to the "looking for excuses" comment.

Agreed.  Andy, that was a bit over the top.  Please check tone before hitting send.

Kent // pick a hat

---

From: Jeff Tantsura <[email-masked]>
Date: Tue, 21 Feb 2017 11:54:12 -0800

/wearing my new IAB hat

Andy,

This conversation has been started to find common pain points and come up with a solution that works for all of the parties involved.

What is not part of it is:

-I know better

-not made here

-passive-aggressive behavior

-discussing to death and finger-pointing 

Your knowledge and expertise are highly valued in the industry, your help to solve the issues is very much needed and more than welcome.

Cheers,

Jeff

From: Andy Bierman <[email-masked]>
Date: Tuesday, February 21, 2017 at 09:24
To: Benoit Claise <[email-masked]>
Cc: Rob Shakir <[email-masked]>, Lou Berger <[email-masked]>, Jeff Tantsura <[email-masked]>, "Acee Lindem (acee)" <[email-masked]>, RTG YANG Design Team <[email-masked]>, Xufeng Liu <[email-masked]>, Yingzhen Qu <[email-masked]>, "[email-masked]" <[email-masked]>, Alia Atlas <[email-masked]>
Subject: Re: [yang-doctors] [Rtg-dt-yang-arch] Open Config on ietf-routing-types

Hi,

does not seem true.

 

Andy

Regards, Benoit

r.

Jeff,

Kent?

Thanks,

Lou

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Feb 2017 12:41:49 -0800

OK -- sorry.
I'm not interested in a new version of YANG,
but if that's what the WG wants to do, that's fine.

Andy

On Tue, Feb 21, 2017 at 11:28 AM, Kent Watsen <[email-masked]> wrote:

> appear to me to be >exactly counter to the "looking for excuses" comment.
>
>
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 21 Feb 2017 21:59:05 +0100

> On 21 Feb 2017, at 19:52, Rob Shakir <[email-masked]> wrote:
> 
> Until very recently, OpenConfig models did use ietf imports. This would appear to me to be exactly counter to the "looking for excuses" comment.
> 
> That's fine. Other than the specification mandates an implementation of a particular regexp set of functionality. Driving specific implementations and code maintenance for a standard negatively impacts its adoption. RFC5218 did a good job of talking about this - https://tools.ietf.org/html/rfc5218#section-2.1.3

It is not that the specification mandates an implementation, but it obviously has to standardize a particular language of regular expressions so that everybody's notion of whether a given string is a valid value for a given type is the same.

Of course we can discuss what language to use but I don't see any clearly superior alternative. It is true that XSD regular expressions deviate in some aspects from the mainstream but at least they are defined in an open standard, very readable textbook-style descriptions are freely available [1], and open source implementations do exist.

Lada

[1] http://books.xmlschemata.org/relaxng/relax-CHP-9.html

---

From: Andy Bierman <[email-masked]>
Date: Tue, 21 Feb 2017 19:07:12 -0800

Hi,

The pattern syntax has not really changed since the early drafts in 2008,
so it is a bit surprising to hear that it is a show-stopper in 2017.
Seems to me that changing the pattern syntax would require a new
version of YANG. Modules written in version 1.0 and 1.1 might not conform to
the new pattern syntax, which seems like a problem.

It is expensive to introduce a new version of a programming language.
Some people might prefer stability over incremental features.

Andy

On Tue, Feb 21, 2017 at 11:54 AM, Jeff Tantsura <[email-masked]>
wrote:

> /wearing my new IAB hat
>
>
>
> Andy,
>
>
>
> This conversation has been started to find common pain points and come up
> with a solution that works for all of the parties involved.
>
> Your knowledge and expertise are highly valued in the industry, your help
> to solve the issues is very much needed and more than welcome.
>
>
>
> Cheers,
>
> Jeff
>
>
>
> *From: *Andy Bierman <[email-masked]>
> *Date: *Tuesday, February 21, 2017 at 09:24
> *To: *Benoit Claise <[email-masked]>
> *Cc: *Rob Shakir <[email-masked]>, Lou Berger <[email-masked]>, Jeff
> Tantsura <[email-masked]>, "Acee Lindem (acee)" <[email-masked]>,
> RTG YANG Design Team <[email-masked]>, Xufeng Liu <
> [email-masked]>, Yingzhen Qu <[email-masked]>, "
> [email-masked]" <[email-masked]>, Alia Atlas <
> [email-masked]>
> *Subject: *Re: [yang-doctors] [Rtg-dt-yang-arch] Open Config on
> ietf-routing-types
>
>
>
> Hi,
>
>
>
>
>
> > I have been having some discussion with OC folks (privately), I’d really
> want to see convergence between OC and IETF.
> >
> > When I've raised this privately, there seems to be a bunch of opposition
> to even discussing such implementation questions - such that forking and
> moving forward seems a much more effective solution. Whilst I can't speak
> for OC, personally, I'd oppose trying to split our types across IETF and OC
> as it adds to the implementation complexity.
> >

---

From: Jeff Tantsura <[email-masked]>
Date: Tue, 21 Feb 2017 23:57:25 -0800

Lou,

Yes, absolutely!

Please note - my goal is NOT to create more formal documents/YANG versions/etc – it is to get to the point we could reduce (eliminate would be the grand goal) discrepancy between IETF and OpenConfig (by now implemented by every vendor out there) models. Having OC using ietf-routing-types without sacrificing functionality would be a great starting point, tangible deliverable, achievable given both sides are willing to co-operate.

I’ll be talking to some of you during this and next week.

I believe RTGWG should provide venue for the discussion, let me discuss this with co-chair and AD
 
Thanks,
Jeff

On 2/21/17, 09:38, "Lou Berger" <[email-masked]> wrote:

    Jeff,

    Thanks,
    
    Lou

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 09:25:22 +0100 (CET)

Hi,

Jeff Tantsura <[email-masked]> wrote:
> Lou,
> 
> Yes, absolutely!
> 
> Please note - my goal is NOT to create more formal documents/YANG
> versions/etc – it is to get to the point we could reduce (eliminate
> would be the grand goal) discrepancy between IETF and OpenConfig (by
> now implemented by every vendor out there) models.

I just want to comment on this last point.  It is simply not true.
YANG is designed for and used in *lots* of other types of devices than
routers, many of which do not implement the OC models.

> Having OC using
> ietf-routing-types without sacrificing functionality would be a great
> starting point, tangible deliverable, achievable given both sides are
> willing to co-operate.

My question about what the problem is is still not answered.  Is the
problem that the XSD regexp dialect is too primitive, or is it that
the open implementations that exist cannot be used?

> discuss this with co-chair and AD

A clarifying question  - what does "the discussion" refer to?  Is it
about making sure ietf-routing-types can be used by others, or is it
about making changes to the YANG language?

/martin

---

From: Jeff Tantsura <[email-masked]>
Date: Wed, 22 Feb 2017 00:44:35 -0800

Martin,

please see inline

Given your deep YANG expertise and contributions to both sides, I’d really appreciate your advice.

Cheers,
Jeff

On 2/22/17, 00:25, "Martin Bjorklund" <[email-masked]> wrote:

    Hi,

    I just want to comment on this last point.  It is simply not true.
    YANG is designed for and used in *lots* of other types of devices than
    routers, many of which do not implement the OC models.
[jeff] This discussion has started in context of OC using ietf-routing-types, which is a product of RTGWG and meant to be used in routing realm

    My question about what the problem is is still not answered.  Is the
    problem that the XSD regexp dialect is too primitive, or is it that
    the open implementations that exist cannot be used?
[jeff] Rob did shed some light, I’ll be working towards a better problem definition, please give me some time

    A clarifying question  - what does "the discussion" refer to?  Is it
    about making sure ietf-routing-types can be used by others, or is it
    about making changes to the YANG language?
[jeff] obviously, RTGWG is not the right place to decide on any YANG changes, it is to discuss what is the problem and potential solution.
[jeff] /*routing context*/ the fact that part of the industry can’t use what has been produced by IETF is a serious issue that needs to be resolved, it doesn’t do any good to either implementers or users. Rather than keeping on duplicating the work, and calling each other names we should work together!
    
Jeff 
   
    /martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 10:12:17 +0100 (CET)

Jeff Tantsura <[email-masked]> wrote:
> [jeff] This discussion has started in context of OC using
> ietf-routing-types, which is a product of RTGWG and meant to be used
> in routing realm

Ok.

> [jeff] Rob did shed some light, I’ll be working towards a better
> problem definition, please give me some time

I didn't quite understand Rob's comment, so I look forward to a
problem definition.

> [jeff] obviously, RTGWG is not the right place to decide on any YANG
> changes, it is to discuss what is the problem and potential solution.

Ok.

> [jeff] /*routing context*/ the fact that part of the industry can’t
> use what has been produced by IETF is a serious issue that needs to be
> resolved, it doesn’t do any good to either implementers or
> users. Rather than keeping on duplicating the work, and calling each
> other names we should work together!

I don't think I have been calling anyone names.  I always try hard to
keep the discussions at a technical level.

Going back to the issue on regexps, I (and probably others) have
proposed a way for OC to work around the problem - use an oc-specific
extension statement, so instead of doing:

  type string {
      pattern '^[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}$';
  }

one could do:

  type string {
      oc:pattern '^[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}$';
  }

This would avoid the fork of YANG that actually has happened now.

/martin

---

From: Lou Berger <[email-masked]>
Date: Wed, 22 Feb 2017 06:52:42 -0500

Jeff,

> Yes, absolutely!

Glad to hear that you'll lead this discussion.

....

> this with co-chair and AD

I may have misunderstood something here. If the issue is specific to 
routing types, it certainly should be discussed in the context of the rtg 
area dt and the routing wg.  But if the context is yang in general, which I 
thought was your point, the discussion belongs in the netmod working group. 
This will ensure that the right expertise is in the room to help *solve* 
the issue being raised.

I of course will defer to the ADs on this.

Lou

---

From: Lou Berger <[email-masked]>
Date: Wed, 22 Feb 2017 07:02:01 -0500

See below.

>> ietf-routing-types, which is a product of RTGWG and meant to be used

As stated in my previous mail, if the problem is with routing types it is 
certainly appropriate to fix it in the routing working group.

>> changes, it is to discuss what is the problem and potential solution.

Once you get into potential solutions that go beyond a single module, i.e., 
are changes to yang, it's really best to have the discussion in the 
location the general Yang expertise, ie net mod.

>> users.

100% agree.

>> Rather than keeping on duplicating the work, and calling each

This is a good approach if it works - it's certainly an order of magnitude 
or two simpler than bumping yang revs!

Is there any reason to *not* include it in routing types if it addresses 
the base comment?

Rob/Anees,
   Would this address your issue?

Lou

PS thanks for illustrating why it's so important to ensure that the right 
expertise is part of the discussion, as well as the value of focusing on 
solving a raised issue.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 22 Feb 2017 13:15:55 +0100

On Wed, Feb 22, 2017 at 07:02:01AM -0500, Lou Berger wrote:
> Is there any reason to *not* include it in routing types if it addresses the
> base comment?
> 
> Rob/Anees,

I think we need a clear problem statement before discussing any
solutions. Adding a new extension statement that none of the existing
tools understands also comes with a cost. There is a difference
between OC adding an extension for OC usage or the IETF adding an
extension for general usage.

/js

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 22 Feb 2017 12:26:40 +0000

See a couple inlinesŠ

On 2/22/17, 7:02 AM, "Lou Berger" <[email-masked]> wrote:

>>> Given your deep YANG expertise and contributions to both sides, I¹d
>>> really appreciate your advice.
>>>
>>> Cheers,
>>> Jeff
>>>
>>>
>>>     > versions/etc  it is to get to the point we could reduce
>>>(eliminate

The issue is obviously related to YANG in general. There is currently no
YANG construct to use POSIX expressions to model strings.

>>> [jeff] Rob did shed some light, I¹ll be working towards a better
>>>     > I¹ll be talking to some of you during this and next week.
>>>     >
>>> [jeff] /*routing context*/ the fact that part of the industry can¹t
>>> resolved, it doesn¹t do any good to either implementers or
>>> users.
>
>100% agree.
>

With this YANG extension, would we specify both types of pattern for
common reusable types? I think we need a quick POSIX extension draft to
precisely define the semantics.

Thanks,
Acee

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Wed, 22 Feb 2017 12:30:46 +0000

>extension for general usage.

But this is the case for any extension. I think it has to be a general
IETF extension so that it can be readily used by IETF modules including
routing types. 

Acee

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 13:36:32 +0100 (CET)

> On February 22, 2017 4:12:54 AM Martin Bjorklund <[email-masked]>

[...]

> This is a good approach if it works - it's certainly an order of
> magnitude or two simpler than bumping yang revs!
> 
> Is there any reason to *not* include it in routing types if it
> addresses the base comment?

I think there are two issues.  The first is how OC can specify
patterns in their preferred regexp dialect in their own modules.  My
proposal above is one way of doing that, in a way that is 100% YANG
compliant.  [Side note: this is common practise; we have our own
extension statements that can be used to restrict certain types, and I
am pretty sure that yumapro and libyang etc all have similar
extensions.]

The other issue is how OC can utilize IETF modules with regexps
written in the standard dialect.  IMO this is more an implementation
matter.  The "pattern" statement formally defines a constraint on
the value space, but there is nothing that requires an implementation
to use the pattern exactly as it is specified.  A toolchain that
doesn't understand the XSD regexp dialect can simply ignore the
pattern; or if it is clever and maybe doesn't really care about
unicode it can translate the pattern to another dialect automatically;
or if it is even more clever it can translate if possible or otherwise
use a user-provided regexp in another dialect.  These are just
suggestions of course, I'm sure there are other ways of dealing with
this.

This said, I don't think it's a good idea to define an openconfig
specific extension in the ietf-routing-types document, and I also
don't think that the IETF should define a second way of defining
regexp patterns.  Having two very similar ways of doing the same thing
often leads to more problems than it solves.

/martin

---

From: Christian Hopps <[email-masked]>
Date: Wed, 22 Feb 2017 07:47:03 -0500

> This would avoid the fork of YANG that actually has happened now.

If after discussing the merits of POSIX regexp we decide that adding
POSIX regexp is useful beyond just OpenConfig, would we just create a
new posix-regex module that adds the POSIX regular expression syntax
then? If so that sounds like a great solution to me.

As to any possible 100 message long email thread that I probably don't
have time to participate in, I'd say perfect is the enemy of good, and
if the feature is optional letting the market decide is sometimes a very
effective way of judging somethings usefulness. :)

Thanks,
Chris.

---

From: Christian Hopps <[email-masked]>
Date: Wed, 22 Feb 2017 07:52:25 -0500

> so it is a bit surprising to hear that it is a show-stopper in 2017.

To be fair I think a lot of routing folks (and perhaps others) only
started trying to use YANG in the last couple of years.

I don't think it's a show-stopper (Lada's did use this term referring to
regexp without unicode support), but it's causing the Open Config folks
to define their own common type modules, if I understand the issue
correctly.

Thanks,
Chris.

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 22 Feb 2017 13:54:57 +0100

> IETF extension so that it can be readily used by IETF modules including

So someone has to write an I-D that motivates why an extension is
needed and which defines the extension in some level of detail. And
in this particular case, it is likely also desirable to include
guidelines when to use which pattern syntax.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 22 Feb 2017 14:46:49 +0100

> On 22 Feb 2017, at 10:12, Martin Bjorklund <[email-masked]> wrote:
> 

Actually, such an extension is a fork of YANG as well. Constraints are the substance of YANG and we should not change their semantics through extensions.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 14:54:10 +0100 (CET)

> guidelines when to use which pattern syntax.

But I hope we wait for the problem definition before providing a
solution...

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 22 Feb 2017 14:54:44 +0100

> On 22 Feb 2017, at 13:36, Martin Bjorklund <[email-masked]> wrote:
> 

This is a very slippery slope. If every vendor defines their own extensions that change YANG semantics, we will end up with many YANG dialects (silos) even though all will report the same YANG version.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 14:58:52 +0100 (CET)

Ladislav Lhotka <[email-masked]> wrote:
> This is a very slippery slope. If every vendor defines their own
> extensions that change YANG semantics, we will end up with many YANG
> dialects (silos) even though all will report the same YANG version.

They don't necessarily change YANG semantics.  In this particular case
it just provides a way to do exactly the same thing as 'pattern'
already does.  And any vendor can do whatever they want in their
extensions.  The result may or may not be generally useful, but that's
up to the vendor.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 22 Feb 2017 15:00:45 +0100

> On 22 Feb 2017, at 14:54, Martin Bjorklund <[email-masked]> wrote:
> 

Right. What I am still missing is an answer to your original question: are XSD regexs not powerful enough, or is language/library support insufficient?

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 22 Feb 2017 15:10:37 +0100

> On 22 Feb 2017, at 14:58, Martin Bjorklund <[email-masked]> wrote:
> 

If two YANG validators end up with with different results regarding validity of the same document, then something appears to be broken. What's then the value of YANG as a standard?

Lada

---

From: Kent Watsen <[email-masked]>
Date: Wed, 22 Feb 2017 17:08:36 +0000

> suggestions of course, I'm sure there are other ways of dealing with

Right, a server doesn't have to compile/execute the XSD 'pattern' 
expression at all, so long as it can implement the validation rules
by other means, perhaps by translating the XSD pattern to another
dialect or by just using 'C' code.  If the server's implementation
of a pattern is not a 100% accurate, then customers will report the
issue and the server will be improved.

As for Unicode support, some servers don't even support Unicode, so
translating the XSD pattern to another regex that doesn't support
Unicode would be a non-issue for these systems.  Perhaps a generic
XSD-regex to POSIX-regex convertor could be used by such systems.
Searching online, I don't find any such convertor available (but I
didn't look very hard).  Perhaps this would make for be a good 
Hackathon project? ;)

Kent // as a contributor

---

From: Rob Shakir <[email-masked]>
Date: Wed, 22 Feb 2017 18:34:32 +0000

Note, my mail didn't ask for anything to be changed. It just noted that
there are implementation complexities of having an XSD regexp dialect that
is not widely supported in non-XML libraries. This is an implementation
barrier for servers that want to use YANG to model their schema. Kent is
right, translation is an approach - but transcoding regexps is something
that adds more code to maintain for those building systems that use these
data models.

I know, from comments made elsewhere, that this isn't just something that
is an OpenConfig issue - and I'm fine if NETMOD/IETF doesn't want to work
on a solution. Merely, as someone that has spent a couple of years writing
actually writing models, and implementing tooling around YANG that's used
to run networks - I provided feedback as to what the issue was for the
systems that we're considering.

r.

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 19:43:49 +0100 (CET)

Hi,

Rob Shakir <[email-masked]> wrote:
> barrier for servers that want to use YANG to model their schema.

For my understanding, did you consider libxml2?  I would like to
understand what "implementation barrier" and "implementation
complexity" really means.

/martin

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Wed, 22 Feb 2017 19:45:02 +0100

On Wed, Feb 22, 2017 at 05:08:36PM +0000, Kent Watsen wrote:
> Unicode would be a non-issue for these systems.

I think strictly speaking, a server not supporting Unicode is not
compliant. So why care about how a non-compliant server deals with the
fact of not being compliant? I understand that for global operators
Unicode may not be important since they use English as their lingua
france (love this expression). But the technology is not just for big
global operators.

/js

---

From: Rob Shakir <[email-masked]>
Date: Wed, 22 Feb 2017 18:49:40 +0000

> complexity" really means.

Pyangbind uses lxml for some elements of its functionality - particularly
to support a lightweight XML backend for XPATH querying. It doesn't use
libxml2 for regexps. When I looked originally, I didn't find useful
language bindings to libxml2 in Python.

In other projects, we don't have any XML library dependencies, adding
libxml2 as a dependency seems overkill simply to parse regexs when there
are built in libraries in many languages without libxml2.

r.

---

From: Andy Bierman <[email-masked]>
Date: Wed, 22 Feb 2017 10:52:39 -0800

Hi,

Rob's last comment is actually quite important because SMIv2 veered off from
the mainstream and it really diminished the available tools for SNMP.
We should avoid the same fate for YANG.

Has the mainstream moved away from the XSD regexp and towards Posix regexp?
And by Posix regexp, do you mean sec. 9 of this spec:
http://pubs.opengroup.org/onlinepubs/9699919799/
(This is a really heavyweight regexp in its entirety)

YANG automation tools would be diminished if the standard pattern-stmt
was ignored and an extension was used instead. This is my main concern
with an extension.

Andy

---

From: Rob Shakir <[email-masked]>
Date: Wed, 22 Feb 2017 18:54:39 +0000

I suspect, based on this position, that this discussion will diverge. I
suspect the community of interest is some implementation community, rather
than a standards one. Taking the academic case that servers that don't
support unicode shouldn't be cared about, when there are many, real world,
systems that fall into this category is nice, but doesn't help operators
move forward.

Given the oft-cited motivation for YANG/NETCONF is a 10+ year old study of
operators, yet again, one feels that the IETF isn't able to effectively
interact with those for whom standards are more a means to an end, than the
end itself.

It's not about big operators (I've worked for smaller ones than my current
position), and I was under the impression that we were not wearing hats
within the IETF. If this isn't the case - that's fine, but I'll reserve my
right to not contribute to the discussion.

r.

On Wed, 22 Feb 2017 at 10:45 Juergen Schoenwaelder <
[email-masked]> wrote:

> Phone: +49 421 200 3587 <+49%20421%202003587>         Campus Ring 1 |
> 28759 Bremen | Germany
> Fax:   +49 421 200 3103 <+49%20421%202003103>         <
> http://www.jacobs-university.de/>
>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 19:59:50 +0100 (CET)

> language bindings to libxml2 in Python.

pyang is using libxml2 to validate the patterns.

> are built in libraries in many languages without libxml2.

Hmm.  In what way overkill?  For ConfD/NCS we actually took the regexp
part of libxml2 and we build a separate .a/.so file; the .a file is
~130K stripped.  Compared to ~3M for libxml2.  If it would help I can
publish this project on github.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 20:03:32 +0100 (CET)

> systems that fall into this category is nice, but doesn't help operators

I think that maybe the position should be that we need to care also
about implementations that *do* need unicode.  So picking a regexp
dialect that doesn't support unicode in the standard is probably not a
good idea - NOTE: I don't know if you have proposed this or not, but
others have.

/martin

---

From: Rob Shakir <[email-masked]>
Date: Wed, 22 Feb 2017 19:07:54 +0000

I'm not asserting it is not possible to do so - of course it is, and I
encourage anything to be open sourced that helps YANG adoption (hey, I'm
actually trying to do that everywhere I can).

But there are other problems too - what common tooling is available for
non-developers that are writing models? For example, there are many, many
different sites that will give you a way to validate that a regexp matches
some test content that you're interested in (e.g., http://regexr.com/,
https://regex101.com/). I have found none of these that support XSD
regexps. The key point is  that picking a road less travelled comes with
costs around the ecosystem.

I don't think anyone on this thread has asserted "The regexp forward must
be POSIX". I am definitely sympathetic to the need for unicode support.
Let's debate the merits of the different solutions - with everything in
mind, not a single implementation iff we want to change something in YANG.

OC - which targets specific use cases - simply because we are use-case
focused, may make specific decisions that work with the systems that it is
interested with a view to getting to the stage of being able to move
network management forward. The latter's what I care about most. YANG is
really just a means to an end.

r.

---

From: Kent Watsen <[email-masked]>
Date: Wed, 22 Feb 2017 20:03:34 +0000

FWIW, I found a tool that can convert XSD regex to ~23 other regex dialects (including POSIX) here:

    https://www.regexbuddy.com/xml.html

It's a commercial, and Windows-only, but at least shows that it is doable.

K.

On 2/22/17, 1:34 PM, "Rob Shakir" <[email-masked]<mailto:[email-masked]>> wrote:

Note, my mail didn't ask for anything to be changed. It just noted that there are implementation complexities of having an XSD regexp dialect that is not widely supported in non-XML libraries. This is an implementation barrier for servers that want to use YANG to model their schema. Kent is right, translation is an approach - but transcoding regexps is something that adds more code to maintain for those building systems that use these data models.

I know, from comments made elsewhere, that this isn't just something that is an OpenConfig issue - and I'm fine if NETMOD/IETF doesn't want to work on a solution. Merely, as someone that has spent a couple of years writing actually writing models, and implementing tooling around YANG that's used to run networks - I provided feedback as to what the issue was for the systems that we're considering.

r.

Kent // as a contributor

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 22 Feb 2017 21:47:29 +0100 (CET)

> Has the mainstream moved away from the XSD regexp and towards Posix regexp?

It's not that simple.  There are many dialects available.  See
http://www.regular-expressions.info/tools.html for a list.  Regardless
of which dialect you pick, someone will prefer another one.  OTOH you
can also argue that people are used to the fact that different
tools/systems use different dialects, so they adapt to whatever the
tool they are using support.

> was ignored and an extension was used instead. This is my main concern

Agreed - but I suggested this as an alternative to what they (OC) are
using now.  The current OC models use the standard "pattern"
statement, but with POSIX syntax.  An extension would be better IMO.

/martin

---

From: Jeff Tantsura <[email-masked]>
Date: Wed, 22 Feb 2017 13:37:31 -0800

Lou,

Exactly, this is what had been stated in the previous email. 

 Cheers,
Jeff

|   Once you get into potential solutions that go beyond a single module, i.e., 
|    are changes to yang, it's really best to have the discussion in the 
|   location the general Yang expertise, ie net mod.

---

From: Andy Bierman <[email-masked]>
Date: Wed, 22 Feb 2017 19:28:19 -0800

> tool they are using support.

It is important that there is a clear mandatory syntax that all YANG tools
can count on.
Pick your own flavor is what I already have to deal with for 20+ Linux
distros, MacOSX,
and Windows.

> statement, but with POSIX syntax.  An extension would be better IMO.
Obviously better if one cares about interoperability.
The extension seems reasonable, especially if it is possible
to always specify the XSD pattern-stmt when the extension is used.

I think you should put your parser from libxml2 on github.
It is more relevant than ever, since RESTCONF does not require
XML at all.  It would help vendors who use something else besides libxml2.

Andy

---

From: Alia Atlas <[email-masked]>
Date: Mon, 27 Feb 2017 13:39:00 -0500

First, while YANG models are obviously used in many places and for many
purposes other than in routers, a great amount of the work on YANG modules
in the IETF is in the Routing Area and targeted at routers.   We need YANG
modules that are practical and useful for routers and operators.

Second, there is a split happening where many OpenConfig models are
implemented on routers.  These, of course, address limited use-cases.
Doing what we can to reduce that split is useful.

Getting a list of the perceived issues is useful.  Claiming that we must
have full agreement and understanding of the problem - when it is others
who are experiencing it and making clear decisions based on their
experience of the problems - is not.  Understanding enough of the problem
to propose solutions and iterate to get to acceptable solutions that solve
the problem is important.

As you all well know, we are working on getting YANG models completed and
published as quickly as possible.  Proposing the need for slow process,
trying to put not-my-problem-fields on others' issues and implementations,
and trying to make any change sound radical (YANG 2.0????) is not
constructive.

I would welcome Jeff organizing discussion in RTGWG around the issues with
ietf-routing-types and, where they are more general, in NetMod.

IETF YANG modules done in the Routing Area MUST be useful to router vendors
and operators.

Regards,
Alia

On Wed, Feb 22, 2017 at 2:07 PM, Rob Shakir <[email-masked]> wrote:

> some test content that you're interested in (e.g., http://regexr.com/, htt
> ps://regex101.com/). I have found none of these that support XSD regexps.
> The key point is  that picking a road less travelled comes with costs
> around the ecosystem.
>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 27 Feb 2017 10:50:59 -0800

Hi,

I don't really care what process is used.
That is for the IESG to figure out.

Here is the text in RFC 7950:
https://tools.ietf.org/html/rfc7950#section-9.4.5

That's your problem, not YANG doctors, but normative text in
a standards track RFC.

I am not in favor of changing that text, but if others want it changed,
it seems that a new version of the YANG language would be required.
The [XSD-TYPES] reference is not a typo, not an error, so an errata
cannot be used to change it.

Andy

On Mon, Feb 27, 2017 at 10:39 AM, Alia Atlas <[email-masked]> wrote:

> IETF YANG modules done in the Routing Area MUST be useful to router
> vendors and operators.
>
> Regards,
> Alia
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 27 Feb 2017 19:57:39 +0100

> the problem is important.

Alia,

the pattern statement is part of a standards-track RFC (in its second
revision) and it has been implemented in running code. I believe it is
_required_ to have a clear problem statement if the pattern statement
is not good enough and needs modifications.

/js

---

From: Alia Atlas <[email-masked]>
Date: Mon, 27 Feb 2017 14:01:22 -0500

Juergen,

> is not good enough and needs modifications.

There are several ideas in this thread of ways to handle the issue - ranging
from an extension to regex translation.   Jumping to the most extreme
possibility
can easily be seen as trying to shut down the conversation.

A quick iteration of "problem, proposed solution, remaining issues, ok -
new proposed
solution & updated problem description" etc - works better for speed and
ending up
with something that we can all use.

Regards,
Alia

---

From: Andy Bierman <[email-masked]>
Date: Mon, 27 Feb 2017 11:10:12 -0800

> with something that we can all use.
The only problem statement I have heard is that the XSD pattern
is too hard to implement, but several implementations exist countering
the claim that it is too hard.

Assuming there is agreement to use a different regexp spec,
then what is it?  Please provide a URL to the regexp spec that
is being proposed instead.

It is very common in the IETF to ask "what problem are you trying
to solve", and "what is your solution proposal".  You act like these
questions are inappropriate.

Andy

---

From: Alia Atlas <[email-masked]>
Date: Mon, 27 Feb 2017 14:19:53 -0500

Hi Andy,

> the claim that it is too hard.

Earlier in this thread, Jeff mentioned that he is talking more and working
on clarifying
the details of the problem.  Rob said that he's seen this issue during
implementations;
it might be that the smaller library suggested by Martin can help.  I
currently don't know
and am awaiting more details.

Please wait to form judgments.

> is being proposed instead.

First, I am responding with an AD hat - not as individual contributor.
What I have read is that an implementation is using a different regexp
because
of the issue - and this is driving the OpenConfig depedencies away from
ietf-routing-types.

> questions are inappropriate.

I am quite aware of methods for engaging productively and for invoking
excess process in
an effort to slow needed work down.

Regards,
Alia

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 27 Feb 2017 20:24:51 +0100

> First, I am responding with an AD hat - not as individual

Thanks Alia for helping us to get a better understanding of the
problem space and the solution space. I am looking forward to Jeffs
and Robs input.

/js

---

From: Lou Berger <[email-masked]>
Date: Mon, 27 Feb 2017 14:31:02 -0500

Just an FYI - In today's RTG Area YANG Arch DT meeting we discussed
Jeff's plans and it includes presentation on requirements in Chicago. 
Let's given him some time to collect this and put it in a good form for
further discussion.   We'll ID WG in the 1-2 weeks for this presentation
after more discussions with chairs and (if needed) ADs.

Lou

---

From: Andy Bierman <[email-masked]>
Date: Mon, 27 Feb 2017 11:27:07 -0800

> an effort to slow needed work down.
These same "tools" are also used to build consensus.
Genuine lack of consensus should not be confused with excess process.

Andy