Subject: Re: [yang-doctors] YANG Tool Requests

From: Benoit Claise <[email-masked]>
Date: Mon, 10 Jul 2017 10:14:16 +0200

Hi Acee,
> Hi Benoit, Mahesh,
>
> With the risk of being chastised for not contributing to the 
> development ;^), I have a couple feature and fix requests.
You won't be chastised. See you at the hackathon ;-)
>
>    For www.yangvalidator.com:
>
>     1. Fix: It doesn’t pick up the latest draft version of models. In 
> fact, in some cases there are really out of date, e.g., 
> ietf-routing-types.yang.
ok. Noted.
>     2. Enhancement: Allow upload of multiple co-dependent drafts along 
> with the multiple models capability.
ok. Noted.

>
>   A new Web based tool that would allow a set of YANG patterns to be 
> input and tested against input strings for validity.
Yes, this is on our to do list.
Martin Bjorklund has got some code that checks for regex, but we need a 
GUI around it.
Something like https://regex101.com/

Regards, Benoit
> This would be similar to a lot of the web-based tools that exist today 
> for POSIX regular expressions.
>
> Thanks,
> Acee
>
>
>

---

From: Benoit Claise <[email-masked]>
Date: Mon, 17 Jul 2017 08:15:34 +0200

Hi Acee
> Martin Bjorklund has got some code that checks for regex, but we need 
> a GUI around it.
Results from the hackathon.
Radek developed such a tool - yangre(1) is part of the libyang package

    Here is yangre help:
    $ yangre -h
    YANG Regular Expressions processor.
    Usage:
         yangre [-hvV] -p <regexp1> [-i] [-p <regexp2> [-i] ...] <string>

    Returns 0 if string matches the pattern(s), 1 if not and -1 on error.

    Options:
       -h, --help              Show this help message and exit.
       -v, --version           Show version number and exit.
       -V, --verbose           Print the processing information.
       -i, --invert-match      Invert-match modifier for the closest preceeding
                               pattern.
       -p, --pattern="REGEXP"  Regular expression including the quoting,
                               which is applied the same way as in a YANG module.

    Examples:
       pattern "[0-9a-fA-F]*";      -> yangre -p '"[0-9a-fA-F]*"' '1F'
       pattern '[a-zA-Z0-9\-_.]*';  -> yangre -p "'[a-zA-Z0-9\-_.]*'" 'a-b'
       pattern [xX][mM][lL].*;      -> yangre -p '[xX][mM][lL].*' 'xml-encoding'

And Pieter (also in cc) integrated a GUI around.
See https://www.yangcatalog.org/
     => A YANG Regex Validator <http://yangcatalog.org:5000/>, a YANG 
regular expression validator to experiment with W3C YANG "pattern" 
statements

Features:
     - one or multiple YANG patterns
     - invert match
     - two outputs: one from YANGRE (yanglint) and W3CGREP (code 
provided by Martin Bjorklund)
     - color-coded results: green/red

Please test the code, provide feedback on the GUI, and let us know.

Regards, Benoit

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Thu, 27 Jul 2017 20:00:09 +0000

The link to the YANG pattern validator seems to be gone - YANG Regex Validator<http://yangcatalog.org:5000/>

Thanks,
Acee

From: "Benoit Claise (bclaise)" <[email-masked]<mailto:[email-masked]>>
Date: Monday, July 17, 2017 at 2:15 AM
To: Acee Lindem <[email-masked]<mailto:[email-masked]>>, "Mahesh Jethanandani (mahesh)" <[email-masked]<mailto:[email-masked]>>
Cc: RTG YANG Design Team <[email-masked]<mailto:[email-masked]>>, Ing-Wher Chen <[email-masked]<mailto:[email-masked]>>, "Carl Moberg (camoberg)" <[email-masked]<mailto:[email-masked]>>, YANG Doctors <[email-masked]<mailto:[email-masked]>>, "Joe Clarke (jclarke)" <[email-masked]<mailto:[email-masked]>>, Henrik Levkowetz <[email-masked]<mailto:[email-masked]>>, "Pieter Lewyllie (pilewyll)" <[email-masked]<mailto:[email-masked]>>, Radek Krejčí <[email-masked]<mailto:[email-masked]>>
Subject: Re: YANG Tool Requests => regex validator

Hi Acee

  A new Web based tool that would allow a set of YANG patterns to be input and tested against input strings for validity.
Yes, this is on our to do list.
Martin Bjorklund has got some code that checks for regex, but we need a GUI around it.
Something like https://regex101.com/
Results from the hackathon.
Radek developed such a tool - yangre(1) is part of the libyang package

Here is yangre help:
$ yangre -h

Options:

Examples:

And Pieter (also in cc) integrated a GUI around.
See https://www.yangcatalog.org/
    => A YANG Regex Validator<http://yangcatalog.org:5000/>, a YANG regular expression validator to experiment with W3C YANG "pattern" statements

Features:
    - one or multiple YANG patterns
    - invert match
    - two outputs: one from YANGRE (yanglint) and W3CGREP (code provided by Martin Bjorklund)
    - color-coded results: green/red

Regards, Benoit

---

From: "Pieter Lewyllie (pilewyll)" <[email-masked]>
Date: Fri, 28 Jul 2017 09:16:34 +0000

Hi Acee,

The validator can be found at https://yangcatalog.org/yangre

Cheers,
Pieter

On 27 Jul 2017, at 22:00, Acee Lindem (acee) <[email-masked]<mailto:[email-masked]>> wrote:

Thanks,
Acee

Hi Acee

Here is yangre help:
$ yangre -h

Options:

Examples:

Features:

Regards, Benoit

---

From: Benoit Claise <[email-masked]>
Date: Fri, 28 Jul 2017 16:53:07 +0200

Best way is to go via the yangcatalog.org => YANG Regex Validator

B.
> The link to the YANG pattern validator seems to be gone - YANG Regex 
> Validator <http://yangcatalog.org:5000/>
>
> Thanks,
> Acee
>
>
>
> To: Acee Lindem <[email-masked] <mailto:[email-masked]>>, "Mahesh 
> Jethanandani (mahesh)" <[email-masked] <mailto:[email-masked]>>
> Cc: RTG YANG Design Team <[email-masked] 
> <mailto:[email-masked]>>, Ing-Wher Chen 
> <[email-masked] <mailto:[email-masked]>>, "Carl 
> Moberg (camoberg)" <[email-masked] <mailto:[email-masked]>>, 
> YANG Doctors <[email-masked] <mailto:[email-masked]>>, 
> "Joe Clarke (jclarke)" <[email-masked] <mailto:[email-masked]>>, 
> Henrik Levkowetz <[email-masked] <mailto:[email-masked]>>, 
> "Pieter Lewyllie (pilewyll)" <[email-masked] 
> <mailto:[email-masked]>>, Radek Krejčí <[email-masked] 
> <mailto:[email-masked]>>
>>>       A new Web based tool that would allow a set of YANG patterns
>>>     to be input and tested against input strings for validity.
>         => A YANG Regex Validator <http://yangcatalog.org:5000/>, a
>     YANG regular expression validator to experiment with W3C YANG
>     "pattern" statements
>
>
>     Features:

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Fri, 28 Jul 2017 15:23:38 +0000

Hi Pieter,

I have set up a link to the tool with my other YANG links. I’m having some success with the YANGRE output but have not matched anything against W3CGREP.

It seems that YANGRE allows the pattern to be concatenating strings using “+” which is what is needed to cut and paste a multi-line pattern directly from a .yang file. Correct?  I’d ditch W3CGREP.

Even with YANGRE, I’m trying a very basic pattern matching the pattern <2-octet-hex>:<6 octet hex>. The first 2-octet number cannot be 0-2 since these are assigned.

Pattern:  '(([3-9a-fA-F]|[1-9a-fA-F][0-9a-fA-F]{1,3}):[0-9a-fA-F]{1,12})’

I’d expect this to match 3:0 but it doesn’t.

From: "Pieter Lewyllie (pilewyll)" <[email-masked]<mailto:[email-masked]>>
Date: Friday, July 28, 2017 at 5:16 AM
To: Acee Lindem <[email-masked]<mailto:[email-masked]>>
Cc: "Benoit Claise (bclaise)" <[email-masked]<mailto:[email-masked]>>, "Mahesh Jethanandani (mahesh)" <[email-masked]<mailto:[email-masked]>>, RTG YANG Design Team <[email-masked]<mailto:[email-masked]>>, Ing-Wher Chen <[email-masked]<mailto:[email-masked]>>, "Carl Moberg (camoberg)" <[email-masked]<mailto:[email-masked]>>, YANG Doctors <[email-masked]<mailto:[email-masked]>>, "Joe Clarke (jclarke)" <[email-masked]<mailto:[email-masked]>>, Henrik Levkowetz <[email-masked]<mailto:[email-masked]>>, Radek Krejčí <[email-masked]<mailto:[email-masked]>>
Subject: Re: YANG Tool Requests => regex validator

Hi Acee,

Cheers,
Pieter

Thanks,
Acee

Hi Acee

Here is yangre help:
$ yangre -h

Options:

Examples:

Features:

Regards, Benoit

---

From: "Joe Clarke" <[email-masked]>
Date: Fri, 28 Jul 2017 12:23:31 -0400

On 28 Jul 2017, at 11:23, Acee Lindem (acee) wrote:

> Hi Pieter,
>
>
>
>
> I have set up a link to the tool with my other YANG links. I’m 
> having some success with the YANGRE output but have not matched 
> anything against W3CGREP.
>
>
>
>
> It seems that YANGRE allows the pattern to be concatenating strings 
> using “+” which is what is needed to cut and paste a multi-line 
> pattern directly from a .yang file. Correct?  I’d ditch W3CGREP.
>
>
>
>
> Even with YANGRE, I’m trying a very basic pattern matching the 
> pattern &lt;2-octet-hex&gt;:&lt;6 octet hex&gt;. The first 2-octet 
> number cannot be 0-2 since these are assigned.
>
>
>
>
> Pattern:  
> '(([3-9a-fA-F]|[1-9a-fA-F][0-9a-fA-F]{1,3}):[0-9a-fA-F]{1,12})’
>
>
>
>

YANGRE is complaining about the count qualifiers {1,3} and {1,12}.  If I 
remove those, the 3:0 simplistic pattern matches in both.

The problem is with quoting (or the lack of it in this case).  When I 
play with yangre on the command line, I get this to work:

yangre -p 
‘”(([3-9a-fA-F]|[1-9a-fA-F][0-9a-fA-F]{1,3}):[0-9a-fA-F]{1,12})”’ 
3:0

Same for the UI.  If I put the pattern in double quotes, it works.

Pieter is working on adding hints to the UI on how quoting works with 
respect to patterns.

Joe

>
> **
> **
>
>
>
>
> From: "Pieter Lewyllie (pilewyll)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;
> To: Acee Lindem &lt;[[email-masked]](<mailto:[email-masked]>)&gt;
> Cc: "Benoit Claise (bclaise)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, "Mahesh 
> Jethanandani (mahesh)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, RTG YANG Design 
> Team 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, 
> Ing-Wher Chen 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, 
> "Carl Moberg (camoberg)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, YANG 
> Doctors 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, "Joe 
> Clarke (jclarke)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, Henrik 
> Levkowetz 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, Radek 
> Krejčí &lt;[[email-masked]](<mailto:[email-masked]>)&gt;
> Subject: Re: YANG Tool Requests =&gt; regex validator
>
>
>
>
>
>> Hi Acee,
>>
>>
>
>>
>> The validator can be found at <https://yangcatalog.org/yangre>
>>
>>
>
>>
>> Cheers,
>>
>> Pieter
>>
>>
>
>>
>>> On 27 Jul 2017, at 22:00, Acee Lindem (acee) 
>>> &lt;[[email-masked]](<mailto:[email-masked]>)&gt; wrote:
>>>
>>>
>
>>>
>>> The link to the YANG pattern validator seems to be gone - [YANG 
>>> Regex Validator](<http://yangcatalog.org:5000/>)
>>>
>>>
>
>>>
>>> Thanks,
>>>
>>> Acee
>>>
>>>
>
>>>
>>>
>
>>>
>>>
>
>>>
>>> From: "Benoit Claise (bclaise)" 
>>> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;
> To: Acee Lindem &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, 
> "Mahesh Jethanandani (mahesh)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;
> Cc: RTG YANG Design Team 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, 
> Ing-Wher Chen 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, 
> "Carl Moberg (camoberg)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, YANG 
> Doctors 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, "Joe 
> Clarke (jclarke)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, Henrik 
> Levkowetz 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, "Pieter 
> Lewyllie (pilewyll)" 
> &lt;[[email-masked]](<mailto:[email-masked]>)&gt;, Radek 
> Krejčí &lt;[[email-masked]](<mailto:[email-masked]>)&gt;
> Subject: Re: YANG Tool Requests =&gt; regex validator
>
>>>
>>>
>
>>>
>>>> Hi Acee
>
>>>>
>>>>>
>
>>>>>
>>>>>>
> Something like <https://regex101.com/>
>
>>>>
> See <https://www.yangcatalog.org/>
>     =&gt; A [YANG Regex Validator](<http://yangcatalog.org:5000/>), a 
> &lt;ggfnbnnkddhlaoko.png&gt;
> Features:
>     \- one or multiple YANG patterns
>     \- invert match
>     \- two outputs: one from YANGRE (yanglint) and W3CGREP (code 
>     \- color-coded results: green/red
>
>>> &lt;ggfnbnnkddhlaoko.png&gt;
>>
>>

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Fri, 28 Jul 2017 18:03:33 +0000

Hi Joe, 

>Same for the UI.  If I put the pattern in double quotes, it works.

https://yangcatalog.org/yangre doesn’t work for me, no matter how I quote
it.

I tried with the inet:ipv4-address pattern and I did get this to work with
W3CGREP but not yangre no matter how I quoted it. For W3CGREP, I had to
remove all the quoting and concatenation. This could be error prone for a
very complex pattern.

Thanks,
Acee

---

From: "Joe Clarke" <[email-masked]>
Date: Fri, 28 Jul 2017 14:12:28 -0400

> remove all the quoting and concatenation. This could be error prone 

Hmmm.  This pattern from ietf-inet-types (note: I had the single quotes 
in the UI):

'(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(%[\p{N}\p{L}]+)?'

Works for me with this string (only with YANGRE):

192.168.1.1

Joe

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Fri, 28 Jul 2017 18:23:59 +0000

From: "Joe Clarke (jclarke)" <[email-masked]<mailto:[email-masked]>>
Date: Friday, July 28, 2017 at 2:12 PM
To: Acee Lindem <[email-masked]<mailto:[email-masked]>>
Cc: "Pieter Lewyllie (pilewyll)" <[email-masked]<mailto:[email-masked]>>, "Benoit Claise (bclaise)" <[email-masked]<mailto:[email-masked]>>, "Mahesh Jethanandani (mahesh)" <[email-masked]<mailto:[email-masked]>>, RTG YANG Design Team <[email-masked]<mailto:[email-masked]>>, Ing-Wher Chen <[email-masked]<mailto:[email-masked]>>, "Carl Moberg (camoberg)" <[email-masked]<mailto:[email-masked]>>, YANG Doctors <[email-masked]<mailto:[email-masked]>>, Henrik Levkowetz <[email-masked]<mailto:[email-masked]>>, Radek Krejčí <[email-masked]<mailto:[email-masked]>>
Subject: Re: YANG Tool Requests => regex validator

Hmmm. This pattern from ietf-inet-types (note: I had the single quotes in the UI):

'(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(%[\p{N}\p{L}]+)?'

192.168.1.1

Nope – exact same pattern and test string fails for me. If I unquote it, it works with W3CGREP.

Thanks,
Acee

Joe

---

From: "Pieter Lewyllie (pilewyll)" <[email-masked]>
Date: Sat, 29 Jul 2017 10:06:40 +0000

I get the same results as Joe stated before.

I’m wondering why YANGRE requires extra quotes to match with the {3} quantifier.

Using the GUI:

(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]).){3} matches 192.1.1. on W3CGREP, but not on YANGRE ( yangre error: Invalid keyword "3”. )

'(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]).){3}’ matches 192.1.1. on YANGRE, but not on W3CGREP (which is expecting the ‘ in the test string)

The same behaviour can be observed in the CLI, you just have to add wrap the expressions with appropriate quotes.

Radek, do you have an idea why the YANGRE fails with the quantifier? Here are a couple of CLI examples that I observed:
yangre -p "(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(%[\p{N}\p{L}]+)?" "192.168.0.1"
yangre error: Invalid keyword "3”.

/yangre -p "'(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(%[\p{N}\p{L}]+)?'" “192.168.0.1”
matches (no output)

When inverting the quotes: (single quotes on the outside)
yangre -p '"(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(%[\p{N}\p{L}]+)?"' "192.168.0.1"
yangre error: Invalid keyword "(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(%[\p{N}\p{L}]+)?”".

yangre -p '(([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5]).){3}([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(%[\p{N}\p{L}]+)?' "192.168.0.1"
yangre error: Invalid keyword "3".

Cheers,
Pieter

On 28 Jul 2017, at 20:23, Acee Lindem (acee) <[email-masked]<mailto:[email-masked]>> wrote:

192.168.1.1

Thanks,
Acee

Joe

---

From: "Acee Lindem (acee)" <[email-masked]>
Date: Sat, 29 Jul 2017 18:40:35 +0000

Hi Pieter,
Joe and I had taken the thread offline. Once I cleared my cache and history, I got the ipv4-address pattern to work.
Thanks,
Acee

From: "Pieter Lewyllie (pilewyll)" <[email-masked]<mailto:[email-masked]>>
Date: Saturday, July 29, 2017 at 6:06 AM
To: Acee Lindem <[email-masked]<mailto:[email-masked]>>, Radek Krejčí <[email-masked]<mailto:[email-masked]>>, "Joe Clarke (jclarke)" <[email-masked]<mailto:[email-masked]>>
Cc: "Benoit Claise (bclaise)" <[email-masked]<mailto:[email-masked]>>, "Mahesh Jethanandani (mahesh)" <[email-masked]<mailto:[email-masked]>>, RTG YANG Design Team <[email-masked]<mailto:[email-masked]>>, Ing-Wher Chen <[email-masked]<mailto:[email-masked]>>, "Carl Moberg (camoberg)" <[email-masked]<mailto:[email-masked]>>, YANG Doctors <[email-masked]<mailto:[email-masked]>>, Henrik Levkowetz <[email-masked]<mailto:[email-masked]>>
Subject: Re: YANG Tool Requests => regex validator

Using the GUI:

Cheers,
Pieter

192.168.1.1

Thanks,
Acee

Joe