Subject: [Cbor] Regular Expression comments on draft-greevenbosch-appsawg-cbor-cddl-08

From: Sean Leonard <[email-masked]>
Date: Mon, 25 Jul 2016 02:45:00 -0700

While reviewing draft-greevenbosch-appsawg-cbor-cddl-08, I wanted to make a couple of comments. Not sure if this is the best place for them? These comments are specific to regular expression issues.

The annotation .regexp (B.1.3.) seems quite powerful; however, it is underspecified. “PCRE regular expression” is kind of redundant, since the RE in PCRE means “regular expression”. Actually PCRE is a superset of standardized regular expression languages (such as those found in the POSIX or ECMA-262 aka Javascript standards), which significantly limits interoperability to those who actually import the PCRE engine itself. This may not be feasible for constrained devices. Overall this needs to be better specified. While we are at it, “.regexp” is not as appropriate as “.regex”, because PCRE itself exclusively refers to it as “regex”, never “regexp”. (POSIX also refers to it as “regex”, e.g, <regex.h>. “RegExp” is a JavaScript/ECMAScript term.

I strongly disagree with the regular expression being “anchored on both sides”. This means that ^ and $ are at the ends of every RE. This is bad because if you don’t want anchoring, or if you want mixed anchoring and non-anchoring, it is very non-intuitive to achieve. I shall use the verbs “moor” and “unmoor” to refer to adding and removing anchors ^ and $ (nouns).

Suppose you want mixed anchoring, namely, the expression:
foo|^bar$

means that the tstr can contain “foo” anywhere, or must be comprised of exactly “bar”.

If you assume anchors, then you have:

(^)foo|^bar($)

First of all, this leads to unintended behavior because you can actually sandwich unmoored expressions in between large quantities of alternatives.

How do you unmoor the first half of the alternative? It is proposed that you use .*, as:

(^).*foo|^bar($)

.* is greedy; you may not want that behavior: it may be extremely inefficient. Furthermore, .* sometimes consumes newlines and sometimes does not, depending on switches. We see that ^bar is explicitly moored on the left end, but not explicitly on the right end (it’s implicit). But if we switched the order of the alternatives, it would be bar$ instead (with the ^ implicit). It could be said that | unmoors the expressions. This is confusing.

Overall, the production after .regex(p) should just be a standard (Perl-compatible) regular expression, sandwiched between / / and with optional modifiers, namely i (case-insensitive) tacked on to the end. (Currently, optional modifiers such as i have way to be expressed in the draft-08 syntax.) This is how JavaScript does it, even though JavaScript isn’t Perl, and it works great. Syntax highlighters can figure out that it’s a RE and do the right coloring things with it too.

I also propose a binary regex(p) annotation (“.bregex” ?) on bstr, whose domain is the octets \0 - \xFF, in contrast to tstr, whose domain include the Unicode scalar values in \0 - \u{10FFFF}.

(Compare with the “bregex” provided in draft-bormann-cbor-tags-oid.)

Best regards,

Sean