Subject: [EAI] ISSUE: dsn HEXPOINT

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 25 Sep 2007 12:45:56 +0200

Hi, 

I've now read <http://perldoc.perl.org/perluniintro.html>
about the \x{...} notation, and I'm not sure that it's
clear what \x{5C} and \x{80} up to \x{FF} actually mean.

The example on the perluniintro page is:

| perl -e 'print "\x{DF}\n", "\x{0100}\x{DF}\n"'
| produces a fairly useless mixture of native bytes and
| UTF-8, as well as a warning

Apparently this construct enumerates octets in the local
charset up to \x{FF}, switching to Unicode points only
for \x{100} up to \x{10FFFF}.

That's some kind of backward compatibility feature, it
can't be removed by decree in an RFC, perl users would
expect that this notation works as is for them.

If that's correct this notation is also unsuited for
http://tools.ietf.org/html/draft-klensin-unicode-escapes

Minor point, the perl manual page apparently doesn't
say that leading zeros are "verboten", but the ABNF
in the I-D specifies this.

 Frank

_______________________________________________
IMA mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ima

---

From: John C Klensin <[email-masked]>
Date: Tue, 25 Sep 2007 11:56:01 -0400

> for \x{100} up to \x{10FFFF}.

As I read this, it is all tied up with the environment, the
presence of absence of a "PerlIO layer" and the presence of
absence of an "encoding" pragma (see
http://perldoc.perl.org/perluniintro.html#Legacy-Encodings).
The "legacy" material seems to say that, if the encoding is
specified, Perl will perform legacy-> Unicode conversion.  I
would hope that something like
   use encoding 'UTF-8' ;
would cause all of the right things to happen in Perl, but we
don't have that construction in DSN.  And I agree that the
description in the Perl manual is somewhere between ambiguous
and utterly confusing.

> http://tools.ietf.org/html/draft-klensin-unicode-escapes

I have now fixed the next version of that draft, which I'm
working on this week anyway, to note this problem, refer to the
Perl document, and issue a general "there be dragons" comment.

> in the I-D specifies this.

That is the ABNF in "dsn", not the ABNF in "unicode-escapes",
yes?  The latter seems to be consistent with the Perl man page.  

But I think all of this suggests that DSN needs either a lot
more specificity (possibly defining some reasonable and clear
rules and describing them as Perl-like, rather than as Perl
rules) or a different syntax entirely.

    john

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 25 Sep 2007 20:04:51 +0200

> Perl document, and issue a general "there be dragons" comment.

Thanks.  I just met the "LWSP-dragon" again, it tried to hide
itself as 1*FWS in the utf8headers-07 draft.

> yes?  The latter seems to be consistent with the Perl man page.

Yes, the ABNF in "dsn".  I guess the authors were forced to
eliminate leading zeros, because they wanted to eliminate
surrogates in ABNF.  IMO they could use the simple ABNF in
your draft, and let other texts discuss surrogates.  

Ideally your I-D is published before "dsn", and "dsn" could
then simply reference it.  

 Frank

---

From: John C Klensin <[email-masked]>
Date: Tue, 25 Sep 2007 14:32:30 -0400

> then simply reference it. 

I hope to get my (newer) draft posted this week.  Assuming that
there are no major comments, it will presumably then go out for
a four-week IETF Last Call.   But, if DSN is modified to reflect
WG LC comments and reposted, it could presumably go into a
two-week (as a WG document) IETF Last Call as early as next
week.  So, whether the race here will be won or lost is going to
be a close call.   I definitely do not think it worthwhile to
hold up an _experimental_ DSN document for the "Unicode escapes"
piece, but that is just my opinion.

    john

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 27 Sep 2007 13:02:55 +0100

Frank Ellermann wrote:

>surrogates in ABNF.
Yes.

>  IMO they could use the simple ABNF in
I slightly prefer the current syntax. Other opinions?

---

From: Harald Alvestrand <[email-masked]>
Date: Thu, 27 Sep 2007 15:06:54 +0200

Since the DSN document as of version -03 contains no reference to Perl, 
and has all the documentation for HEXPOINT internal to itself, I fail to 
see how this is an issue.

No issue # recorded.