Subject: some proposed edits for the next (final!?) draft

From: Michael Mealling <[email-masked]>
Date: Thu, 17 Aug 2000 09:42:17 -0400

Coming out of Pittsburgh I have a short list of the edits that need
to be made to the protocol and URI documents. I hope to get these
done in the next few days. Please comment on these if you have a relevant
opinion:

1) In the HTTP Transport section there is no spec on what PATH you actually
post to. It seems that everyone interpreted this to be the "/" path
which is kind of neat that we all thought that way.

2) A few new status messages concerning datasets 'n stuff.

3) white space needs to be encoded if its important since XML collapses it.

4) in the DTD the error tag needs to be renamed to 'status'.

5) The current DTD allows you to have a resourcedescriptor with a required
serviceref/datasetref and not have a service block. Roland and I think
we have a version worked out that looks like this:

<!ELEMENT results (
( status? | ( service+, (status*, resourcedescriptor*, referral* )* )* )>

PLEASE look at this closely! The boundary conditions are hard to see.

6) The URI doc's ABNF needs to be fixed to show that you can't include
properties with an ID query.

7) A short list of just clarifications and/or rewording of particular
sentences....

-MM

--
--------------------------------------------------------------------------------
Michael Mealling        |      Vote Libertarian!       | www.rwhois.net/michael
Sr. Research Engineer   |   www.ga.lp.org/gwinnett     | ICQ#:         14198821
Network Solutions       |          www.lp.org          |  [email-masked]

---

From: "Marshall L. Moseley" <[email-masked]>
Date: Thu, 17 Aug 2000 09:41:18 -0700

>>) white space needs to be encoded if its important since XML collapses
it.>>

Encoding white space would be a mistake.  That means that any extra spaces
would give different common names different meanings.  Two spaces between
words, or worse yet, a space after a phrase, would produce different
results.  So.

    Acme<sp>Widget
        or
    Acme<sp><sp>Widget
        or
    Acme<sp>Widget<sp>

Would all produce different results.  But they are all common typing
mistakes, and it's pretty certain the person typing is referring to one
thing.

--Marshall

---

From: Michael Mealling <[email-masked]>
Date: Thu, 17 Aug 2000 13:32:24 -0400

> mistakes, and it's pretty certain the person typing is referring to one

I think I didn't emphasize a particular word enough:

The encoding of whitespace should be done only if the particular
application thinks its important to that particular application.
Its just a warning, not a requirement to encode all whitespace.

-MM

---

From: Leslie Daigle <[email-masked]>
Date: Thu, 17 Aug 2000 22:29:39 -0400

Howdy,

The one additional issue that I don't think we've resolved is
that of character encoding.  I think there are 2 separate issues
we're trying to reconcile:

        . different contexts (e.g., XML, URIs, various databases) have
          certain predispositions towards character encodings -- simply
          not UTF-8, or different representations of UTF-8 bytes (e.g.,
          the %%-encoding in a URL).

        . it is not "well-understood" how to do interoperability
          if multiple character encodings are permitted -- it
          makes the issue of "canonical form" both more critical and
          harder to define

So, I _think_ what we're trying to say is that a common name "is"
a stream of UTF-8 bytes (i.e., is valid UTF-8 encoded _something_),
although representations of the UTF-8 may vary by context -- e.g.
%%-hex encoding in URLs.

Here's a proposed re-write -- probably awful, but please suggest
improvements in the light of the above.

"3.4 Character encoding

   The UTF-8 character encoding is the basis for all common names.

   However,
   o  Within an XML query or response, the UTF-8 may be represented
      by numeric character references.
   o  Within a CNRP URI, the representation of the common name is
      hex-encoding of the UTF-8 common name -- to meet requirements of
      URI syntax.
   o  Similarly, in other environments (e.g., particular display
      devices) the common name may be presented/accepted in a different
      character encoding, but the display/entry software is responsible
      for the correct translation, and non-UTF-8 representations
      are never considered normative."

Fire away...

Leslie.

P.S.:  Michael -- did you catch Martin's editorial comment that all
instances of "character set [encoding]" should be "character encoding"?

P.P.S.:  FWIW, here's the existing wording of the draft:

> 3.4 Character sets
>
>    The character set encoding of a common name is always UTF-8.
>
>    Specifically:
>    o  Within an XML query or response, the common name is represented
>       in UTF-8, WITHOUT any hex-encodings
>    o  Within a CNRP URI, the REPRESENTATION of the common name is
>       hex-encoding of the UTF-8 common name -- because this is an issue
>       for the CNRP URI, not the common name.
>    o  Similarly, in other environments (e.g., particular display
>       devices) the common name may be _presented_ in a different
>       character-set, but that is a display/data entry issue. It is not
>       a "common name" unless it's encoded in UTF-8.
>

--

-------------------------------------------------------------------
"Reality with a delicate splash of the imaginary...
    ... or was that the other way around?"
   -- ThinkingCat

Leslie Daigle
[email-masked]
-------------------------------------------------------------------

---

From: "Martin J. Duerst" <[email-masked]>
Date: Fri, 18 Aug 2000 12:42:28 +0900

>3) white space needs to be encoded if its important since XML collapses it.

Such a general statement is just not true. White space treatment in
XML is quite complex, but usually does what you want, and can
be influenced. There is also a difference between textual
content and attribute values, and so on.

And I wonder what you mean by 'encoded'. Any specific proposals
that where floated?

>PLEASE look at this closely! The boundary conditions are hard to see.

Just a quick look shows that there are four '('s, but only three ')'s.
And in the above, (status*, resourcedescriptor*, referral* )* can (and should)
be simplified to (status | resourcedescriptor | referral )*.

Please describe exactly what you want (and what you don't want),
otherwise it's impossible to check this.

Regards,   Martin.

---

From: "Martin J. Duerst" <[email-masked]>
Date: Fri, 18 Aug 2000 13:16:26 +0900

>           makes the issue of "canonical form" both more critical and

Up to here, it makes sense.

>a stream of UTF-8 bytes (i.e., is valid UTF-8 encoded _something_),

Sorry, but this is just completely wrong. UTF-8 is an encoding.
If you transcode e.g. to UTF-16, it's no longer UTF-8. Also,
a common name 'is' a string of characters. Defining it as a string
of bytes is a straightforward layer violation. It is also a
violation of the principle that the IETF specifies protocols,
and not something else (such as the 'nature' of a common name).

>although representations of the UTF-8 may vary by context -- e.g.

Please don't be confused by the fact that the %HH encoding of URIs
is based on UTF-8. There is some 'policy-level' reason (and some
generic technical reasons) for this fact, but if e.g. URIs
were defined to use %uABCD to represent characters, your whole
way of explaining things would just break down.

>       by numeric character references.

As I have said earlier, within XML, there are only characters.
There is no way to represent UTF-8 by numeric character references.

>       are never considered normative."

So I propose the following rewrite:

3.4 Character encoding

   To guarantee interoperability, the following provisions apply:

   o  XML queries and responses MUST be encoded as UTF-8.

      Note: As in any XML document, numeric character references may
            be used.

   o  The encoding of characters in the CNRP URI is based on UTF-8;
      for details, please see [URI I-D].

   o  Interfaces (e.g. APIs) for CNRP must clearly document what encoding
      they use for each of their parameters. In general, UTF-8 is
      preferable, but if the dominant string representation is different
      (e.g. UTF-16 for Java), it should be followed.

   o  User interfaces (display, input) should allow interaction in
      terms of characters, not bytes or codepoints. Appropriate care
      must be given to conversion between the character encoding in
      the protocol and the character encoding in the user interface
      if these differ.

What do you think?        Regards,   Martin.

---

From: Michael Mealling <[email-masked]>
Date: Fri, 18 Aug 2000 07:18:51 -0400

> content and attribute values, and so on.

Sure. And I don't want to repeat all of that in this document. I just
want to warn implementors that they can't just 'use' XML blindly since,
as you say, "White space treatment in XML is quite complex".

> And I wonder what you mean by 'encoded'. Any specific proposals

None other than what XML already does with whitespace. I guess I used
the wrong word yet again. What word is used to meet this definition:
the function for making a particular piece of whitespace (space, tab, crlf)
and making it so that XML doesn't treat it as whitespace but instead
treats it as some immutable code point.

Is there a vocabularly somewhere I can normalize to because I'm apparently
picking all the wrong words....

-MM

---

From: Leslie Daigle <[email-masked]>
Date: Fri, 18 Aug 2000 11:34:54 -0400

Howdy,

Executive summary --  proposed revision:

"3.4 Character encoding

  Any interfaces electing to present/accept protocol elements in other
  representations are responsible for accurate transcoding for
  use in CNRP protocol elements, per the above provisions.
"

Discussion:

"Martin J. Duerst" wrote:
> Sorry, but this is just completely wrong.

Sorry, but that's just completely provoking.  [Further snarky reply
deleted.]

I believe you're thinking "name" (which is independent of
representation) and I'm talking about protocol elements (for which
it is valid to define a limited set of representations).

The 3 cases where I believe it's particularly critical for CNRP to
be unequivocal about representation are:

        . messages (query & results):  XML (subset?)
        . common name property value:  UTF-8
        . cnrp URLs:  UTF-8, encoded per URI rules if necessary

What happens outside the protocol is irrelevant -- you can represent
common names by pebble patterns on Venice Beach for all I care,
as long as you know how to represent that in the appropriate encoding
when it comes time to use CNRP.  All I've ever been trying to
make clear is that  a) there is a restricted set of encodings that
are considered valid within the scope of CNRP, and b) any bytes
brought within the context of CNRP will be interpreted according to
those encoding rules (which implicitly puts the onus on any API
or interface-writer to 'do the right thing').

> UTF-8 is an encoding.
> If you transcode e.g. to UTF-16, it's no longer UTF-8.

Sure.  And if UTF-16 isn't a valid encoding for the common name
property value, the transcoded value is no longer a valid
protocol element.

> Also,
> of bytes is a straightforward layer violation.

Not for "common name" as protocol element. (Forget that the tag
of the element contains the word "name").

Leslie.

--

Leslie Daigle
[email-masked]