Subject: Definition of utf8string

From: David Mazieres <[email-masked]>
Date: Mon, 6 Mar 2000 13:27:55 -0500 (EST)

The draft NFS spec defines a utf8string as:

   utf8string    typedef opaque          utf8string<>;

There is an implicit requirement here that file names be able to
contain embedded 0 bytes, which would lead to undesirable behavior
when implementing an NFS4 server for a Unix file system (since most
Unix file system do not support embedded 0 bytes in file names).

One of the goals of UTF8 was to make UTF8 strings work well with the
standard C string functions.  Thus, all ASCII characters and the byte
0 are self-encoding and self-synchronizing.  In particular, the byte 0
does not appear in any runes other than 0.

Thus, I suggest that utf8strings actually be encoded as strings:

   utf8string    typedef string          utf8string<>;

The only difference between this and the proposed opaque is that now
strings cannot contain the byte 0.

I discussed this issue with one of the members of the Plan9 operating
system group at Bell Labs (the group from which UTF8 originated), and
he agreed that utf8string should be represented as a null-terminated
string, not as opaque data.

David

---

From: Mike Eisler <[email-masked]>
Date: Mon, 6 Mar 2000 10:48:44 -0800 (PST)

> strings cannot contain the byte 0.

I disagree on two counts.

First, nothing in RFC 1831 says that the string data type can't have a 0
(nul).  The issue of whether a utf8string incudes a nul is no much different
than the issue as to whether utf8string used for a file name component has an
embedded '/'. Some implementations will accept the nul, some (most) won't,
some will accept the '/', some won't. That most implementations of RFC 1831
are written in C and C uses nul terminated strings is a personal problem on
the part of those implementations. 

Second, the string data type defintion in RFC 1832 explicitly and unambiguously
states that it defines a "string of N ... ASCII bytes". Since we
want to include non-ascii bytes in the utf8string type, defining it
terms of "string" violates the definition of "string"; a correct
implementation of the string decoder would flag an error if it
encountered a non-ASCII octet.

> string, not as opaque data.

On the wire, both utf8string, and string look exactly the same. utf8string
is variable length data of octets, string are variable length data of ascii 
octets. Compare the diagrams of sections 3.10 and 3.11 of RFC 1832.

	-mre