Subject: linktext: must it be UTF-8?

From: "Noveck, Dave" <[email-masked]>
Date: Tue, 23 Apr 2002 09:25:30 -0700

Here's a question I couldn't find the answer to in 
the spec.  Is linktext in the CREATE request (for
a symbolic link) required to be UTF-8 or can it be
any set of bytes?  The text does not explicitly
mention returning INVAL in this case, as it does
in some (probably not all) cases in which UTF-8
is required.  One might take this is as (weakly)
implying that anything goes in the linktext.  On
the other hand, the spec says that UTF-8 is the
answer to the problem introduced as follows:

   The primary issue in which NFS needs to deal 
   with internationalization, or I18N, is with 
   respect to file names and other strings as used 
   within the protocol.

Since linktext would seem to fit squarely within the 
"other strings" category, does this argue that it will 
be interpreted as UTF-8 and must be a UTF-8 string to 
be valid?

A related question is whether a server may restrict
linktext according to the character set that it supports,
an issue which we discussed earlier in the context of 
file names.  For example, may a server whose underlying 
filesystem is UNICODE-based (i.e. 16 bit characters) 
reject linktext containing characters which, when expanded 
according to the rules of UTF-8, do not fit in 16 bits?

The issue of the interpretation of linktext has most of
its implications for something that is formally out of 
scope, but nevetheless of major concern to those implementing
servers, which is how to accommodate symlinks stored with
other versions of NFS, when fetched using NFS-v4.  In
particular, a linktext-is-totally-uninterpreted-entirely-
opaque-to-the-server interpretation is not a good fit
when existing symlinks contain non-ASCII characters that
were stored using V3 in an environment assuming a 
particular locale.  The sensible thing to do in this case
is to return the UTF-8 equivalent of the non-ascii characters
represented by the symlink, rather than the bytes actually stored.

So here is my take on these issues:

     the text of a symbolic link is a string of uninterpreted
     UCS characters, rather than a string of uninterpreted 
     bytes. (However the size is the size in bytes).
 
     Specifying a linktext which is not a valid UTF-8 string is
     an error (NFS4ERR_INVAL).

     The server may impose rerstrictions on the set of valid 
     UCS characters in a link, returning a new error
     NFS4ERR_BADLINKCHAR when an unsupported character is
     encountered.  The server may not impose other restrictions
     (other than size limits) on the linktext.  In particular it
     should not restrict links according to its own notions of
     well-formedness.

Other opinions?

The big issue with trying to define appropriate errors for these 
kinds of situations is that we need to get these in the spec so
that people who are coding this can actually use them as it gets
less and less likely these will be fixed the longer we have to 
wait for a spec that has the errors defined  In particular,  
the discussion about errors for server restrictions on names,
which starts at 
http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/2002/0003.html
still is not reflected in the draft that we are all 
using to code with.  There seemed to be general agreement
that INVAL-for-everything stinks and needs to be fixed but
right now all we have is INVAL.  Can we get the working
draft updated now, and save ourselves grief in the future?

---

From: Peter Åstrand <[email-masked]>
Date: Tue, 23 Apr 2002 21:23:08 +0200 (CEST)

> is required.  

Yes, I think it should be UTF-8. I mentioned this in my mail
http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/2001/0757.html.

>One might take this is as (weakly)

Since linktext is defined as:

linktext4     typedef utf8string      linktext4;

...I think it would be crazy to allow non-valid UTF-8 strings. 

Maybe the spec could be simplied by just adding a note about that all data 
types derived from "utf8string" should obey the ISO10646-UTF-8 definition. 

>      should not restrict links according to its own notions of

Sounds reasonable. 

> that INVAL-for-everything stinks and needs to be fixed but

Yes, Spencer said he had placed an item on the TODO list for adding
another error code for the "invalid name" case
(http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/2002/0016.html), but
still, three months later, I cannot find it on the TODO-list 
http://www.nfsv4.org/rfc3010updates/issues.html. 

I agree with you that this is a problem. Testing at Cthon was a bit
interesting: "Your server is not returning the correct error code, but I
cannot tell you which one you should return instead, because it does not
exist yet" :-)

-- 
Peter Åstrand                Telephone: +46-13-21 46 00
Cendio Systems               E-mail: [email-masked]
Teknikringen 3
583 30 Linköping
Sweden