Subject: Re: Invalid UTF-8 strings

From: Peter Åstrand <[email-masked]>
Date: Tue, 1 Jan 2002 14:05:40 +0100 (CET)

> >In my opinion, the protocol spec. should be changed to use the Unicode
> >definition of UTF-8. This would limit NFS4 to codepoints between U+000000
> >and U+10FFFF (with surrogate pairs, ~20 bits), but I think thats enough.
> 
> That proposal would preclude certain languages from being fully supported
> that are currently fully supported, so is a non-starter, though I am sure 

Please give an example of such language. 

> the proposal works fine for Swedish.  The IETF practice is always to specify
> ISO-10646 and ought not be changed peculiarly just for NFS.  

I see. If IETF practice is to always use ISO-10646, i agree that NFS4 
should not make exceptions. 

However, NFS4 should have a well-defined behaviour for UTF-8 error 
conditions, like what a Unicode-system should do with a UTF-8 value 
greater than U+10FFFF. One possibility is of course to return 
NFS4ERR_INVAL. 

-- 
Peter Åstrand                Telephone: +46-13-21 46 00
Cendio Systems               E-mail: [email-masked]
Teknikringen 3
583 30 Linköping
Sweden

---

From: Peter Åstrand <[email-masked]>
Date: Tue, 1 Jan 2002 14:32:30 +0100 (CET)

> Can a NFS4 server impose local filesystem rules against client
> requests?  If so, NFS4 over a UTF-16 filesystem simply never allows
> codepoints beyond U+10FFFF in filenames, just as over a Unix
> filesystem it never allows "/" or "\0" in filenames.  RFC 3010 stated

The following characters are invalid in Windows XP file names:

/ \ : * ? " < > |

But Unix systems typically have many files with, for example ":". As I 
understand, we have two choices here:

1. Allowing "all" characters i NFS4 and adding an error indicating that 
the server didn't accept the file name. I think Windows has an error 
called ERROR_INVALID_NAME. Maybe something similiar would be useful in 
NFS4. 

or:

2. Explicitly defining which characters/names that are invalid i NFS4 (if 
any). Servers will have to use some kind of "escape" mechanism to store 
strange file names in their filesystems. 

I'm surprised this problem has not been discussed earlier. 

> > Should a Unix-kernel-client pass the octets given in open()
> > unmodified over the wire, to the server?
> 
> I suppose the only other option is a libc wrapper that runs the
> filename through mbstowcs() in the current locale and then wcstombs()
> in a UTF-8 locale, just as Win32's CreateFileW() will have to convert
> from UTF-16.  Do ANSI or POSIX have anything to say about I18N of
> open() or fopen(), or is the filename arg just assumed to be in some
> unknown single-byte encoding?

Good question. 

BTW1: Linux (at least 2.2.19) returns ENOENT when trying to do
open(name-with-invalid-windows-character) in a mounted SMB file system.

BTW2: Where can I find sample code for validating ISO10646-UTF-8?

Happy new year!