Subject: Re: Two comments on the filexfer draft

From: Jeffrey Altman <[email-masked]>
Date: Mon, 15 Jan 2001 15:45:32 EST

> "Joseph Galbraith" <[email-masked]> writes:
> 
> > 1. We need to be careful of international language issues.
> >    I would recommend all file / path names should be specified
> >    as UTF-8.
> 
> This could get messy, but perhaps it's necessary.
> If we do it, the document must also specify that either
> 
> A. the server is responsible for respecting unicode canonical
>    equivalence, or
> 
> B. the client should normalize all filenames (according to some
>    reasonable rules for normalization).
> 
> When we were discussion usernames and passwords previously, I think
> the wg tended towards the former option. Again, it would be
> interesting to know what other file sharing protocols do.

For the most part most file transfer protocols ignore the issue of
path names since directory naming conventions and character set usage
is not portable among operating systems.

FTP uses the Unix conventions and recently adopted the notion that all
names should use UTF-8.  The IETF's requirement for the use of Unicode
require the use of canonical representation in order to avoid security
issues.  This is true of all external representations. 

As for Usernames and Passwords, FTP punts.  I believe the assumption
is that all filenames and passwords are 7-bit US-ASCII or that the
client and server are using common character-sets.

 Jeffrey Altman * Sr.Software Designer      C-Kermit 7.1 Alpha available
 The Kermit Project @ Columbia University   includes Secure Telnet and FTP
 http://www.kermit-project.org/             using Kerberos, SRP, and 
 [email-masked]          OpenSSL.  SSH soon to follow.

---

From: [email-masked] (Niels M�ller)
Date: 17 Jan 2001 19:14:06 +0100

Tero Kivinen <[email-masked]> writes:

> When the recipient receives that it should convert the filename to
> format that is suitable for the file system. If the filesystem names
> are already in unicode (or the file system does not have defined
> character set for the filenames) it can directly store the filename in
> the format the other end sent it.

I'm not sure. If the OS/filesystem claims to be unicode compliant, it
must treat names that are unicode canonical equivalents as the same
name, but I'd prefer not doing any normalization inside a kernel
filesystem driver. I don't know what existing unicode filesystems do.
Or do they use some kind of retain-and-ignore-differences, just like
some systems ignores case in filenames?

> Also if we do normalization in both ends then we might end up having
> two different files that have identical names after the normalization
> process, thus one of them is inaccesable if we do normalization. 

You should never have two separate files whose names are unicode
canonical equivalent. Either applications or the OS should stop you
from doing that.

Anyway, doing normalization at both ends seems unnecessary; it should
be good enough to specify which end is responsible.

I'd like the spec (preferably the architecture spec) to say that

1. Implementations MUST NEVER use overlong utf-8 representations. Then
   we only have to care about names that are equivalent in unicode.
   Does anybody disagree with that?

2. Either
  a) say that the server SHOULD respect unicode canonical equivalents,
     or 
  b) say that the client is responsible for normalization.

For 2b), perhaps we can get some guidance from

	Title		: Character Normalization in ITEF Protocols
	Author(s)	: M. Duerst, M. Davis
	Filename	: draft-duerst-i18n-norm-04.txt
	Pages		: 12
	Date		: 13-Sep-00

I believe those normalization rules should be friendly to applications
that only want to translate common 8-bit character sets (like latin-x)
to and from unicode.

If we can't get utf-8 done right, it would be better not to do it at
all. Problems caused by incompatible character sets should be a lot
easier to understand and work around (programmers and users know how
to recognize and deal with those problems) than problems caused by
incompatible normalization and equivalence rules.

/Niels