Subject: RE: [nfsv4] symlinks: opaque or UTF-8

From: "Noveck, Dave" <[email-masked]>
Date: Sat, 1 Dec 2007 15:15:26 -0500

> I think it's time to declare that sub-thread dead.  You and 
> I disagree about the likelyhood of having two client and server 
> implementations that get I18N right.  No need to pursue that.

I can't speak for Mike but I don't think that's really the disagreement.
I can very well believe that there will be the requisite number of
implementations to satisfy the IETF someday.  The issue is that if
almost everybody is not using them, either because they are Linux and
don't do that sort of thing, or because they are servers that have to
interoperate with Linux, or because they are clients that have to
interoperate with servers that have to interoperate with Linux, then who
cares.  Utf8-false is everyday reality and utf8-true has a somewhat
insubstantial existence. 

In that environment, even though I think I18N is the right thing to do,
I can't justify actually working on it, from a business point of view.
But if it will make you and Spencer happy and the IESG happy, then it is
OK with me if it is in the protocol.  So to continue the analogy, the
baby will not die, but it may never get off life support either. 

> The only other protocol-level issue is whether the protocol 
> should use an XDR type that might denote, to an XDR tool chain, 
> that values of that type can only contain valid UTF-8 sequences
> -- but I think this is a non-issue today, and a warning in the 
> text of the RFC would suffice to avoid it being an issue in the 
> future.

This is not just an issue for an XDR tool chain.  RFC3530 uses these
types to indicate various stringprep requirements and states that when
such types are used the values MUST be checked for UTF-8-ness and
NFS4ERR_INVAL returned if they are not.  There are plenty of uses of the
types utf8str_cs, utf8str_cis, uf8str_mixed that are not component4 or
linktext4 and I'm assuming that these stringprep requirements will
continue to apply them.  

We then have the issue of what to do with component4 and lintext4 and
whether we can or should simply typedef them as opaque<> or some type
that specifies that the stringprep requirements apply conditionally,
based on fs type. We also have to address the issue of squaring this
sort of change with the minor versioning rules.  I'm not saying that
this can't be done, but there are issues to be resolved in making it
happen.

Before we get to that, let me understand what you are proposing
substantively.  Is it,

    A mandatory per-fs option: utf8.

    If utf8 is true, then path components are required to be utf8, and
an error 
    MUST be returned if they are not.  The error could be NFS4ERR_INAL
or we
    could decide that it would be clearer if it were an explicit
NFS4ERR_NOT_UTF8.
    The sever may also reject such a string with NFS4ERR_BADNAME or
NFS4ERR_BADCHAR.

    If case_insensitive is set for an fs, utf8 MUST be true.

    If utf8 is true, then link contents are required to be utf8.  Same
error
    requirements as for path components except NFS4ERR_BADNAME is not
allowed.

    If utf8 is false, both path components and link contents are treated
as 
    opaque<> with no restrictions as to UTF-8-ness.  However, the server
may
    reject either with NFS4ERR_BADCHAR and may reject path components
with
    NFS4ERR_BADNAME.

Is this compatible with what you were thinking?

-----Original Messa
ge-----
From: Nicolas Williams [mailto:[email-masked]] 
Sent: Friday, November 30, 2007 6:58 PM
To: Mike Eisler
Cc: [email-masked]
Subject: Re: [nfsv4] symlinks: opaque or UTF-8

On Fri, Nov 30, 2007 at 02:29:46PM -0800, Mike Eisler wrote:
> --- Nicolas Williams <[email-masked]> wrote:
> > On Fri, Nov 30, 2007 at 04:11:59PM -0600, Nicolas Williams wrote:
> > > As a server vendor you can comply if we simply add a per-fsid
> > option to
> > > just-use-8.
> > 
> > I.e., no need to throw out the baby with the bathwater.
> 
> Right instead just let the baby die of heat stroke in the sun.

I think it's time to declare that sub-thread dead.  You and I disagree
about the likelyhood of having two client and server implementations
that get I18N right.  No need to pursue that.

> If an attribute, per fsid or per system (I'd recommend implementing it

> per system regardless) would allow us to reach consensus on changing 
> i18n, I'd live with it. Indeed, it would give Dave others time to undo

> the beautiful UTF-8 translation code they wrote.

I agree that having a per-fsid UTF-8/non-UTF-8 knob is desirable and
would settle the issue (NOT per-system -- we need to support migration
on a per-dataset basis).  The only other protocol-level issue is whether
the protocol should use an XDR type that might denote, to an XDR tool
chain, that values of that type can only contain valid UTF-8 sequences
-- but I think this is a non-issue today, and a warning in the text of
the RFC would suffice to avoid it being an issue in the future.

Nico
-- 

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Nicolas Williams <[email-masked]>
Date: Sat, 1 Dec 2007 17:23:44 -0600

> implementations to satisfy the IETF someday.  The issue is that if

There is a requirement that interop testing show that for every feature
of the protocol there are two interoperable implementations in order to
take a document from Proposed Standard all the way to Standard (no need
to go into the full details of the process right here).  There is no
need to have two implementations that implement *all* features of the
protocol interoperably.

> almost everybody is not using them, either because they are Linux and

"almost everybody" isn't enough.  See above.

> [...]

No matter.

> baby will not die, but it may never get off life support either. 

It's not about what Spencer and/or I want.  The IESG forced this WG to
account for I18N.  The IESG will not relax this requirement, nor is it
necessarily the case that if it did the IETF would follow.  That said, I
want I18N to remain part of the spec, and I'm pretty certain that
Spencer does too (Spencer can correct me if this is incorrect).  (It
helps that we're almost there, implementation-wise, but I have been
pushing for FS I18N for a long time and I wouldn't give up yet even if
we still had a long way to go.)

> > [...]
> NFS4ERR_INVAL returned if they are not.  [...]

If you had an XDR toolchain that supported such types directly then it
would be an XDR toolchain issue.  But as long as noone does and we warn
that these types are not native XDR types but an NFSv4.x convention,
then we're OK with allowing non-UTF-8 content in them when just-use-8 is
negotiated for some filesystem.

> that specifies that the stringprep requirements apply conditionally,

It would suffice that the doc say that the type names denote
application-enforced constraints and that the XDR toolchain must not
enforce these constraints -- that these types can carry non-UTF-8 under
certain circumstances.

> this can't be done, but there are issues to be resolved in making it

Absolutely.

>     A mandatory per-fs option: utf8.

More or less:

 - two per-fsid properties:

    - fs-contains-non-UTF-8 (boolean)  (anal retentive clients can then
					refuse to mount filesystems w/
					fs-contains-non-UTF-8 == TRUE)

    - fs-allows-only-UTF-8 (boolean)   (just-use-8 clients can then warn
					users to stick to US-ASCII-only
					filesystem object names)

>     If case_insensitive is set for an fs, utf8 MUST be true.

We could also add a per-fs property naming a codeset to be used for fs
object names therein, but I'm inclined to reject such a knob.  So, yes.

>     error requirements as for path components except NFS4ERR_BADNAME
>     is not allowed.

Yes.

>     If utf8 is false, both path components and link contents are
>     treated as opaque<> with no restrictions as to UTF-8-ness.
>     However, the server may reject either with NFS4ERR_BADCHAR and may
>     reject path components with NFS4ERR_BADNAME.

Yes.

> Is this compatible with what you were thinking?

More or less, yes.

Nico
--

---

From: "Noveck, Dave" <[email-masked]>
Date: Sat, 1 Dec 2007 19:57:06 -0500

> If you had an XDR toolchain that supported such types directly then 
> it would be an XDR toolchain issue.  

I guess but you would be insane to do that.  The errors that result when

these constraints are not met are NFSv4 errors and not XDR errors and if
you considered them XDR errors you would be violating the XDR spec.

> But as long as noone does and we warn that these types are not native 
> XDR types but an NFSv4.x convention, 

They are typdef'd as opaque<>.  If that doesn't make clear to you that 
they are not native XDR types, plus the fact that they are not specified

as native XDR types (I assume that's specified in the XDR spec), I don't

see what would.
 
> then we're OK with allowing non-UTF-8 content in them when just-use-8 
> is negotiated for some filesystem. 

I don't see any negotiation here.  The proposal as I understand it is
that
the server defines the handling.

-----Original Message-----
From: Nicolas Williams [mailto:[email-masked]] 
Sent: Saturday, December 01, 2007 6:24 PM
To: Noveck, Dave
Cc: Mike Eisler; [email-masked]
Subject: Re: [nfsv4] symlinks: opaque or UTF-8

On Sat, Dec 01, 2007 at 03:15:26PM -0500, Noveck, Dave wrote:
> > I think it's time to declare that sub-thread dead.  You and I 
> > disagree about the likelyhood of having two client and server 

> [...]
> 
> In that environment, even though I think I18N is the right thing to 
> do, I can't justify actually working on it, from a business point of
view.

No matter.

> But if it will make you and Spencer happy and the IESG happy, then it 
> is OK with me if it is in the protocol.  So to continue the analogy, 
> the baby will not die, but it may never get off life support either.

> > The only other protocol-level issue is whether the protocol should 
> > use an XDR type that might denote, to an XDR tool chain, [...]
> 

Absolutely.

More or less:

Yes.

Yes.

More or less, yes.

Nico
--

---

From: "Noveck, Dave" <[email-masked]>
Date: Sun, 2 Dec 2007 07:49:30 -0500

> There is no need to have two implementations that implement *all* 
> features of the protocol interoperably. 

a. I never said otherwise.

b. It has no relevance to the issue.  I've conceded that there may 
be enough interoperable implementations to satisfy the IETF.

BTW, it is amusing to consider the siutation if we don't have the
two implementations for the utf8-true case.  If that happens, then
we'll have a case, essentially, of the IETF forcing us to take utf8
out after forcing us to leave utf8 in.  I'm not looking for that 
result but it would rate a 9.5 on the irony meter.

Isn't enough for what?

The actual statement, which you elide, was that if almost everybody
is not using utf8 filesystems, i.e. if almost nobody is using them, 
then

> who cares.  Utf8-false is everyday reality and utf8-true has a 
> somewhat insubstantial existence.

and I stand by that.  I don't care very much about options that
almost nobody uses.  If you do, fine. 

-----Original Message-----
From: Nicolas Williams [mailto:[email-masked]] 
Sent: Saturday, December 01, 2007 6:24 PM
To: Noveck, Dave
Cc: Mike Eisler; [email-masked]
Subject: Re: [nfsv4] symlinks: opaque or UTF-8

view.

No matter.

Absolutely.

More or less:

Yes.

Yes.

More or less, yes.

Nico
--

---

From: Nicolas Williams <[email-masked]>
Date: Sun, 2 Dec 2007 10:23:49 -0600

> a. I never said otherwise.

I was preempting a possible line of argument :)

> result but it would rate a 9.5 on the irony meter.

There's no irony.  I think the IETF would simply no allow NFSv4.x to go
to Standard if doing so meant dropping I18N, particularly when waiting a
bit longer might suffice.

> is not using utf8 filesystems, i.e. if almost nobody is using them, 

Enough to justify yanking I18N out of the protocol.

> almost nobody uses.  If you do, fine. 

That's enough to put this long thread to rest :)

Nico
--