Subject: RE: id's as strings in the spec

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 14 Jun 2002 21:38:51 -0700

> > > > Now let's consider the complementary problem.  It may also be 
> > > > the case that GETATTR (or READDIR) encounters a numeric id 
> > > > for which there is currently no string defined.  This may be 
> > > > the result of deleted users or simply the fact that v2/v3 and
> > > > local file systems allow you to set owner and group to any
> > > > numeric value.  An error here would be highly undesirable. One 
> > > > thing to do is to map these to nobody.  The problem is that 
> > > > this makes the semantics of a directory to local disk different
> > > > from v2/v3.  One might copy a file, even to local disk, and 
> > > > as a result change the ownership of files with obsolete or 
> > > > other not-currently-set-up users.  
> 
> I'm not convinced that your approach with "12345" usernames works. 
> Consider what happens if the server and the client does not have the same 
> uid/username mapping: You read in a file from tape with uid 4711, which 
> does not exist on the client. The client contructs the username "4711" and 
> sends over the wire. The server stores the file with uid 4711. Then the 
> client does GETATTR. The server, which has a local user with uid 4711, now 
> returns "[email-masked]" instead of "4711". The local client does not 
> know anything about "[email-masked]", and cannot translate this back 
> to uid 4711.

The current approach requires that clients and servers maintain agreement.
If they don't funny things will happen, such as in your example.  However,
that is true whether we have the 4711 case or not.

> So, the idea of letting the server interpreting "4711" username strings 
> does not work if the client and the server does not have the same 
> uid/username mappings. And as far as I understand, v4 does not mandate 
> that they must. 

The spec does not mandate that the mapping be the same.

In fact they may be different but they have to match in one particular.
If the client and the server have to have the same set of username that
have matching uid's.  The uid's don't need to be the same, but I can't
think of reasonable method of assuring that the set of usernames that
have uid's at all is the same on the client and server, that doesn't,
as a side-effect, provide that the uid's for a given username are the
same.  The spec doesn't say this, but if you work out any particular 
case, things just don't work right if the client and server have
different sets of users.

> I thought the whole idea of string based usernames was to 
> eliminate the need for syncronizing uids between servers and clients. 
 
If so, it doesn't accomplish it.  I think the basic idea was to provide
us a path to a future environment in which a numeric UID has ceased to 
be the way in which users are identified.  It's easy to see that such an
approach has definite limitations.  Having string-form owners is an
investment for that future environment, although as I work with the
details I'm starting to have doubts about the ROI.

> > If client cannot take a numeric id and determine what the matching
> > username is, then the whole approach of owners as strings cannot
> > work.  It's based on the idea that through some unspecified mechanism
> > the client and the server can maintain agreement on the valid set 
> > of usernames.
>
> Really? Where can I find this in the spec?

It isn't in the spec.  It is an inference that goes approximately as
follows:

     Mike says he believes that most servers will use numeric uid's.

     Unix clients use numeric uid's in well established API's and
     will continue to do so for a long time to come.

     If a client starts with a numeric uid, and converts that to a 
     string and the server does not have a numeric id corresponding
     to that string, the server cannot return the same string to the
     client and the client will not be able to convert the string it
     gets to the uid it wants.

     Since the thing will break without a mechanism to assure compatible
     mapping and since no method assuring compatibility is specified then
     I conclude that it assumed that some mechanism will be adopted, even
     though the spec does not call one out.

> After thinking about these issues, I (currently :-) believe that this is 
> the way to go:

> 1) The server SHOULD preserve the string in a SETATTR-GETATTR sequence
> (at least as long as the server uid/name mapping is not changed). I 
> believe that this is necessary to avoid all sorts of strange cases. 

You say that the server "SHOULD" preserve the string.  I can't argue with
that.  Of course it should.  It should, except when it can't.  But what does
that mean?  Then you have your "at least" clause, which refers to the uid/name mapping, which is really outside the scope of spec since refers to 
aspects of a UNIX implementation.

You have an implementation in mind, as outlined below.  You are saying that
wierd behavior that results from the mapping changing is OK, but that which
results from a bad setup is not OK.  Why not leave this as a quality-of-
implementation issue.  The server SHOULD return what was stored (or what's
an attribute for) and leave it at that. 

> 2) How should the server handle a SETATTR for a "unknown" user? I think
> there are no "invalid users" from the v4 point of view. The server should
> accept ALL username strings, regardless of wether there is a local uid
> mapping or not. So, I think NFS4ERR_BADOWNER is a bad idea (yes, I've
> changed my mind...). The question is of course how servers that stores
> uids in the filesystem should deal with this. I suggest that they can use,
> for example, the method described by Marius
> (http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive/2002/0209.html):  
> Files owned by "unknown" users can be stored under a special uid (like
> -2). The real username then has to be stored somewhere else, for example
> in an extended attribute (EA) or some other, local database.

> Of course, the implementation burden is a bit higher, but not impossible I 
> think. 

I agree that it is "not impossible" but I would say those words with a
somewhat different intonation.

I think this is an unreasonable burden to place on implementations,
especially at this late date.  I know there are people who are thinking
about plans to QA and ship.  I think we should avoid substantive changes 
with large implications for implementations, unless they are absolutely 
required.  The spec is not closed so we can make changes and this is a
judgement call but people have invested considerable time developing 
under the assumption that existing fs's which do not store strings are 
OK and can be adapted to the details of the V4 protocol.

Being able to store arbitrary usernames when almost all curent clients
will use a fixed set is unreasonable.  The protocol allows a server to
accept all usernames, when that becomes neceassry.  He's not required 
to restrict the set, but he may.

> 3) Is it OK to have several usernames for the same uid in NFSv4? Sure,
> since the v4 spec does not talk about uids. But you must still ensure that
> a SETATTR-GETATTR sequence preserves the username. The easy way to ensure 
> this is not to have several users with the same uid, or several uids for 
> the same user. 

> In practice, UNIX-based servers can detect "wrong setups" by using 
> "reverse lookups", in a way similiar to what sshd does with domains & 
> IPs: 

> On all SETATTR("someuser") requests, the server can assert that
> getname(getuid("someuser")) == "someuser". If not, then you have two users
> with the same uid. The server logs and returns an error.

> On all GETATTR(someuid) requests, the server can assert that 
> getuid(getname(someuid)) == someuid. If not, you have several uids for the 
> same user. The server logs and returns an error.

You can do this but it doesn't guarntee that you will meet the 
SETATTR-GETATTR invariant.  It only does that if the mapping never 
changes.  If it can change, then you can change the username of an
existing uid in such a way was to break your requirement.  The result
is weird behavior.  

If GETATTR always returning what was SETATTR was a protocol requirement
(and it does seem reasonable in the abstract), then you either have to
store the string for each file or if you represent it by a number in
the inode, then the string corresponding to that number must be in a
separate db maintained by the filesystem, regardless of any changes
to mappings external to the filesystem.  This would be a lot of work.
A server is free is take on that work but it isn't required.  I would 
the same about your setup checks above:  You want to do it, fine.

> 4) What should the server do for a GETATTR on a file with a unknown uid?  
> Log and return an (server) error. This will only happen if the
> uid/username mapping is removed on the server.

> 5) What about users from different domains? When the server recieves an
> userstring (like "[email-masked]"), it should first check if the domain
> part matches the servers local domain. If it does, the domain part is
> stripped off and the username ("joe") is used. If this username is
> unknown, or the domain-part didn't match, the user is treated as an
> "unknown" user. The username is stored in EAs or separate metadata
> database.

This sounds like a reasonable implementation for servers that want to
implement this but it should nt be required.  I think it is totally
reasonable for a server to say it does not support any ownership of
files by owners from another domain, returning NFS4ERR_BADOWNER.

> 6) Should a special "UID string form" username (like "4711") be included
> in the v4 spec? Yes, but this form should not be interpreted by servers;
> only by clients. Servers should treat these strange usernames just like
> any other unknown username (that is, use EAs or metadata database). (Well,
> of course, it's also possible to allocate uids for some of these strange
> names. For example, a server can allocate uids 65535-131070 to usernames
> "[email-masked]"-"[email-masked]", and use uids < 65535 for normal
> usernames.)

> Clients can translate from this special form to uids. 

> I think this special form should also include the domain name, like
> "[email-masked]". This will prevent collisions for clients in different
> domains. There is still a chance of collisions if there are several
> clients in the same domain that does not have syncronized uid/users
> mapping, but I think we can live with that.

> I have probably missed something, but to me, it looks like this scheme 
> would solve most problems:

> * Servers will always return the same username in GETATTR as was used in 
> SETATTR

Except when they don't.  Your don't is smaller than the one I would implement
but it isn't never.  I think we should leave it there as far as the spec goes,
and let implementation experience decide.

> * Clients can store files with "unknown" uids, for example, when reading 
> in files via tar from tape. 

> The answers to Daves questions with this scheme would be:

> >     Mapping that map lots of strings to the same id or restricting
> >     the range of valid string by returning an error.
>
> A third approach: All strings are valid. 

> >     What to do about id's in the fs that don't map to any string.  
> >     Numeric id escape or something else?  (what?)
>
> When the server finds a file in it's local fs that don't map to any string
> (on the server), return an error. When clients need to store files with 
> uids that don't map to any string (on the client), the client can use the 
> escape mechanism. 

> >     If special escapes, what are rules on when thay may be used?

> I prefer "[email-masked]", but "4711" is also fine with me.  
> "[email-masked]" will prevent using usernames beginning with #, but I
> don't think that will upset many users.

---

From: Peter Åstrand <[email-masked]>
Date: Sun, 16 Jun 2002 14:52:38 +0200 (CEST)

>      client and the client will not be able to convert the string it

The server can return the same string, if it uses EAs or metadata db.

> You say that the server "SHOULD" preserve the string.  I can't argue
> with that.  Of course it should.  It should, except when it can't.  But
> what does that mean?  Then you have your "at least" clause, which refers
> to the uid/name mapping, which is really outside the scope of spec since
> refers to aspects of a UNIX implementation.

You're right. This text is not ment to be included in the spec as is, I'm 
just trying to communicate my view of model that looks good to me. 

What I mean is that the string should always be preserved in a
SETATTR-GETATTR sequence if there is no administrative action is taken in
between. If an administrator is changing the uid/name mapping between 
SETATTR and GETATTR there is not much we can do about it. After all, he 
chould do a chown() as well. 

> You have an implementation in mind, as outlined below.  You are saying
> that wierd behavior that results from the mapping changing is OK, but
> that which results from a bad setup is not OK.

Yes, pretty much so. If the server can detect bad setups, which can lead
to wierd behavior, it should not be silent about it. Detecting bad setups
in the UNIX/uid case is possible. 

>Why not leave this as a quality-of-

Yes, the spec can probably not be more specific than this. 

> > there are no "invalid users" from the v4 point of view. The server should
...
> will use a fixed set is unreasonable.  The protocol allows a server to

I think you are right. I haven't considered release dates, protocol 
changes implications and such; I've just looked at how good model it is 
possible to achieve. 

If we reach consensus on that the principle of the model I've described is 
the way to go, at least we know which goal to work towards. 

Maybe it's possible to relax some of my requirements and end up with 
something that is implementable right now without too much effort. Such a 

>>believe that this is necessary to avoid all sorts of strange cases. 

This general idea could still hold. 

>>there are no "invalid users" from the v4 point of view. The server
>>should accept ALL username strings, regardless of wether there is a
>>local uid mapping or not. So, I think NFS4ERR_BADOWNER is a bad idea
>>(yes, I've changed my mind...). The question is of course how servers
>>that stores uids in the filesystem should deal with this. I suggest that
>>they can use, for example, the method described by Marius

We can recommend servers to accept all users (and recommend the use of EAs 
or metadata db for systems that store uids), but still allow 
NFS4ERR_BADOWNER for servers. 

>>since the v4 spec does not talk about uids. But you must still ensure
>>that a SETATTR-GETATTR sequence preserves the username. The easy way to
>>ensure this is not to have several users with the same uid, or several
>>uids for the same user.
>>
>>getname(getuid("someuser")) == "someuser". If not, then you have two
>>users with the same uid. The server logs and returns an error.
>>
>>getuid(getname(someuid)) == someuid. If not, you have several uids for
>>the same user. The server logs and returns an error.

We could recommend servers to try to detect bad setups which leads to 
"weird behaviour". 

>>Log and return an (server) error.

I think this could still hold. 

> files by owners from another domain, returning NFS4ERR_BADOWNER.

Yes, in a "relaxed model", I agree. 

> > mapping, but I think we can live with that.

I think this should still hold, even in a "relaxed model". The important
part is that servers should not interpret/convert the uid in username
strings of typ "#uid" or "uid" as a real uid. I think that is simple
wrong, and will make it necessary to syncronize users & uids between
servers and clients, forever.

So, with servers that does not support arbitrary user names (does note
have EAs or metadata db), will it be impossible to store these "#uid"
escaped owner strings? Yes, sometimes. 

One method of solving this problem is to simply get rid of unknown uids:
The naming service can define mappings for all uids. So, if you ask for
the "unused uid" 4711, the naming service will give you the username
strings "#4711". And a lookup of the username "#4711" will give you uid
4711. If both the server and the client uses this naming service, there
will be no problem with storing "unknown" uids (because they aren't any
unknown uids any longer)

Anyway, if none of these methods are used, they server may reject owner
strings of typ "#4711" with NFS4ERR_BADOWNER, which in turn means that you
cannot use "unknown uids" at clients (so if you do untar + tar, you may
end up changing uids).

I think this is a reasonable sacrifice. When v4 servers starts to 
implement support for arbitrary owner strings (which should be recommeded 
by the spec, IMHO) (with the help from EAs or metadata db), this 
problem will go away. 

The answers to Daves original questions with this scheme "relaxed model" 

>     the range of valid string by returning an error.

Restricting to the range of valid strings by returning an error. 

>     If special escapes, what are rules on when thay may be used?

My latest idea is "4711#example.com" (@ replaced with #). Just "4711",
without the domain name, is also fine with me, but increases the risk of
collisions.

It feels like Daves and my model are beginning to get closer. The 
difference between our models is:

>* Anyway, I would propose that servers may return @# id's if there is
>no other way to encode the user identification present in the local
>filesystem but MUST not otherwise.

I don't agree here, I want an error. 

>* Further they may accept @# id's (for a SETATTR) if there is no other
>string form id which maps to the same local filesystem owner.

No, servers should never translate a uid number in these escaped 4711@#
strings to a real uid, blindly. If the naming service (at the server)  
includes this strange user 4711@#, or the server accepts arbitrary owner
strings (because it uses EAs or other mechanism) the owner will be
accepted, otherwise rejected with NFS4ERR_BADOWNER.

By using the naming service, administrators can choose if the want to
accept escaped uid owners like 4711@#, or not. I think this is better than
specifying that UNIX-based v4 servers should always translate 4711@# to
uid 4711. It should be legal for UNIX-based servers to store files owned 
by the owner "4711@#" under uid 4711+65535, for example. 

-- 
/Peter Åstrand <[email-masked]>