Subject: [nfsv4] Individual i-d on v4.0 migration issues

From: <[email-masked]>
Date: Tue, 30 Aug 2011 13:35:39 -0400

Just wanted everybody to be aware of the attached.

It is in the informational category, although its premise, that things need to be fixed in subsequent standards-track documents does have normative implications.

I think it is now time to discuss the document and its implications within the working group.

---

From: Trond Myklebust <[email-masked]>
Date: Tue, 30 Aug 2011 14:41:54 -0400

> I think it is now time to discuss the document and its implications within the working group.

In Section 3.2.1 you debunk the text that is currently in RFC3530/bis
w.r.t. whether or not a SETCLIENTID will cause a reboot if you reuse the
same long form clientid to the same server on 2 different IP addresses.
While I agree with your argument, I don't think that it is sufficient,
and that your 'uniform model' cannot work.

The main problem with a client that is unaware of the fact that it is
talking to the same server over 2 different IP addresses is the fact
that if it mounts the same filesystem over both addresses, it may end up
performing conflicting stateful operations.

IOW: If my client OPENs a file on one IP address using a particular open
owner string, then OPENs the same file on the other IP address using the
same open owner string, then that client is going to find tracking the
seqids to be _very_ difficult, since it thinks it is working with 2
different seqid streams because it thinks it is dealing with 2 different
servers.

Even if the client uses different open owner strings in the above
example, then it can run into trouble: for instance, if the server
decides to give out a write delegation. A write delegation is supposed
to guarantee that the client can cache aggressively, but only the server
knows that the client is accessing the same file through two different
IP addresses. As far as the client is concerned, it will see changes
that appear to be in violation of the write delegation.
Also, what happens in the case of a delegation recall? How does the
client know that the recall applies to _both_ filesystems (assuming that
the server handed out the same delegation to both)?

Cheers
  Trond
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
[email-masked]
www.netapp.com

---

From: <[email-masked]>
Date: Wed, 31 Aug 2011 08:39:07 -0400

> While I agree with your argument, I don't think that it is sufficient,

What is it insufficient for?  It is intended to show that the only reason 
Given in RFC3530 and RFC3530bis-{1..13} is not  valid one. Since you agree 
with the argument, I take it you agree that it is sufficient for that purpose.

Do you think it is insufficient to cause us to go from "should" to "MAY"?
My approach is different.  In the first place I don't think "should" makes
sense for things like this, and if you don't like "MAY" you need to suggest
some argument for "SHOULD" (use the non-uniform model) that is sufficient to 
make that case.  Now you have offered some sort of argument which if valid
might demonstrate that (and I'll have some criticisms of it below), but I
think that the burden is on you to show that that uniform model cannot work and
I don't think you understand how difficult that is to show.

When you say it "cannot work" you are saying that no algorithm can be devised
to deal with this situation and that's why you need a real proof to show that
since there are an infinite (although denumerable) set of possible algorithms.
You have to show a situation that no algorithm can deal with and that is hard to do. 

> talking to the same server over 2 different IP addresses

He is unaware that he is talking to the same server but he has to be 
aware that he might be talking to he same server.

The current text tries to make it be clear that sometime he knows the servers
Are different and sometimes he can find out they're the same, but some of 
the time he may not know for a while and has to be able to deal with that 
uncertain state.

> it may end up

"conflicting" in what sense?  I don't think you've cited anything
Below where the operations cancel each other out.

> then that client is going to find tracking the

We can get into a not very helpful dispute as you say "_very_ difficult"
and I say "somewhat difficult" or "reasonably difficult"

But as far as the overall argument is concerned, the important point is that

"_very_ difficult" != "impossible" and "impossible" is tough to show.

The thing that limits the difficulty is that when things happen that are incompatible 
with your current assumption (as far as server identity) about a pair of IP addresses,
you don't have any more uncertainty to deal with.  The algorithms are different if
you start with the assumption of server non-identity and allow proof of identity or 
or start by assuming identity when clientid4's match and allow proof of non-identity,
but the problem seems workable to me.  But I didn't say that in he spec. I said you may
have problems with this and should make your own implementation
choice. 

> but only the server
> IP addresses.

The client doesn't know that but when he gets the same clientid4 from two
different IP addresses he has got to know that it is likely that they are
the same.  Certainly it is possible even if it is not a sure thing. 

> it will see changes

What it will see is a change that is in conflict with 

 "A has a rd delegation" & "A and B are different servers"

Which propositions we will call X and Y respectively.

If something happens that is incompatible with X & Y then if X is true,
Then Y is false and so now the client knows hat A and B are the same.

Of course the delegation D might have been revoked so you use it on a
Read to A.  If that is OK then you know Y is false, which you can
Verify by doing the read using D over address B.

You can't do this with a vanilla client implementation but some people may
choose Cookie Dough ice cream, and the spec allows that.

Another way to do this is to reduce the period of uncertainty by resolving server 
identity questions as soon as possible.  Once you get the first stateid from a
server, use it on all IP addresses sharing the same clientid4.  Those that recognize it
Are connected to the same server.

I think it is becoming clear why v4.1 was defined to make this easier, but it
still isn't impossible and if you find it onerous, you are free to choose the
non-uniform model.

Summing up where we are:

   do you still object to MAY here?

   If so, I think we need a solid reason to justify "SHOULD" (for the non-uniform model).
   Anyone want to take a shot at this?

   Or I it just that we differ on the tone of the discussion of the uniform model?
   
   If so, tell me what you'd like to see and I think we can work toward something
   reasonable.
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Trond Myklebust
Sent: Tuesday, August 30, 2011 2:42 PM
To: Noveck, David
Cc: [email-masked]
Subject: Re: [nfsv4] Individual i-d on v4.0 migration issues

Cheers
  Trond

NetApp

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: "Myklebust, Trond" <[email-masked]>
Date: Wed, 31 Aug 2011 09:48:18 -0700

> From: [email-masked] [mailto:[email-masked]]
> Sent: Wednesday, August 31, 2011 8:39 AM
> To: Myklebust, Trond
> Cc: [email-masked]
> Subject: RE: [nfsv4] Individual i-d on v4.0 migration issues
> 
> might demonstrate that (and I'll have some criticisms of it below),
but
> I
> But as far as the overall argument is concerned, the important point
is
> that
> you start with the assumption of server non-identity and allow proof
of
> identity or
> or start by assuming identity when clientid4's match and allow proof
of
> non-identity,
> identity questions as soon as possible.  Once you get the first
stateid
> from a
>    If so, tell me what you'd like to see and I think we can work
toward
> something

I think that in practice the client does need to know a priori whether
or not it is talking to the same server through 2 different IP addresses
in order to make the "uniform model" a viable solution.

Once I've decided to mount what I think are 2 different filesystems,
then attempting to merge them at a later stage is impractical due to
general VFS design limitations (at least in Linux that is the case).
Once I get round to actually opening files, I will already have 2
separate trees of cached files/directories that are attached to 2
different super blocks, and so I'd have to do all sorts of distasteful
surgery in order to transparently merge those into a single tree, and
then migrate locks, migrate file descriptors, etc.
If I were rather to assume that the filesystems are the same, and then
try to fork them at a later stage when I find out that they are in fact
different, then I worry that could under some (admittedly rare)
circumstances lead to data corruption: e.g. if 2 filehandles that look
the same turn out to belong to 2 different servers. Again, the forking
process is not trivial to implement in practice.

IOW: I would find it hard to justify implementing all this extra
machinery in order to get a migration solution for NFSv4.0, particularly
given the fact that NFSv4.1 already offers me a way to avoid the entire
problem.

How about something like the following suggestion?

1) Servers that support trunking SHOULD allow a client to determine all
ports on which the server is listening using a COMPOUND of the form
PUTROOTFH; GETATTR fs_locations.

2) A client that want to use the "uniform model" MUST query fs_locations
on the server root filehandle prior to calling
SETCLIENTID/SETCLIENTID_CONFIRM. If the server does not support
fs_locations on the rootfh, then the client SHOULD use a unique long
form client id and MAY assume that the server does not support
migration(?), OR that a migration event must follow the non-uniform
model(?). Otherwise, the client may use RENEW requests on all the IP
addresses in order to discover on which ones a given lease applies.

Cheers
  Trond

> > It is in the informational category, although its premise, that
> things need to be fixed in subsequent standards-track documents does
> have normative implications.
> >