Subject: RE: Comments on draft-ietf-nfsv4-repl-mig-proto-00.txt

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 19 Dec 2002 12:04:29 -0800

> > I realize that some of the things I'm observing are a
> > result of this being a strawman, but I can also see us
> > getting bogged down in specifying what is effectively a
> > layer in the protocol stack, when we already have one, ONC
> > RPC, to use today. Time is of the essence.
>
> So let's open this up for discussion.  Suppose we make at
> least a first attempt (in the Connectathon 2003 timeframe)
> with an RPC-based protocol, based on this strawman.  Does
> that make sense for others who might want to try this at
> C-thon 2003?  I can certainly be convinced, because I have
> less time for prototyping than I'd initially hoped.  What
> about some of the other design team folks?

Let's put to one side the specific schedule.  I'm not in a
position to be able to commit to anything in that time frame.
I'm going to be spending all my available time getting a
good v4 out, adding features, and testing against the clients
as they are developed, and fixing bugs.

But regardless of the specific time-frame, I don't see the
point of doing throw-away work.  It sounds to me like that
is what you are proposing.  If we want the protocol not
to be RPC-based, then that's what we should work on.  If
we do want RPC, and I think that is the better choice, then 
let's do that.  If we find that RPC-based doesn't work 
well for some reason, I wouldn't stand in the way of 
change, but let's not do RPC supposing that the final 
protocol will be something else.  We're all too busy for 
that. 

> > In section 2.2, the RMmessage_id type ... does the first
> > message in a session start at zero?
>
> Likely; shoulda been specified.

> > In section 2.5 (and section 7), why is the nfsace4 type
> > imported from nfsv4? If the other attribute types aren't
> > importanted, there should be no need for the ace type.
>
> I brought in fattr4's as well; I wanted to use them verbatim
> when we're working with NFSv4 metadata.

Interesting change control issue.  V4 is going to have minor 
versions which may add attributes and so taking the v4.0
verbatim is not what we want.  We should probably negotiate
the minor version level that the attributes should correspond
to.

> > In section 4.4, regarding the RM_UTF8NAMES capability flag,
> > why doesn't the nfsv4 migration protocol assume UTF8 file names?
> > So why does this bit need to exist?
>
> What I was thinking about here was to try to encode if
> the server supports arbitrarily complex UTF8 names, or
> if they will be mapped to something like simple ASCII.
> The hope was that the source server could know this
> early on and inform the admin that metadata might be
> impacted on this destination server.

But that's soooooo inadequate.  Between arbitrary UTF8-
encoded UCS and simple ASCII there are lots of interesting
places to stop: UTF-8 encoded Unicode for example.  (i.e.
No characters characters above 64K so no Elvish, and Elvish
is kind of big right now :-)  This is part of the server 
semantics stuff that I was mentioning.  That may take
a while, but at least I'll try to come up reasonably soon
with something for the character set issue.

> > Regarding the RM_FHPRESERVE capability, unless the source
> > and target know the fh format, this capability seems useless.
> > Since NFSv4 defines file handles as opaque, the capability is
> > indeed useless, unless source and target know they have compatible
> > file handles. My suggestion: 
> > 
> > 	Include in the OPEN_SESSION_NEW arguments a field
> > 	of type utf8str_cis (from the nfsv4 spec) which is
> > 	a dns domain name that serves as an
> > 	"implementation" identifier to uniquely and
> > 	unambiguously identifies the file handle format. If
> > 	a particular implementer has multiple fh formats
> > 	(e.g. because it has both big endian and little
> > 	endian platform support), then it can use multiple
> > 	domain names or subdomain names. If the source and
> > 	target have different domain names, it is still
> > 	possible the target understands the source's file
> > 	handle format, and it can still turn on the
> > 	PRESERVE capability.

> Yes, this was underspecified and you have pointed towards
> a way out.  I had also thought (fantasized?) about trying
> to encode the way the filehandle was put together - i.e.,
> these bytes are invariant in this filesystem, these bits
> represent the specific file, and these bits represent the
> share point.  The biggest problem is that this assumes a
> filehandle will consist of all of these parts, which is
> probably not sufficiently general.

And some filehandles may be engineered to be opaque in the
colloquial sense (so hackers can't get information from them)
so they wouldn't have those nice (for other purposes) divisions.

> > Does RM_FHPRESERVE also imply preservation of fileid?
>
> For domains where it is useful, it probably will, but I
> don't know if it should be specified so.

> > On a related note, preservation of directory cookies is
> > another capability that would be good to support.
>
> Agreed - I hadn't thought about that.

> > Also, since RM_FHPRESERVE might not be supported, it
> > implies, to me at least, that any server planning on
> > supporting the source side of this this protocol and
> > currently returning a value of FH4_PERSISTENT in the
> > fh_expire_type attribute should instead warn its NFSv4
> > clients up front, and return the FH4_VOL_MIGRATION bit.
> > Otherwise, clients will be out of luck when their data is
> > migrated away.
> 
> Definitely agreed.

I agree with this but ... I have problems with the other bits,
in particular the one saying that filehandles will not 
expire while the file is open.  Without that, using volatile
file handles and doing migration is rather ... "courageous".

I have some thoughts in this area and I'll try to get a 
note out on this issue early January.

> > In Section 5.1, Data transfer phase messages, shouldn't
> > there be a SEND_DELEGATION_STATE message as well?
> 
> Possibly helpful; I'd expected we'd recall delegations
> just before pushing the final set of changes, and not
> issue new delegations until completion.

This reminds me of another issue.  The locking state is
sent over but no stateid information.  So is the model
that this looks like a reboot and the cleint has to 
reclaim his locks during the grace period.  In that
model, the purpose of sending the lock information is
just to make reclaim more reliable.  Is that what you
were thinnking?

> > In section 5.3 there is a RM_CIFS_ATTR type. Why is this
> > needed? This is the NFSv4 migration protocol, and in any
> > case, the set of NFSv4 attributes is rich enough to
> > encompass what Windows environments need.
> 
> That's there so that vendors that sell multi-protocol
> servers can express any CIFS attributes we can't encode
> with NFSv4, and to illustrate the intent that we're not
> supporting NFSv4 only, just NFSv4 first.  If this is not
> useful, it can go, but I'd like to maintain support for
> alternate attribute "profiles" in the future.

Possibly but we pulled a lot of CIFS attributes into v4.

The big problem I have with the way you specified this
is precisely that a multiprotocol server cannot sent over
information including fs attributes for all protocols.
Having it being able to send over fs attributes to support
CIFS or attributes to support V4 is not going to support
multi-protocol.

> > In section 5.8 SEND_RENAME, it seems that a SEND_LINK
> > message is also needed.
> 
> Agreed.

> > In section 5.12, CONFIRM_MESSAGE, it says CONFIRM_MESSAGE
> > is used to confirm every data transfer message. First this
> > certainly looks like RPC. But second, I don't think it is
> > desirable to confirm every data transfer message, because
> > while I think ONC RPC should be used beneath this protocol,
> > I also think there is benefit to a paradigm that doesn't
> > ACK each and every message from the target. Instead, delete
> > CONFIRM_MESSAGE, and replace with with a true RPC that
> > allows the source to ask the target the sequence number of
> > the last data transfer message it has received.
> > 
> > SEND_CHECKPOINT though, ought to be a traditional RPC.
>
> Good points; more fodder for just using RPC.

> > In section 6.3, ABORT_SESSION overloading ABORT_SESSION to
> > be both the confirmation from the target, or an initiated
> > message from the target is going to make implementation
> > complex.  Just have two different message types, or better
> > yet, RPCs.  (and make CLOSE_SESSION an RPC as well).
>
> How does the destination perform an abort?  Do we have to
> define that the source implement a server side to receive it?

Why doesn't he just give an error back the next time he is
sent something?

---

From: Robert Thurlow <[email-masked]>
Date: Thu, 19 Dec 2002 17:32:32 -0700 (MST)

[RPC]
> let's do that.

Okay, let's see what we get on the consensus call.  It's
looking good for that viewpoint :-)

> the minor version level that the attributes should correspond

Good point.

> But that's soooooo inadequate. ... This is part of the server 

That would be great - thanks!

> sent over but no stateid information.

I'd consider this an oversight; I've had brain aches in the
past thinking about how we offer a grace period to clients
using a single filesystem, and would prefer not to go there
unless it's important.

[attributes profiles]
> CIFS or attributes to support V4 is not going to support

I guess I don't see what makes it just not work.  I guess
I expected there to be CIFS-specific attributes we didn't
include, and that there was going to be a need to be able
to send them.  If the profile listed only the ones that
we didn't define with V4, would that help?  If not, what
is the problem you see?

Rob T