Subject: [nfsv4] Global Namespaces Chapters Review #2 - minutes

From: Trond Myklebust <[email-masked]>
Date: Tue, 24 Jul 2007 17:37:05 -0400

Daniel Ellard		(NetApp)
Craig Everhart		(NetApp)	Reader
Trond Myklebust		(NetApp)	Scribe
Dave Noveck		(NetApp)	Author
Theresa Raj		(NetApp)

Renu Tewari		(IBM)

Robert Thurlow		(SUN)
Spencer Shepler		(SUN)		Moderator

------------------------------------------

Spencer started off with reminding us of the IETF blurb...

-----

11357	Myklebust	How is staleness reported by the server?
	Noveck		Text is modelled on v4.0. Might need to go back
			now that we have sessions and see whether we need
			to expand text or reference.
-----
11446	Myklebust	Do we need a discussion here about the combination
			of volatile filehandles and volatile fileids.
	Noveck		Agrees we need a paragraph based on the
			discussion we had on the list.
	Ellard		Is there a situation where either filehandles or
			fileids (but not both) are volatile?
	Noveck		Yes.
	Thurlow		You cannot rely on fileids since they are not
			mandatory.
			Maybe there are some shoulds that should be
			SHOULDs....
	Agreed to take this to the list
-----
11458	Noveck		Maybe FSLI4F_MULTI_FS should be a separate
			section since it is being introduced here.
-----
11511	Everhart	Dave promised a discussion on filesystem
			replication.
	Trond		Will bring this up on the list.
-----
11529	Everhart	'it does necessarily mean' should be '...does not...'
-----
	Trond		Need a better 'reclaim is done' marker.
	Noveck		Can do RECLAIM COMPLETE.
	Everhart	How does the server know all clients have sent
			reclaim complete?
	Trond		Could we discuss an alternative based on a special
			session? Will bring to the list.
-----
11653	Everhart	See [comment.8].
	Noveck		Need to look at this more. Will return to the group
-----
11638	Raj		Does the server need to remember that it has sent
			an ERR_MOVED? You need to send LEASE_MOVED prior to
			that.
	Everhart	Also need to wait for the GETATTR w/ fs_locations
	Noveck		Once client gets LEASE_MOVED, it needs to iterate
			through filesystems and do a GETATTR to get
			ERR_MOVED. The iteration should only stop when the
			LEASE_MOVED flag is cleared.
	
	Thurlow		Should we talk about fs_status rather than
			fs_locations_info since that is a lighter weight
			operation.
	Noveck		The GETATTR doesn't need to be fs_locations_info.
			It is the ERR_MOVED result that counts.
			fs_status might be good alternative to ERR_MOVED
	Trond		Could you use NVERIFY with fs_status followed by
			a fs_locations_info in order to do one-pass?
	Noveck		The server will clear LEASE_MOVED on first
			successful fs_locations_info
-----
11722	Everhart	The NOTE there is obsolete. The paragraph already
			addresses the issue of scope.
	Noveck		Agrees.
-----
11728	Ellard		The server may declare that all verifiers are invalid.
	Everhart	Yes. Client should fall back to synchronous writes.
			This would be the case if server keeps going up
			and down...
	Noveck		The thing to stress is to not recognise a verifier
			as valid when it is not. Then delete the rest of
			the text.
-----
Pause while we waited for the author to change rooms....
-----

11935	Thurlow		Much handwaving there is in this text! Could we
			simplify it?
	Noveck		Agrees
-----
12077	Trond		rdattr_error should be NFS_OK because the READDIR
			request included fs_locations_info?
	Noveck		Agrees
12709	Noveck		Typo...
12085	Thurlow		What is the reference.
	Noveck		attributes are not included 'cos of NFS4ERR_MOVED
------
12087	Noveck		Delete section 10.8 since we don't need fs_absent.
-----
12167	Everhart	should -> SHOULD
	Noveck		OK.

	Trond		Should we have any recommendations about when it
			is incorrect to fail to implement fs_locations_info.
			Basically list the cases that the fs_locations_info
			enables, and that cannot be supported by fs_locations.
	Noveck		Agrees. In cases where fs_locations_info will
			return different results to fs_locations you SHOULD
			implement former. Otherwise client has no way of
			knowing what it should do.
			Extend paragraph starting at 12212.

			Trond will suggest text.
-----
12255	Thurlow		Need to add context in order to explain that
			is offsets into fs_locations_info.
-----
	Everhart	What is the scope of fs_locations_info?
			Will all replicas give the same fs_locations_info?
	Noveck		No. There is no requirement that fs_locations_info
			be synchronised. One replica may know about the
			existence of other replicas that are not known to
			all.
			Even attributes may differ. Filehandles may be of
			the same class when you go in one direction (from
			replica A to replica B), but not when you go in the
			reverse direction.
			Need text to discuss this at end of 12339.
-----
12345	Raj		Is fls_currency supposed to be globally consistent?
	Noveck		Yes.
-----
12553	Everhart	Do we need a READDIR cookie change class?
	Noveck		Hadn't thought of that. Maybe...
	Trond		Could we use READDIR verifiers?
	Noveck		Problem is that you need to agree on verifiers
			across servers.
		Consensus is add READDIR change class
	Noveck		Need to change CLVERIFIER to reflect it is write
			verifier.
			Don't need change directory verifier. That is
			part of cookie change class.
-----
12424	Thurlow		Guarantee implied here. Is that enforcable?
	Noveck		Issue is what clients can depend upon. If client
			migrates, it needs to know that it doesn't have
			to replay committed writes.
	Thurlow		A client may still want to migrate if it has no
			choice....

	Thurlow will initiate further discussion on the lists.
-----
12567	Thurlow		Don't like the fact that server can order me to
			a particular replica.
	Noveck		Add language to clarify. Load balancing considerations
			may not be part of a client calculation. Otherwise
			server should be conservative with ordering.
-----
12628	Thurlow		What is fli_valid_for? Not defined in the structure.
	Noveck		Will add to structure. Needs to be int32.
-----
12648	Thurlow		Incomplete sentence?
	Noveck		Need to extend sentence. Governs what 12660 is about.
			Move this sentence over to 12662
-----
12660	Thurlow		We need a centralised location for documenting
			substituted variables so that administrators can
			know.
				IANA registry?
				Informational RFC?
	Needs further discussion.	

_______________________________________________
nfsv4 mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/nfsv4

---

From: Robert Thurlow <[email-masked]>
Date: Wed, 25 Jul 2007 10:16:39 -0600

In a message with Subject "Global Namespaces Chapters Review #2 - 

> 	Thurlow will initiate further discussion on the lists.

Here's what that was about.

The clause in question is from 10.10.1, regarding fs_locations_server4:

12419     The general file system characteristics flag (at octet index
12420     FSLI4BX_GFLAGS) has the following bits defined within it:
12421
12422     o  FSLI4GF_WRITABLE indicates that this fs target is writable,
12423        allowing it to be selected by clients which may need to 
write on
12424        this file system.  When the current file system instance is
12425        writable, then any other file system to which the client might
12426        switch must incorporate within its data any committed write 
made
12427        on the current file system instance.

My read of this is placing high standards on how the server
does replication of received changes.  The language seems to
want to make it safe for the client to switch without facing
any possibility that a previous change was not reflected.

This seems different in intent from a distinct theme present
in other parts of the draft that want to give the client good
information about the currency of replicas and let them choose.
I would expect that if a very good replica is available, there
will be no difference in outcomes, but if the only writable
copy is out-of-date, the language above implies to me that the
client will not be offered that copy.  I think we should move
towards providing good information to the client rather than
trying to require the server to meet this guarantee in all
cases.

Thoughts?

Rob T

---

From: "Everhart, Craig" <[email-masked]>
Date: Wed, 25 Jul 2007 13:54:28 -0400

Perhaps the goal is to treat writable replicas in a fundamentally
different way from read-only ones?  Read-only ones may age but writable
ones may not?

The NFSv4 spec probably can't legislate against multiple writable
replicas that can be mutually out-of-date, whether we individually think
that's a good idea or not.  But I think we need something more than the
current language and semantics to describe such cases.  Today the
fs4_status structure has fsstat_age and fsstat_version, which make
perfect sense if there's only one sequence of updates for that
filesystem, but which make much less sense if multiple streams of writes
can be sustained.  Well, perhaps fsstat_version could make some sense
since it's a timestamp, but it doesn't seem very usable since a client
really doesn't know whether a prior change is present or will ever be
present.  fsstat_age doesn't make sense among mutually-inconsistent
writable replicas.

Rob's scenario could make sense more or less within how I read
draft-11's intent if what he means is that the only repository for the
most recent writes has been deemed lost, so a set of servers is now
promoting a relatively-recent replica to writable status with
concomitant data losses.

		Craig

> -----Original Message-----
> From: Robert Thurlow [mailto:[email-masked]] 
> Sent: Wednesday, July 25, 2007 12:17 PM
> To: Trond Myklebust
> Cc: [email-masked]
> Subject: [nfsv4] Replica currency
> 
> In a message with Subject "Global Namespaces Chapters Review 
> #2 - minutes", Trond Myklebust wrote:
> 
> I would expect that if a very good replica is available, 
> there will be no difference in outcomes, but if the only 
> writable copy is out-of-date, the language above implies to 
> me that the client will not be offered that copy.  I think we 
> should move towards providing good information to the client 
> rather than trying to require the server to meet this 
> guarantee in all cases.
> 
> Thoughts?
> 
> Rob T
>

---

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 26 Jul 2007 14:41:14 -0400

We can legislate all we want.  We just don't have any enforcement
powers.  Not that I really want any, mind you.

I think the important thing is to give the servers a way to communicate
the guarantees they are giving the clients and to make it clear to the
clients how to interpret the those guaranees, so that nobody is
surprised.

As regarding the currency issue, I don't really think that multiple
independently-updated replicas really conform to that model very well.
If we are going to allow this, I think we just have to say that when
there no authoritative source (which is the case when there are multiple
writable versions), we can't have a meaningful concept of being current
and the thing we have to do is not pretend to provide what we can't
provide.

-----Original Message-----
From: Everhart, Craig 
Sent: Wednesday, July 25, 2007 1:54 PM
To: Robert Thurlow; Trond Myklebust
Cc: [email-masked]
Subject: RE: [nfsv4] Replica currency

		Craig

> My read of this is placing high standards on how the server does 
> replication of received changes.  The language seems to want to make 
> it safe for the client to switch without facing any possibility that a

> previous change was not reflected.
> 
> This seems different in intent from a distinct theme present in other 
> parts of the draft that want to give the client good information about

> the currency of replicas and let them choose.
> I would expect that if a very good replica is available, there will be

> no difference in outcomes, but if the only writable copy is 
> out-of-date, the language above implies to me that the client will not

> be offered that copy.  I think we should move towards providing good 
> information to the client rather than trying to require the server to 
> meet this guarantee in all cases.
> 
> Thoughts?
> 
> Rob T
>