Subject: RE: NFS over TCP stuff

From: "Noveck, Dave" <[email-masked]>
Date: Thu, 18 Jul 2002 12:58:47 -0700

Mike Eisler wrote:
> Brent Callaghan wrote:
> 
> > Having said that, it's unfortunate that the protocol doesn't allow
> > the server to advertise its capacity to accept load and provide some
> > gentle back-pressure, rather like a TCP window.  This would be
> > particularly valuable for read-ahead and write-behind.  If the
> > client gets too aggressive with async ops, it can exceed the
> > server's capacity.  The only way the server has to signal this
> > it to drop requests, and this has a significant performance impact.
> 
> v3 added NFS3ERR_JUKEBOX. v4 has others in addition.
> They are primitive, 

They're not nearly as primitive as just dropping the request.  Think 
of it this way.  They both belong to the stone-age, but one's paleolithic
and the other neolithic. 

> but the fact that they don't get
> get used suggests to me that a more complex protocol is
> even less likely to be used.

I'm not sure that follows.  I think the major reason they aren't used
is the pervasive idea that NFS means you can drop requests and never
have to say you're sorry.  

I don't think we are ready for a space-age solution to this problem.
I think developing the more complex protocol is just too much to take
on right now.  I think v4.0 can just about manage a neolithic solution.

Even if we don't decide to go all the way to you MUST NOT drop 
requests on TCP and you MUST not retry, we can at least say that
that you SHOULD NOT drop them and let clients really crank up their
retry times, without having to worry very much about servers dropping
their requests because it is the simplest way to shed load.

---

From: David Robinson <[email-masked]>
Date: Thu, 18 Jul 2002 15:56:04 -0500 (CDT)

> A "back pressure" protocol would allow NFS clients to exploit
> unused server capacity.  For example, I think most implementations
> have hard-coded limits on the amount of read-ahead and
> write-behind concurrency.  A performance advantage might
> be a more compelling feature.

Because NFSv4 is required to run over TCP, we already have
a back pressure protocol. The TCP window.  If a server can't
handle more requests it can simply stop pulling them off
the connections and the flow will stop.

	-David

---

From: "Noveck, Dave" <[email-masked]>
Date: Fri, 19 Jul 2002 00:24:59 -0700

One thing to add is that good performance in such networks is one
of the goals of NFS-v4.  We are doing COMPOUND primarily to help
deal with high-latency connections.  Delegations, although they are
of benefit in other situations, are of particular help when network 
connections are less than ideal.

So, if we are willing to do something as big as COMPOUND to help in
such environments, we certainly should be willing to avoid wasting
precious bandwidth re-sending stuff that doesn't need to be resent.  

-----Original Message-----
From: RJ Atkinson [mailto:[email-masked]]
Sent: Thursday, July 18, 2002 11:07 PM
To: nfsv4-wg
Subject: Fwd: NFS over TCP stuff

	Something to consider (that I maybe should point out) is that my 
network
situation is often a bit different from other folks'.

	I regularly am behind a low-bandwidth link into a public WAN using 
an IPsec VPN
(IPsec endpoints are in my basement and at HQ) to connect to several NFS 
servers
that are on the opposite coast of the US.  Bandwidth saved from NFS 
re-sending
big blobs can be a high percentage of my total uplink capacity (and I'm 
concurrently
doing other stuff besides NFS that really can use that uplink capacity).
There are occasions when I have as much as shared-T1 into the public 
Internet,
but more typically I've got 128Kbps shared uplink into the public 
Internet.

	This is likely different from the situation in many people's minds,
such as LAN with 10/100baseT to the desktop and maybe a 100/1000 
Ethernet interface
into the NFS server in the same building, often to an NFS server on the 
same floor
of the same building.

	I would gently suggest that one reason I'm using NFS/TCP for my 
situation
is that SMB is totally unworkable (way too chatty on the wire and way too
intolerant of high latency paths).  So this is an area where NFS already 
has some
advantages and could have a lot more operational advantage for little 
implementation
cost.

Thanks for listening,

Ran
[email-masked]

---

From: "Talpey, Thomas" <[email-masked]>
Date: Wed, 24 Jul 2002 13:01:04 -0400

The layered approach to the problem! I agree completely, and want
to add that it makes retransmission on live TCP links completely
pointless.

The layered approach also obviates the server using any kind of
negative acknowledgement like NFS3ERR_JUKEBOX, which by
the way leads to reuse of RPC XIDs and therefore munges the
protocol hopelessly.

Why reimplement transport semantics in NFS, except over
historical transports such as UDP?

Tom.