Subject: NFS Vendor's Conference / Interest in replication and migration

From: Robert Thurlow <[email-masked]>
Date: Tue, 4 Dec 2001 11:47:34 -0700 (MST)

Hello all,

At the NFS Vendor's Conference (http://www.nfsconf.com/) held October
23-24, I presented a talk on Migration and Replication for NFSv4. (See
http://www.nfsconf.com/pres01/thurlow-color.pdf for the slides.)  This
is a business-oriented conference focusing on future trends in NFS, as
many of the old NFS hands here know.

In the resulting discussion, it was clear that several participants
were interested in finding a way to proceed on this work.  Brian
Pawlowski of Network Appliance expressed an opinion that NFSv4 was
incomplete without a heterogeneous protocol for migration and
replication.  Network Appliance, UMich/CITI, HP, Hummingbird and
Sun were all interested in moving forward with this.  This seemed
to be a good number of interested parties and a large part of the
audience.

I want to find out how this response lines up with that of this
working group, and have received some time on the agenda at the
NFSv4 session at the coming IETF.  I think the time has come to
hash out if we should do this work in the NFSv4 working group or
in some other way (e.g. a separate IETF WG or outside the IETF),
and to find out who is interested.  Please post your opinions here
or bring them to Salt Lake City.

Rob T

---

From: Brent Callaghan <[email-masked]>
Date: Tue, 04 Dec 2001 11:51:14 -0800

> or bring them to Salt Lake City.

I'm a big fan of migration and replication. I use it every day to
sync web pages outside the Sun firewall to a web server. The
NFSv4 mailing list archive (incl this message) is propagated this
way ( http://playground.sun.com/pub/nfsv4/nfsv4-wg-archive ).
The files comprising the Hypermail archive are sync'ed with a
lovely utility called "rsync"  (see http://www.rsync.org )

Rsync is fast and efficient.  It synchronizes just the files that
have changed, and uses a clever rolling hash algorithm to compute
the chunks of a file that have changed (good when syncing large
tar or zip archives).

Although rsync is a great tool for me, it just whets my appetite
for this level of filesystem synchronization and file movement.
It would be great if there were a tool that would allow NFS
servers (not just v4 ones) to relocate and synchronize files
amongst themselves.  Unfortunately, I can't recommend rsync
unreservedly because:

	- The protocol is not published.  It exists only within the
	  implementation of rsync.

	- Current implementations are all UNIX or LINUX based.
	  There appears to be no attempt at adapting to Windows
	  or MacOS.

	- It doesn't scale.  It's great for sync'ing small
	  collections of files, but for large filesystems
	  with a million files or so, most of the sync 
	  time would be taken up just walking the filesystem
	  to find changes.

My guess is that an industrial strength filesystem sync'er
that fixed these problems would be quite popular.  The big
advantage over block-level protocols is that you can sync
between different filesystem types, e.g. ext2fs and WAFL.
It would be great not *having* to know what kind of filesystems
they are - just as with NFS you can access files without knowing
the filesystem.

My hope is that we can come up with a protocol that will spawn
a variety of implementations: from small rsync-style apps, to
heavy-lifter, industrial file syncers that handle millions of
files at a time.

The IETF would be a great venue for this protocol work - and
currently the NFSv4 WG seems like a good WG to base it.

A thumbs up from me - FWIW.

	Brent

---

From: Erez Zadok <[email-masked]>
Date: Tue, 4 Dec 2001 19:58:58 -0500

In message <[email-masked]>, Brent Callaghan writes:
[...]

> 	  time would be taken up just walking the filesystem

That's the worst feature about rsync, IMHO a short-sighted design wrt
scalability.  I've had it blow up and die w/ ENOMEM for lack of memory on
the server side to build a file list for an entire directory tree.  Rdist
OTOH will walk the f/s one directory at a time, handling things more
incrementally, and is thus a more scalable system.  Today, I use a
combination of rsync+rdist (over SSH of course) to sync up several Gigs of
my files, email, etc. over several locations.

Anyway, once rsync finishes producing its file list, I have to admit it's
really fast.  So I guess we can sum it up as:

- rdist is faster in sync'ing meta-data
- rsync is faster in sync'ing data

This WG should also probably look at: SUP, Coda and a number of other
systems (including block-level syncing systems).

> heavy-lifter, industrial file syncers that handle millions of

A long time ago, in my sysadmin days, a colleague and I jotted down a list
of requirements and features of a file syncing protocol that uses both a
push and pull (rdist vs. sup).  We liked both tools, but wanted something
that'll combine the benefits of both.

> A thumbs up from me - FWIW.

As a mostly-lurker on this list (who just caught up to 9 months of WG mail
:-), I agree.  This will be a very useful tool and protocol.

> 	Brent

Erez.