Subject: Re: NFSv4 Replication and Migration: design team conference call/Draft

From: Peter Staubach <[email-masked]>
Date: Wed, 5 Jun 2002 15:13:39 -0600 (MDT)

> 
> > I think compression could be worthwhile to explore.  It's not clear
> > that we can take advantage of similar blocks except for blocks of
> > zeros, which we should certainly optimize.  What amount of block
> > replication would you expect, and under what circumstances?
> 
> I'm not sure if I read it in the paper Juan referenced, but
> you could in theory avoid some data transfers if you kept
> track of each block of transferred data together with a
> checksum.
> 
> Then when you need to transfer a block of new data in
> a file, you checksum the block and see if you've transferred
> it previously as part of another file.  If you get a hit,
> then instead of transferring the block, you just tell the
> other end where it is, and where it needs to go.
> 
> I've no idea whether this happens often enough to be
> worthwhile - maybe I should go back and read Maziere's paper
> again ...
> 

Between this and the sparse file detection, the cpu overheads are
possibly starting to become significant.  Looking at a block to see
whether it is all zeros is really expensive.  There needs to be a
cheaper way.

This also doesn't match real life semantics.  We will need to be able
to replicate files created with mkfile(1M) for example.  They tend
to be populated with zeros, but are fully populated for good reasons.

A block of zeros does not mean that the block is not populated.

		ps

---

From: Brent Callaghan <[email-masked]>
Date: Wed, 05 Jun 2002 15:13:31 -0700

> whether it is all zeros is really expensive.  There needs to be a

Yes, if moving data over a multi-gigabit network, then it's
likely cheaper just to move the zeros.  But I think a typical
application of migration will be to move the data over a
relatively low speed network, where a little CPU effort
will go a long way.

> A block of zeros does not mean that the block is not populated.

It's not necessary to send the zeros - it's much faster just
to tell the other end to imagine some :-)

Actually, in my experience with ISDN and DSL links, the
link hardware is pretty good at simple compression.
So heroic efforts at zero compression in the protocol may
be wasted.

However, if it's something more subtle, like sending
block hashes and checksums, then that's something the
hardware cannot easily do.

	Brent

---

From: Robert Thurlow <[email-masked]>
Date: Wed, 5 Jun 2002 16:28:29 -0600 (MDT)

> to be populated with zeros, but are fully populated for good reasons.

Some operating environments have an interface to inform them
what parts of the file are populated; it would be a shame to
not be able to take advantage of this for large, sparse files.
This is the main thing I was thinking about.  This could be
as simple as sending an offset with every block, and not
sendign anything for the "holes".

Detecting zero blocks as the source server reads data is more
expensive, but might be worth it in some cases, e.g. Oracle
tables, if you could make the right thing happen.  I guess the
repl-mig protocol could express zero-filled extents plus the
populated-or-not status.  We need to evaluate if this is a
problem we really should solve or not; I'm inclined not to.

Rob T

---

From: Eric Werme USG <[email-masked]>
Date: Thu, 06 Jun 2002 10:41:29 -0400

> whether it is all zeros is really expensive.  There needs to be a

Tru64 has a fcntl option to return this info, added for a backup
proect.  There may also be a ioctl interface, which seems confusing,
but I get the sense other systems may have the same fcntl mechanism.

It looks like this is implemented on AdvFS, UFS, CFS (cluster FS), but not
CDFS, DVDFS, or NFS.

  F_GETMAP
      [Tru64 UNIX]   The F_GETMAP request gets the sparseness map of the file
      referred to by the fildes parameter. The argument parameter, taken as a
      pointer to type struct extentmap, is filled in with data that describes
      the extent map of the file.

      Each map entry is declared as:

           struct extentmapentry {
                   unsigned long offset;
                   unsigned long size;
           };
           struct extentmap {
                   unsigned long arraysize;
                   unsigned long numextents;
                   unsigned long offset;
                   struct extentmapentry *extent;
           };

      The map returned by this function can be different from the actual
      number of extents (or their definition) when the file is being written.
      It is recommended that you use this function only on files that are not
      being written.

> to be populated with zeros, but are fully populated for good reasons.
   
$ man mkfile
No reference page found for mkfile.

but I can guess....