Subject: Re: [nfsv4] End-to-end corruption checks

From: <[email-masked]>
Date: Fri, 26 Aug 2011 13:19:19 -0400

A lot of the description of adb currently assumes (or sometimes explicitly
States) that this is for hole-equivalent data.  Given that, it is natural
to assume that it only works for the special case of an empty virtual disk.

Is that a correct assumption.

The important issue is that if the framework provided allows
1) "any application to describe that virtual disk"
2) quick initialization of it, and to 
3) detection of corruption based on the state of the block.

What is missing that most virtual disks provide?
4) ability to write data in the virtual disk (and provide pi data)
5) ability to read from the virtual disk  (and get and check pi data)

So what is the state of adb and do we need a separate adb+9 to use
disk pi facilities.

As an example, if we want the file equivalent of the oracle block-oriented
stuff that somebody posted a pointer to (there's a lot of powerpontification 
but there is some real data there too), how could we do it?  

I'd like to understand people's position about the role of adb in doing this.
Some possibilities:
A) the adb approach can do this given a new adb format and some editorial work
   like changing all the text that suggests adb's and holes are basically alike.
B) The adb approach can be built on but needs to be rethought for this 
   different sort of challenge, leading to an adb+9 that is designed
   differently and incorporates adb as a subset.
C) You should drop adb for this and start fresh.

I'm pretty sure A) is not right but I'm basically guessing.

I'm not sure if Tom is arguing for A) or B) or whether he is just saying that C) is wrong.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Dean Hildebrand
Sent: Wednesday, May 18, 2011 3:58 PM
To: Thomas Haynes
Cc: [email-masked]
Subject: Re: [nfsv4] End-to-end corruption checks

>
> But this framework we are presenting does allow for end-to-end checksums
> to be built on top of it. We could define a new ADB format in a future 
> version
> and it could provide the location, length, and algorithm needed to compute
> the end-to-end checksum.
>
> The ADB approach can be thought of as a virtual disk in a file. We provide
> a framework which allows any application to describe that virtual disk,
> to allow quick initialization of it, and to detect corruption based on the
> state of the block.
>
ADB can only detect corruption in 'empty' blocks correct?  How does it 
detect corruption is blocks that have data in them?

Dean
_______________________________________________
nfsv4 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/nfsv4

---

From: Thomas Haynes <[email-masked]>
Date: Fri, 26 Aug 2011 16:49:02 -0500

I've asked somewhat rhetorical questions until I get to the payload at
the end, i.e., I know where you are going and I basically have tried
to file in some history.

Right now the ADB work provides a generic framework to allow
enterprise applications to detect corruption at their level. Just as
we eventually decided split holes out of the ADB format and modify
READ_PLUS to handle sparse files explicitly, we could also provide
a form of READ_PLUS (and WRITE_PLUS) for other use cases
that are willing to pay the price of corruption checks at every layer
of the network stack (application, client OS, server OS, and server
storage layer, etc.).

But just as we finally reached consensus to not force normal sparse
files to pay the price for ADB, we should maintain the current ADB
format for enterprise applications which are interested in performance
over corruption checks at every layer.

> Is that a correct assumption.

No, it also works for a database file.

> 3) detection of corruption based on the state of the block.

Yes, but define how you want to detect corruption and where?

Do you want end-to-end checking? Then no, not provided.

Do you want the application (the VM manager in this case) to be
able to detect corruption on every read? Then yes, it is provided.

> 4) ability to write data in the virtual disk (and provide pi data)

Somewhat agreed, for application data checking (the 'a' in 'ADB'),
it is assumed that initialization writes (via INITIALIZE) can cover
a range of the file, if not the entire file. Likewise, a hole punch can
be accomplished by INITIALIZE.

When actual data is placed in the block, the application reads the
ADB format and places the pi data in the appropriate place. As this
is *not* end-to-end semantics, that suffices and the normal WRITE
op is trivial to use.

> So what is the state of adb and do we need a separate adb+9 to use

It meets the requirements of describing a block format that the application
can use to detect corruption.

It does not meet a requirement that allows the server and client NFS
implementations (or indeed even the underlying storage layer) to
detect corruption in flight.

The new READ_PLUS framework would make adding this easier,
i.e., we don't need READ_PLUS_PLUS. We would need a WRITE_PLUS,
but I'm think we might want to bite that bullet in 4.2 anyway.

>   like changing all the text that suggests adb's and holes are basically alike.

I want to change the text anyway. :->

> I'm not sure if Tom is arguing for A) or B) or whether he is just saying that C) is wrong.

I'm arguing more towards A. We can think of the current ADB as:

header layout information (what are the offsets for info, how big are they, how big is the block, etc.)
logical block number
state to detect corruption
data

We could easily add a new arm in the discriminated union to describe another ADB as:

header layout information
logical block number
enum describing the algorithm to calculate corruption
state for corruption
data

The enum would be an IANA registry of algorithmic approaches to detecting corruption. (Okay,
one could be "application" and indicate that the client, server, and storage not do any checks.)

For example, one could be CRC and the layers could all do an agreed upon crc check to
detect corruption since the last layer. We would probably want to define a new error code
for corruption.

And as you suggest above, both WRITE_PLUS and READ_PLUS would have to understand
this encapsulation.

---

From: <[email-masked]>
Date: Mon, 29 Aug 2011 14:02:51 -0400

Your questions couldn't have been all that rhetorical, since you came
up with some answers.

Now I want to make sure that I understand the answers.

> Yes, but define how you want to detect corruption and where?

It doesn't matter where (and how intensively) I would want to check for
corruption.  People differ in the value that they place on their
data's correctness and their degree of "paranoia" (in quotes because 
I'm not trying to judge levels of pathology here; anything the user is
willing to pay for works for me).

I think our architecture should accommodate user choice to the maximum 
degree possible.  Some people have always taken a dim view of "wasting" 
effort on correctness.  Recall "Parity is for farmers"?

The things I don't want is to have some valid/useful sort of checking
that is available in block environments but not in NFS environments.    

> Do you want end-to-end checking? Then no, not provided.

I understand it is not provided.  The issue I'm concerned about is whether
it is disallowed/prevented.  I really don't want that latter to be
the case.

Are you saying it is?

As far as I can see, if the ADB allows you to define this virtual
disk format (e.g each 256-byte block of data consists of 239 bytes of
data and 17 bytes of checksum -- two prime numbers :-), with the 
checksum format accessible, there is no protocol reason that bits
in the adb can't specifically allow you to:
1) deputize the client to check data correctness at write time, if
   the user is willing to pay the cost.
2) allow the server to be deputize to cheek data correctness on write
   and to have the disk hardware check that.
3) allow a NFSv2/v4.0,v4.1 client to cheek the data validity when it
   is read by someone who doesn't support the adb format.

Note that all of the above would be optional.

> able to detect corruption on every read? Then yes, it is provided.

Personally, I would find that fine.  Being the cynic that I am, I'd
rather have the client/server/fs/disks know nothing about how to make
the data correct.  If they know what it takes to check it, they will
know how to make it (look) correct, which may mean that they can somehow 
inadvertently make the data look correct, when it shouldn't be.

---

From: Thomas Haynes <[email-masked]>
Date: Mon, 29 Aug 2011 11:09:55 -0700

> it is disallowed/prevented.  I really don't want that latter to be

I'm saying as it currently stands, we have no way for the application/client
to tell the server to do checking.

As you point out, we can do that. And the new arm of the discriminated
union that I mentioned would do the same thing.