Subject: Repl-mig protocol skeleton

From: Robert Thurlow <[email-masked]>
Date: Tue, 27 Aug 2002 17:31:09 -0600 (MDT)

Hi all,

I've been working on a strawman protocol for replication/migration,
and I don't have that yet, but I do have this skeleton I can start
gluing straw onto.  This is what the protocol looks like from here;
let me know if you see problems so far.  I'll keep on adding detail
and work towards getting this into Internet Draft format.

Rob T

-----

Initiation Phase: authentication, negotiation, filesystem info
Restart Phase [initiate when restarting]: add restart info - much the same
Data Transfer Phase: send attributes and data for each file
Termination Phase: final handshake

Initiation/restart phase messages:

The source server starts by sending OPEN_SESSION_NEW (for a new
transfer) or OPEN_SESSION_RESUME (to resume an old transfer).
The destination server responds with either OPEN_SESSION_CONFIRM
or OPEN_SESSION_DENY.  SECINFO_REQ and SECINFO_RESP and the
security tokens in each message will be necessary if we are
requiring per-message security.

SECINFO_REQ / SECINFO_RESP - get security flavor
- Should be based on NFSv4

OPEN_SESSION_NEW - create new transfer session
- Security token
- Session ID
- Source FS path
- Destination FS path
- Size of FS
- Expected transfer size
- Expected number of objects
- Permissable compression algorithms
- Permissable encryption algorithms
- Filehandle preservation flag

OPEN_SESSION_RESUME - resume previous transfer session at checkpoint
- Security token
- Session ID
- Checkpoint ID
- Expected remaining transfer size
- Expected remaining number of objects
- Permissable compression algorithms
- Permissable encryption algorithms
- Filehandle preservation flag

OPEN_SESSION_CONFIRM - accept transfer session
- Security token
- Session ID
- Selected compression algorithms
- Selected encryption algorithms
- Filehandle preservation flag

OPEN_SESSION_DENY - decline transfer session
- Security token
- Session ID
- Status code

Data transfer phase messages:

The source server begins processing filesystem objects in some fixed
order which will permit checkpointing and restarting in case of some
problem or operator abort.  SEND_OBJECT_METADATA is sent first, then
SEND_FILE_DATA messages will be sent for non-directory objects.  If
outstanding lock state for an onject exists on the source server, it
will be sent via SEND_LOCK_STATE messages.  If the source server
cannot capture all create and remove operations on a directory
reliably, SEND_DIRECTORY_CONTENTS will permit the destination
server to validate its directory entries.  Named attributes are
handled with SEND_OBJECT_METADATA messages with IS_NAMED_ATTR
set to true, and apply to the previous non-named-attribute which
was handled.  CLOSE_OBJECT is used to indicate that all data and
named attributes of an object have been transferred.  At any time,
the source server may set a checkpoint with SEND_CHECKPOINT.  All
messages are confirmed by the destination server with CONFIRM_MESSAGE.

SEND_OBJECT_METADATA - send metadata about onject
- Security token
- Message ID
- File ID
- Object name
- Object type
- Object ACL
- IS_NAMED_ATTR
- NFSv4 attrs
- Other attrs

SEND_FILE_DATA - send file data
- Security token
- Message ID
- File ID
- Data offset
- Data length
- IS_HOLE
- Data

SEND_LOCK_STATE - send file lock state
- Security token
- Message ID
- File ID
- Array of:
  - Lock offset
  - Lock length
  - Lock owner

SEND_DIRECTORY_CONTENTS - send names of objects in a directory
- Security token
- Message ID
- File ID
- Dir cookie
- EOF flag
- List of object names

SEND_CHECKPOINT - send checkpoint info
- Security token
- Message ID
- Session ID
- Checkpoint ID

CLOSE_OBJECT - signal completion of object
- Security token
- Message ID
- File ID

CONFIRM_MESSAGE - confirm any data transfer message
- Security token
- Message ID

Termination phase messages:

ABORT_SESSION may be used at any time by either server to terminate
a session prematurely; a checkpoint ID is recommended to permit
restart if possible.  CLOSE_SESSION is the normal termination
message, and must be issued by the source server first and then
issued by the destination server as a confirmation.

ABORT_SESSION - end transfer session before natural end
- Security token
- Session ID
- Checkpoint ID

CLOSE_SESSION - complete transfer session normally
- Security token
- Session ID