Subject: Ftp-WG: Fwd: I-D ACTION:draft-saul-ftp-plus-00.txt

From: Mike Saul <[email-masked]>
Date: Wed, 6 Sep 2000 20:20:54 -0500

> > Perhaps a better solution here would be to use the EPRT and EPSV
> > definitions of RFC 2428 in place of the XPRT and XPSV commands.  The only
> > trouble is there is no address family number for XTP in RFC 1700.  We
> > would have to get one from IANA.
> 
> If all you're missing is an assignment of an address family name/number
> from IANA, the best solution would be to take that route.  It should make a
> number of things easier as well; for or instance, I would not be surprise
> to find that, with the assignment, you would find it a lot easier to get
> POSIX acceptance of XTP's address family for such things as the bind()
> function.

Gregory,

I've had to think about this (and other things) for a while.  Let's see how the 
request for XTP address family(s) goes with IANA.  It would make things
a lot easier if we could go that way.  However, RFC 2428 currently refers 
specifically to tcp-port in the format of the EPRT command:

EPRT<space><d><net-prt><d><net-addr><d><tcp-port><d>

so some kind of explanation (or modification is needed somewhere).

The other alternative would be to have a separate command that indicates the
transport (TCP, XTP) in addition to the EPRT/EPSV commands.

> > I understand.  So I would like to propose some other ways to do this.
> > One alternative is to modify the REST command to add an optional length
> > parameter.  The only problem is that in some cases we are not trying to
> > RESTART a transfer and we are just trying to send part of a file.  So
> > instead of overloading the REST command perhaps a better solution would
> > be to create a STPA (Store Parameters) and a RTPA (Retrieve Parameters)
> > command?  There would be no file names associated with these commands, so
> > the "file names with spaces" problem would be eliminated.  Note that
> > having both an offset and a length parameter (to some acceptable command)
> > should be of general interest to this working group and the IETF
> > community.
> 
> Historically, the intention of FTP was _file_ transfer.  Not record
> transfer, nor any form of partial-file transfer.
> 
> The use of REST to restore an interrupted transfer arose from the
> realization that most files available via FTP do not change very often (if
> ever), some are exremely large, and one would desire to not have to spend
> the bandwidth on retransmission of the first few gigabyte of a download
> simply because of a loss of connectivity.
> 
> The problem with REST is there is no way to ensure the file has, in fact,
> not changed.  This is why its use with stream-mode transfers has never been
> sanctioned.  The MLST draft codifies existing practice in this respect,
> simply by pointing out that one must take steps to attempt to avoid the
> problem of pasting the end of one generation of a file onto the beginning
> of an earlier one; it offers suggestions on how to do this, but no
> solutions.
> 
> You will find the FTP Working Group divided on the subject of partial
> transfers.  The majority view, however, has historically been that the FTP
> is not intended to be a replacement for more general file access methods
> (such as NFS and SMB).  So I would expect sufficient resistance to a
> generalized record-update method such as your proposal appears to be
> attempting to provide to effectively block any such effort.
> 
> The question is: what are you attempting to accomplish?  If what you're
> trying to do is provide a means to download (and, perhaps, upload and
> replace), say, a specific frame or scene from a multi-frame or multi-scene
> video file I would suggest the best way to handle this would be to
> synthesis some form of "partioning" on the file.  One could take the view
> that your "video file" is simply a "directory" and that each "file" within
> it corresponds to some "frame" or "scene".
> 
> With such a scheme, one approach would be to normalize the naming
> (addressing) within the FTP NVFS.  Since your current desire is to use raw
> offsets and lengths, this does not seem appealing.  If, however, one
> chooses to view a video file as a "partitioned dataset" where each part has
> a symbolic name, this would seem the way to go.
> 
> Another scheme would be, as you seem to be suggesting, some form of
> parametric lead-in command.  I would envision this working similar to RNFR,
> in that the lead-in command would return an intermediate reply and the
> server would enter a state waiting for a STOR or RETR command.

Yes, this is what I was suggesting.

> Bear in mind that there will be no means to ensure synchonization.  This
> means that read-modify-write sequences will be inherently unreliable.  We
> have that problem now, but only at the "file" level, and the problem is
> mitigated by the fact that most FTP sites separate upload and download
> areas, leaving the problem of in-place replacement of files to the users to
> handle.
> 
> It's late, and the only reason I'm online right now is my cold meds wore
> off.  My mind is spinning with ideas on how to handle what I see as the
> problem you're trying to solve.  I'll leave this subject for now, with the
> following thoughts:
> 
>  - as I see it, there are two issues; the support for XTP as a transport
>    mechanism .. which obtaining IANA assignment would be the first step
>    toward solving .. and the ability to store or retrieve partial files.

Correct.

>  - I do not see how partial file support is a _requirement_ for XTP
>    support.

That is also correct.  That is why your suggested title "XTP Extentions to FTP"
doesn't quite fit.  The SMPTE/EBU task force thought that both of these issues
were important enought to stick under the FTP+ umbrella.  See below.

>  - it would help me if I knew what you're trying to accomplish with partial
>    file support.  Specifically, is raw byte-offset the only acceptable
>    method to you?  If what you're really trying to do is say "replace this
>    scene with this other one", that's a completely different problem from
>    simple byte offsets.

I think that at this time the desire for a raw byte-offset is still a real
goal.  The following is from one of the originators of FTP+, Al Kovalick.

 The comments from Mike and Gregory were very interesting. I have been
involved with the requirements of FTP+ from the start.  I must admit we
didn't see the "file name with spaces" problem although I use such every
day.  Live and learn.  Regarding the read partial file command form, this
was an early requirement. Although I appreciate Gregory's ideas, the world
of media formats will not change for us nor is any type of reformatting
practical.  Users may need to access only a portion of (say a 10 GB) a movie
for a credit check. A partial read is manditory.  Sure we considered NFS and
SMB but we need the functionality in FTP for sake of "one stop shopping".  

>  - the problem of synchonization in partial file support is sufficiently
>    complex that it's unlikely you will be able to solve it within the
>    confines of the FTP.  While it is possible to offer some limited support
>    for it, you might be better off shifting the problem to some other
>    protocol which already has solutions (such as NFS or SMB).

I agree that updating a part of a file (STOR) presents special synchronization
problems, whereas the sending of part of a file (RETR) shouldn't present any 
more synchronization problems than we already have.

>  - separating your needs to support XTP as a transport mechanism from your
>    needs for partial file support is probably a Good Idea(tm).  It means
>    the base support of the transport mechanism can go forward.  At the same
>    time discussion of partial file support could be made more generic.  Put
>    another way: I see apples and oranges in your basket; why not make it
>    two baskets so those who only want apples don't have to take the oranges
>    with them?

Yes, I can see where the separation of the two issues would be a good idea
under this forum.  I'm open to ideas on how I should proceed, and thanks for
all of the input.  At some point my head will also stop spinning.
 
> > Here we are not trying to find out whether the server supports the RATE
> > command; we want to know the current value.  Is FEAT still more appropriate?
> 
> Yes.  And OPTN could be used to modify the value if you so desire.
> 
> > I understand.  The second parameter to SIZE is not really necessary.
> 
> What I was getting at was that you were attempting to discover a "fact"
> about a file.  While MLST is not yet at RFC stage, my suggestion is that it
> would be more appropriate to use MLST to discover the fact.  All that
> should be required in this case (once MLST reaches RFC stage) would be the
> registration of the "fact" code(s) with IANA.
> 
> -- 
> 
> Gregory A Lundberg              WU-FTPD Development Group
> 1441 Elmdale Drive              [email-masked]
> Kettering, OH 45409-1615 USA    1-800-809-2195

Mike Saul
[email-masked]

---

From: Alun Jones <[email-masked]>
Date: Wed, 6 Sep 2000 20:46:09 -0500

>were important enought to stick under the FTP+ umbrella.  See below.

I think "FTP+" is very definitely the wrong term to use.  It implies that 
it continues the generic FTP as it stands into future versions - whereas I 
think your proposal is more aimed at a small subset of possible users' demands.

To add the suggested features in as generic additions to FTP would require 
so much in the way of file locking, etc, that we'd find ourselves heading 
into the territory of reinventing other protocols that are already better 
at that task.

Alun.
~~~~

--
Texas Imperial Software   | Try WFTPD, the Windows FTP Server. Find us at
1602 Harvest Moon Place   | http://www.wftpd.com or email [email-masked]
Cedar Park TX 78613-1419  | VISA/MC accepted.  NT-based sites, be sure to
Fax/Voice +1(512)378-3246 | read details of WFTPD Pro for NT.

---

From: Gregory A Lundberg <[email-masked]>
Date: Thu, 7 Sep 2000 06:17:49 -0500

> Users may need to access only a portion of (say a 10 GB) a movie for a
> credit check. A partial read is manditory.

.. and monumentally WRONG.  I almost find myself in disbelief that
reasoning such as this continues on the Internet.

Your requirement for partial read/write file access is not based upon a
need for shared-access file maintenance (for which FTP is remarkably
unsuited).

Do you have ANY reasons for these features which have nothing to do with
authentication and/or authorization?

If not, from everything I've seen, I suggest dropping all partial transfer
discussion from your draft; leaving you with a simple draft updating
existing RFCs to be less transport-method specific in certain phrases and
identifiers, possibly with a few minor protocol changes in the same vein.

--