Subject: Re: [ftpext] New Version of FTP HASH, RANG, LOCK, and HOST

From: Anthony Bryan <[email-masked]>
Date: Sat, 9 Mar 2013 19:19:23 -0500

thanks for the comments!

see diff for changes: http://tools.ietf.org/html/draft-bryan-ftp-lock

On Sun, Jan 27, 2013 at 6:46 PM, Alun Jones <[email-masked]> wrote:
> After reading the LOCK draft, I have a couple of comments:
>
> 0. RFC 959 already defines a mechanism - Block Mode over the default port -
> which doesn't require PORT or PASV to operate, and doesn't close connections
> between files. This means that under such a scheme, a firewall's life is
> made easier - you open 21 inbound, and allow outbound connections from port
> 20 on the server. Document this in your draft, and people may start using
> that, just because it already works on many servers.

ok, this is now appendix B. if you have any text suggestions, let us know.

> 1. You need an example showing how an upload works, just to make that clear.

added.

> "When uploading files, the client MUST use end-of-marker solution." You
> don't explain this anywhere, or use the term "end-of-marker" anywhere else
> in the draft.

thanks, fixed.

> 2. You need to answer at least the security questions of how a client is
> protected from a malicious server, and how a server is protected from a
> malicious client, as well as how the connection (client and server) is
> protected against a malicious file. Consider what happens if a file contains
> a number of predicted boundary values, and the server or client is not good
> at picking random numbers for those boundaries. Note, for instance, how SMTP
> does this in the DATA command - the DATA ends with a line containing only
> ".". Any line beginning with a "." in the data has to be prefixed by another
> ".". The receiver then simply strips out the first dot on each line. Don't
> give in to the idea that "random numbers can't be guessed", or that
> accidental occurrences or malicious action won't ever successfully reproduce
> the boundary condition. Make it IMPOSSIBLE.

it's not a secret "random" number that would be "guessed". it's
explicitly given by the sender.

> 3. At least three commands are documented to work while a transfer is in
> progress. STAT, which reports the status of the transfer, NOOP, which does
> nothing (but many clients use to keep the control connection open when a
> firewall might close it over a long data transfer), and ABOR, which stops an
> existing transfer. My suggestion would be that STAT will return a response
> after the data transfer has finished (after the response from the transfer
> completion); NOOP the same; ABOR will interrupt the transfer, the transfer's
> end response will be signaled as interrupted, and the ABOR command will send
> a 426 and 226.

Tim Kosse suggested:

The STAT command and Telnet IP and Synch signals are not allowed
during a running transfer.

any other suggestions?

> 4. Answering your question of "how to unlock" - the obvious way is to use a
> PORT or PASV (or similar) command to re-establish such behavior, or REIN if
> you want to go back to default port operation (and blow away your login,
> other settings, etc).

yes, thanks. added.

> 5. You should indicate in your examples whether the boundary and separator
> lines are sent by client or server, just to avoid confusion.

added.

> Alun.
> ~~~~
>
>> -----Original Message-----
>> From: [email-masked] [mailto:[email-masked]] On Behalf
>> Of Anthony Bryan
>> Sent: Sunday, January 20, 2013 9:01 AM
>> To: [email-masked]
>> Subject: [ftpext] New Version of FTP HASH, RANG, LOCK, and HOST
>>
>> we updated the FTP HASH, RANG, and LOCK drafts, while the authors of
>> HOST have updated their (hopefully final, get any comments or reviews in
>> ASAP) draft.
>>
>> we are mostly interested in finishing up HASH & RANG, but also feel free
> to
>> comment on LOCK. fancy diffs are available. HASH, RANG, & LOCK changes
>> are all editorial or just updating the draft.
>>
>>
>> HASH FTP command to be used by clients to request cryptographic hashes of
>> files.
>> http://tools.ietf.org/html/draft-bryan-ftpext-hash
>>
>>
>> RANG command to be used by clients to designate a start and end point to
>> permit restarts and repairs of interrupted data transfers in STREAM mode.
>> http://tools.ietf.org/html/draft-bryan-ftp-range
>>
>>
>> LOCK command to be used by clients to request the server to use the
> control
>> connection for data transfers, using a single port instead of two.
>> http://tools.ietf.org/html/draft-bryan-ftp-lock
>>
>>
>> also, HOST has been updated:
>>
>> FTP command that provides a mechanism for FTP clients and servers to
>> identify individual virtual hosts on an FTP server.
>>
>> http://datatracker.ietf.org/doc/draft-hethmon-mcmurray-ftpext-ftp-hosts/
>> http://tools.ietf.org/html/draft-hethmon-mcmurray-ftpext-ftp-hosts (diffs)

--
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads

---

From: Anthony Bryan <[email-masked]>
Date: Sat, 9 Mar 2013 19:32:06 -0500

Daniel Stenberg wrote elsewhere:

"I agree that sending data in a chunked-encoding style (from HTTP 1.1)
is much easier for both sender and receiver instead of having to check
for a boundary separator in the stream.

It would be good to get an FTP server author's point of view on this."

any other +1s or opinions?

On Sat, Feb 2, 2013 at 12:59 AM, Hans Andersen <[email-masked]> wrote:
> I just wanted to back my colleague, Bruce, on the performance issue related
> to detecting boundaries.  A few weeks ago I wrote code that detects MIME
> boundaries in HTTP-POSTs while piping the data to a file.  The best I
> managed to achieve was significantly slower than simple piping.  I therefore
> think that optionally including the size is important for performance.
>
> On 2 February 2013 14:26, Anthony Bryan <[email-masked]> wrote:
>>
>> right, we could leave the boundary REQUIRED but have an OPTIONAL size
>> that would be used (probably the majority of the time?)

---

From: Anthony Bryan <[email-masked]>
Date: Sat, 9 Mar 2013 19:44:22 -0500

On Mon, Jan 28, 2013 at 4:04 PM, Ángel González <[email-masked]> wrote:
> On 28/01/13 01:01, Alun Jones wrote:
>> After reading the RANG spec, I note that there isn't anything to state what
>> errors are given by RETR if the range specified is incorrect, or by STOR /
>> APPE if the range specified starts more than one byte after the end of the
>> existing file on the server (and therefore, presumably, incorrect?).

current text is:

The server-PI should reject the STOR if a client issues a RANG with a
start-point greater than the actual file size. If the start-point is
less than or equal to the file size, the file is opened for writing,
continuing at start-point.

> I think it should be documented that the server must fill the gap with
> nul bytes,
> with a note that it may use it as a hint to do that using a sparse file.

any other feelings on this?

on a side note, wondering if it might be better to separate RANG and
RETR/STOR/etc_with_RANG into 2 separate drafts.
HASH is also a separate draft that happens to use RANG.