Subject: Ftp-WG: NLST vs MLSD

From: Jeffrey Altman <[email-masked]>
Date: Sun, 13 Oct 2002 19:35:36 -0500

> You are spouting nonsense. Users don't talk to servers. Users talk to
> clients. Clients talk to servers. Specifically:
> 
>    (1) The user tells your software ``mget pics/mae-west-*.jpg''.
>    (2) Your software tells the server ``CWD pics'' and ``NLST''.
>    (3) The server tells your software the filenames.
>    (4) Your software extracts the filenames matching mae-west-*.jpg.
>    (5) Your software retrieves those files, one by one.
> 
> What's the problem?
> 
> If that isn't how your client software works, your software is broken.
> You are relying on features that aren't guaranteed by the protocol and
> aren't provided by all servers. Fix your software.

This is not a question of what Kermit does, it is a question of what
BSD FTP, Netscape, and IE do since they work in the manner that Frank
is describing.  Granted at the moment they clients do not support
MLSD, but when kiddies get around to implementing MLSD they are going
to try to do exactly the things that Frank describes.  And when they 
do they are going to do it because many of the servers that are out
there today allow them to misuse the protocol in the ways in which
Frank has described.

---

On a side note, I'm a bit embarrassed by the behavior of this working
group.  Kermit has been described by some as the second most portable
C language application after "Hello World".  This portability and its
longevity have resulted in Frank gaining a unique experience in
platform capabilities, protocol design, and interoperability.  To hear
his comments being rebutted with arguments that boil down to "you
don't know what you are talking about" or "you can't have things both
ways (large and small)" or "its preferable to choose solutions that
create N x M situations" is more than a little sad.  

In many of the response I hear two unspoken assumptions:

 .  clients are no longer small nor dumb
 .  operating systems that do things differently from Unix / Windows
    will not be created or do not exist

Neither one of these assumptions is true.  What is true is that
clients are becoming even more diverse today than they ever have been.
They are both smaller and larger.  They use both Unix / Windows style
file systems and various object based storage systems.  To assume that
simply because PC and laptop clients are big enough to handle large
amounts of client side processing that it is ok to push processing out
to all clients is wrong.  It is also wrong to assume that bandwidth to
the endpoints can handle large quantities of raw unneeded data.  PDAs,
mobile devices, and next generation micro devices are just not going
to have these capbilities.

Someone asked "why IBM would use FTP as the front end for a transaction
system?"  They use it because in a large number of systems a
transaction is a file.  This is true is medical applications,
financial applications, and military applications.  Why are they using
FTP?  Because they need to move "files" between heterogeneous systems
in a secure manner over public networks; AND they cannot control what
the client device or operating system will be; therefore they must use
an open widely implemented standard protocol.  To say that FTP should
not be used for this class of application is absurd.  

I realize that FTP is no longer considered a cool protocol.  But it is
one of the most widely implemented protocols in the world and those
involved in supporting and extending it must keep the bigger picture
in mind.

 Jeffrey Altman * Sr.Software Designer     Kermit 95 2.0 GUI available now!!!
 The Kermit Project @ Columbia University  SSH, Secure Telnet, Secure FTP, HTTP
 http://www.kermit-project.org/            Secured with MIT Kerberos, SRP, and 
 [email-masked]               OpenSSL.

---

From: "D. J. Bernstein" <[email-masked]>
Date: Mon, 14 Oct 2002 23:58:59 -0500

Jeffrey Altman writes:
> when kiddies get around to implementing MLSD

You are continuing to miss the point. The clients you're talking about
are broken. They are handling wildcards in a way that

   * is not guaranteed to work by the FTP protocol and
   * _does not work_ with some deployed protocol-compliant servers.

Those clients should be fixed. I've explained the correct way to handle
wildcards. The correct way works with _all_ FTP servers. One example of
a working client is ftpfs.

> Netscape

I'm curious: Since when did Netscape have multiple-file FTP retrieval?

> It is also wrong to assume that bandwidth to
> the endpoints can handle large quantities of raw unneeded data.

You are talking about a ridiculously minor bandwidth issue. We could
save more of the Internet's bandwidth by removing the ``P'' on the first
line of every HTTP connection. Don't you have any real work to do?

> To say that FTP should not be used for this class of application is absurd.  

FTP should not be used for this class of application. The protocol does
not provide disk commitment, for example, so it can't be used for
reliable transaction processing.

If you aren't using FTP, but rather a private extension to FTP with some
transaction features added, you're an incompetent protocol designer.
Starting from FTP as the base for a new protocol is like starting from
OS/360 as the base for a new operating system.

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago