Subject: Ftp-WG: MLST implementations?

From: "D. J. Bernstein" <[email-masked]>
Date: Fri, 8 Oct 1999 22:39:25 -0400

Alun Jones writes:
> [but then, MLST also replicates much of what EPLF was trying to do,

It's true that the MLST format is a poor imitation of the EPLF format.
But MLST is missing the fundamental goal of EPLF: namely, to _simplify_
future FTP implementations.

> and merely achieved popularity over EPLF.]

EPLF is supported by Netscape 5.0, Lynx 2.6, Voyager 3, Mirror 2.9,
EMirror 2.1, gFTP 2.0.0, Squid 2.3, and several experimental clients.
How many clients can parse, let alone use, MLST?

The advantage of MLST over EPLF was supposed to be immediate adoption by
today's servers---never mind the corresponding disadvantage for clients.
Okay: How many servers understand MLST?

---Dan

---

From: Alun Jones <[email-masked]>
Date: Mon, 11 Oct 1999 10:15:16 -0400

>future FTP implementations.

No, Dan, MLST is not a "poor imitation" of anything.  It does the same job,
yes, but then so many other people noticed a need for that job to be done.
You're not the only one who's had this idea.  I'd suggest that Peter Lewis'
description of how to produce and parse listings for FTP servers probably
is the earliest attempt to deliver a standard method to display and read
listings across different platforms; it predates EPLF and has the distinct
advantage of also being compatible with the majority of old clients _and_
servers without changing anything.

>How many clients can parse, let alone use, MLST?

Whoop-de-doo.  In case the [clipped] original context of that statement was
unclear, MLST achieved popularity among the working group, and EPLF was
dismissed in large part because it causes an EPLF-compliant server's list
output to be rejected by a non-compliant client.  It was suggested that
perhaps EPLF output could be generated by a separate command, but you
declined that proposal; it was suggested that EPLF capabilities could be
negotiated by compliant clients, but you rejected that idea as well.

In the attitude of "all or nothing" you projected, it was considered by me,
and presumably those others that are putting their weight behind MLST's
development, to choose "nothing" and develop a solution that better fitted
our goals of compatibility with older clients and servers.

>Okay: How many servers understand MLST?

I don't know - your list also leaves out how many servers generate EPLF, of
course.  My server is certainly being tested as an MLST implementation, but
while the spec is still under discussion, it would be really risky to
release an implementation that might be non-compliant when the spec is
finalised.  Of course, this is far less risky than releasing a server that
generates a listing output that is not recognised by older clients
currently in use.

Dan, I _liked_ EPLF.  I thought it was pretty close to something that might
be implementable.  However, you refused point-blank to make any of the
changes suggested by members of this working group.  You should therefore
be unsurprised that members of this working group do not feel inclined to
support your format.  Please don't complain about things that you were not
willing to help address.

Alun.
~~~~

--
Texas Imperial Software | Try WFTPD, the Windows FTP Server. Find it
1602 Harvest Moon Place | at web site http://www.wftpd.com or email
Cedar Park TX 78613     | us at [email-masked].  VISA / MC accepted.
Fax +1 (512) 378 3246   | NT based ISPs, be sure to read details of
Phone +1 (512) 378 3246 | WFTPD Pro, NT service version - $100.
*WFTPD and WFTPD Pro now available as native Alpha versions for NT*

---

From: "D. J. Bernstein" <[email-masked]>
Date: Sun, 17 Oct 1999 00:07:46 -0400

Alun Jones writes:
> It was suggested that
> perhaps EPLF output could be generated by a separate command,

For new people here who haven't seen previous EPLF discussions, I'll
briefly explain why Jones's suggestion is counterproductive.

The fundamental goal of EPLF is to simplify future FTP implementations.
Imagine a world where all file listings use one very simple format, such
as EPLF. Servers won't have to generate ls-style dates. Clients won't
have to bother parsing a jumble of formats.

How do we move from today's situation to that pleasant future world?
EPLF offers a straightforward three-step transition plan: first, teach
clients how to parse EPLF; second, switch servers to EPLF; third, don't
bother with other formats in new clients.

Can we save some of this work by using EPLF with a new listing command?
No. We have to upgrade the clients, no matter what command is used, so
that servers don't have to bother printing the complicated old format. 
Similarly, we have to ugprade the servers for the sake of the clients.
The only way to avoid these upgrades is to ignore the problem! A new
listing command simply means extra implementation difficulty.

Jones says that MLST is optional and doesn't require anyone to upgrade.
But then what's it supposed to accomplish? How does MLST simplify future
implementations? The unfortunate answer is that it doesn't. MLST is
pointless complexity.

---Dan

---

From: Russ Allbery <[email-masked]>
Date: Sun, 17 Oct 1999 01:31:51 -0400

> Similarly, we have to ugprade the servers for the sake of the clients.

If you implement EPLF as a new listing command, you can upgrade the
servers after only some clients have been upgraded without waiting for
most of them, since older clients will continue to work.  If you don't use
a new command, unfortunately you can't do this (every client that you care
about being able to parse directory information has to be upgraded), which
means that fewer EPLF servers will be deployed, which means that client
writers will see less demand for supporting it and won't implement it.
This creates a spiral that locks EPLF out of the running.

I really do think that a new command is the right way of going with this.
I think it's somewhat unfortunate that due to reasons that seem mostly
non-technical (and I'm not blaming either side, nor do I have the
background to since I wasn't here for previous discussions) a solution
which is already deployed and parseable by at least a few clients isn't
the one going into the standard.

If implementing EPLF as a separate command (and advertising it as a
capability to avoid needless attempts by the client to send a command the
server doesn't support) would help resolve this, I encourage you to
consider it again.

It will mean a longer transition period before EPLF can simplify future
implementations, since it would have to wait until clients have pretty
much stopped using the old, non-standardized list command, but I think it
would then have a good chance of accomplishing the goal eventually,
whereas I think the current approach probably won't.

-- 
Russ Allbery ([email-masked])         <URL:http://www.eyrie.org/~eagle/>

---

From: "D. J. Bernstein" <[email-masked]>
Date: Sun, 17 Oct 1999 05:14:15 -0400

Russ Allbery writes:
> most of them,

That's wasted effort. Compare the work necessary to achieve the goal of
simpler implementations:

   EPLF transition plan               What you're talking about
   --------------------               -------------------------
   deploy clients that parse EPLF     deploy clients that try ELST
                                      deploy servers that accept ELST
   simplify servers                   simplify servers
   simplify clients                   simplify clients

Four deployments are more work than three. Supporting two listing
commands is more difficult for BSD-type servers than supporting one.
Sending a possibly unsupported command is more work for clients than
merely parsing EPLF. I understand the attraction of performing two
deployments in parallel, but it doesn't justify this extra effort.

> This creates a spiral that locks EPLF out of the running.

Then how do you explain Netscape 5.0, Voyager 3, Lynx 2.6, Mirror 2.9,
EMirror 2.1, gFTP 2.0.0, Squid 2.3, etc.?

Apparently FTP implementors are smarter than you think. The MLST
approach might be attractive to commercial vendors who want FTP to be
more difficult to implement, but those vendors don't control the net.

---Dan

---

From: Russ Allbery <[email-masked]>
Date: Sun, 17 Oct 1999 22:34:12 -0400

> commands is more difficult for BSD-type servers than supporting one.

Both of these are true.  However, vendors shipping stock ftpds, or major
ftpd implementations like wuftpd, aren't going to switch to EPLF until
basically *everything* supports it, because backward compatibility is one
of their highest goals.  They may add it as an optional feature, but that
goes straight back to the added complexity that you're talking about and
adds even more complexity for the installer if it's an either/or decision.

There are a lot of ftp clients out there, like vendor ftp for many
commercial Unixes and clients like Fetch, that are unlikely to get *any*
new features.  Those of us who run ftp servers are stuck with supporting
them, unfortunately.  There's no way I could deploy an ftp server here
that broke Fetch, even if we started distributing another Mac ftp client;
the installed base around here is huge.

In an ideal world, I agree with your argument, but I'm afraid that given
the backward compatibility requirements, trying to do this in three phases
instead of four may mean that some minor progress will be made on the
first phase and it will then stall.

> EMirror 2.1, gFTP 2.0.0, Squid 2.3, etc.?

The fact that it's a good protocol, and a lot of people will add things
that are well-done just because they're well-done, regardless of how
widely deployed they are.  I'm not arguing that *no one* will put EPLF
support into clients.  I'm saying that I think it's unlikely you'll get
the 90-95% penetration necessary for people to start upgrading their
servers.

---

From: Alun Jones <[email-masked]>
Date: Mon, 18 Oct 1999 08:55:11 -0400

>> commands is more difficult for BSD-type servers than supporting one.
...
>the 90-95% penetration necessary for people to start upgrading their

Since both sides in this discussion seem to be contributing the same
arguments that were originally offered in this mailing list, can we just
point new readers (and posters) to the archives at
http://w3.hethmon.com/ftpext/ and get on with some of the more current
issues?  If someone has anything _new_ to say, then by all means, please
say it, but unless one side feels they have something to say that has not
yet been heard by the other, then we're going to remain stuck at a point
where the majority of the vocal portion of this working group will not
implement EPLF in their servers without requiring either a negotiable
feature setting, or a new command, and where Mr Bernstein will not approve
the use of EPLF in such a setting.

Thanks,

Alun.
~~~~

---

From: "D. J. Bernstein" <[email-masked]>
Date: Tue, 19 Oct 1999 10:34:33 -0400

Russ Allbery writes:
> There's no way I could deploy an ftp server here that broke Fetch,

Nobody's asking you to. _First_ add EPLF support to Fetch. Then, _after_
that's deployed, you can upgrade your servers.

  [ BSD ftpd and wuftpd, the most popular FTP servers ]
> They may add it as an optional feature, but that
> goes straight back to the added complexity that you're talking about

The optional feature already exists! Both of those FTP servers use an
external ~ftp/bin/ls that the sysadmin can easily replace. (In contrast,
MLST requires changing the FTP server itself.)

I agree that the old format will stick around as long as most clients
can't understand anything else. This is why EPLF was designed to make
universal client support as easy as possible.

---Dan

---

From: Robert Elz <[email-masked]>
Date: Tue, 19 Oct 1999 15:49:53 -0400

Date:        Tue, 19 Oct 1999 10:34:33 -0400
    From:        "D. J. Bernstein" <[email-masked]>
    Message-ID:  <[email-masked]>

  | Nobody's asking you to. _First_ add EPLF support to Fetch. Then, _after_
  | that's deployed, you can upgrade your servers.

The worst part about EPLF (as it is defined) is that clients have to
guess whether it is supported or not - just the way thay have to guess
now what format the LIST output appears in.   Guessing just isn't reliable.
I suspect that most people here had encountered situations where a "smart"
FTP client attempts to guess the format, and doesn't get it quite right.

  | The optional feature already exists! Both of those FTP servers use an
  | external ~ftp/bin/ls that the sysadmin can easily replace.

I don't know about wuftpd, I suspect in that the path to the ls command
used can be configured, but for BSD ftpd that is not true.   Original BSD
ftpd uses "/bin/ls" - that happens to be ~ftp/bin/ls after the chroot(~ftp)
that is done in anon ftp, but it isn't any other time.   Replacing /bin/ls
and installing something that produces EPLF is not reasonable...

  | I agree that the old format will stick around as long as most clients
  | can't understand anything else.

That means forever.   Changing all of the clients just is not going to
happen.  Changing 95% might not ever happen.

  | This is why EPLF was designed to make
  | universal client support as easy as possible.

For the client, MLST is easier, as with MLST it *knows* when support is
present, and hence doesn't have to guess any more (falling back to
guessing only when MLST isn't available).  For EPLF guessing will remain
forever - and quite peroably need to deal with EPLF lookalikes where
someone just decides that they can "improve" things a bit (if one person
can simply define what EPLF is, any other person can similarly define a
slightly different, hard to distinguish, improved version).

But as Alun said, all this has been discussed before.

kre

---

From: "D. J. Bernstein" <[email-masked]>
Date: Wed, 20 Oct 1999 10:58:27 -0400

> > This is why EPLF was designed to make
> For the client, MLST is easier,

Then where are all the clients using MLST?

> guess whether it is supported or not

An EPLF line begins with +. This marker was carefully selected to avoid
conflicts with existing LIST formats, and EPLF is widely documented to
avoid conflicts with future protocol extensions.

Yes, it's possible for a rogue server to deliberately use this marker in
a way that shows ``wrong'' file listings to everyone using Netscape 5.0,
the popular Squid proxy, etc. So what?

It's also possible for a rogue server to respond in MLST in a way that
shows ``wrong'' file listings to your nonexistent MLST-using clients, or
to respond to LIST with ``wrong'' file listings in /bin/ls format.

---Dan

---

From: Alun Jones <[email-masked]>
Date: Wed, 20 Oct 1999 11:23:43 -0400

>Then where are all the clients using MLST?

Dan, you make such bogus arguments that it is often difficult for even
those of us that try to acknowledge your contributions to do so without
feeling stupid.

MLST is an IETF draft specification.  As such, it is subject to change, up
until such time as it is frozen into an RFC - which I personally hope will
come soon.  Since the standard is not yet set, it would be mind-bogglingly
stupid for someone to publish a client and claim compliance with a standard
that might change tomorrow and make their software non-compliant.  Hence,
you'll find that MLST implementations are making the rounds only between
MLST implementors, and are not released to the public.

EPLF, by contrast, is a specification that has been fixed, and has remained
fixed despite numerous requests to make changes that would make it more
palatable and even allow it to be implemented as an RFC with widespread
approval.  It costs nothing more than an hour or two's development time for
a client to support it.  I might ask you where are the _servers_,
independent of your own, that currently provide EPLF as their default
output format?  Yes, that's an unfair question, but it's far more fair than
your question asking about published clients for an unpublished standard.

I will repeat, EPLF is a good suggestion in much of its nature.  I don't
think that it bears implementing in a server until such time as I don't
care about customers with clients that won't be able to interpret EPLF.
Regardless of what you have said about EPLF's current penetration, I think
such a time is far into the future, if it will even arrive during the
lifetime of my software.  In the absence of any way for an EPLF-compatible
client to notify my server, there is no reason for me to even start
implementing EPLF.

There is, however, reason for me to work on implementing MLST, since that
can be supported and implemented immediately on the RFC being published
without causing anything in the nature of interruptions to my current
customers.

Alun.
~~~~

---

From: "D. J. Bernstein" <[email-masked]>
Date: Fri, 29 Oct 1999 22:29:37 -0400

Russ Allbery writes:
> new features.

I hear that Fetch 3.5 will support EPLF.

Given the big jump in version number, I'd guess that there will also be
some new user-interface features that your users will be interested in.

Welcome to the real world.

---Dan

---

From: Russ Allbery <[email-masked]>
Date: Sat, 30 Oct 1999 00:03:33 -0400

>> commercial Unixes and clients like Fetch, that are unlikely to get
>> *any* new features.

> some new user-interface features that your users will be interested in.

If someone's actually actively developing Fetch again, I'm *very*
interested.  Kerberos support only took five years or so.  :)  Thanks for
letting me know; that's very good news.