Subject: Ftp-WG: HOST command and client upgrades

From: "Jan Mikkelsen" <[email-masked]>
Date: Wed, 20 Oct 1999 19:44:27 -0400

D. J. Bernstein writes:
>I'm looking forward to seeing your strenuous objections to the HOST
>command proposed here.

Well, as it happens, I do have some problems with the HOST specification.

If a server has a dedicated IP address for each virtual host it serves or if
usernames on all the hosts are allocated from the same namespace, the host
command is not necessary.  Clearly the server can decide which virtual host
to server based on either the IP address, the username or a combination of
the two.

So, the host command is only necessary when multiple virtual hosts are being
served on a single IP address with separate username namespaces for each
virtual host.  This is the important case.

In this case the client MUST tell the server which virtual host is being
used in order to be able to even log in.  As it happens, I run a server with
exactly this problem.  I support the host command, but I haven't seen it
from many clients.

The approach I take is to have users enter the virtual host name they want
as part of the username, in effect making all usernames part of the same
namespace by using a virtual host name to disambiguate them.  Of course, if
the client does send the host command, the user name can be send as per the
spec.

This doesn't conform to the current draft, but, given the requirement of
supporting existing clients it's the best I've come up with so far.

The HTTP/1.1 spec requires servers to reject any HTTP/1.1 request which does
not contain a host header, in order to force clients which claim to support
HTTP/1.1 to also support the host header.  This was done to get widespread
support for the new protocol feature (and to conserve IP addresses) without
breaking old servers or old clients.  (Incidentally, allowing a gradual
migration).

Today it is quite feasible to run a server which depends totally on the host
header to determine which virtual host to serve.

So:  Should the draft be changed?  If a client sends "FEAT" should the spec
mandate that the login be rejected if there was no preceeding "HOST"?

This would force clients and servers which claim conformance with whatever
RFC number the draft turns into to support the HOST command.  The benefit I
see is that in some number of years, FTP servers can cleanly support virtual
hosting, and it would be acceptable to tell clients which break at that time
to upgrade.

The other side is that FTP clients don't seem to be upgraded with quite the
same rapidity as browsers.

My feeling is that host support should be required.

Jan Mikkelsen

---

From: Gregory A Lundberg <[email-masked]>
Date: Wed, 20 Oct 1999 21:21:53 -0400

On Wed, Oct 20, 1999 at 07:44:27PM -0400, Jan Mikkelsen wrote:

> So:  Should the draft be changed?  If a client sends "FEAT" should the
> spec mandate that the login be rejected if there was no preceeding
> "HOST"?

> My feeling is that host support should be required.

I tend agree, but I would wish for something other than FEAT as the
touchstone.  The reason is simply that FEAT is already out as an RFC and
HOST is not.  I'd rather see something new, but cannot think what that new
command or command parameter would be.

The concept that ftp clients don't update as quickly as web browsers is
valid except for one fact: most FTP traffic these days (even from FTP
site administrators) appears to give one of two general forms of password
for anonymous access: IE?0user@ or mozilla@.

The truth is FTP clients are virtually irrelevant in terms of hits or
traffic volume.  A requirement similar to the HTTP/1.1 spec requirement
would force web browsers to update, and that's good enough from what I can
see.

If anyone wants to debate my thesis here, I'd be glad to run an analysis of
the first N hours of downloads following the release of wu-ftpd 2.6.0.  I
think it will give a good cross-section analysis of what FTP
*administrators* are using.

-- 

Gregory A Lundberg              WU-FTPD Development Group
1441 Elmdale Drive              [email-masked]
Kettering, OH 45409-1615 USA    1-800-809-2195

---

From: "Jan Mikkelsen" <[email-masked]>
Date: Wed, 20 Oct 1999 22:00:20 -0400

> command or command parameter would be.

That seems reasonable, but the problem is that everything else happens after
a login.  Another option could be to have commands like MLST or MDTM fail
unless a host command was sent.  Without the client claiming to support a
particular version, either a feature has to be picked from the appropriate
version to use as an indicator, or a guess needs to be made.  And we've
established that guessing isn't a good thing. :-)

> would force web browsers to update, and that's good enough from what I can

Another major use of FTP which I see is users updating their web pages for
others to later download using HTTP.  Anonymous FTP may not even be in use.
FTP traffic profiles at webhosting providers are quite different to that of
the average anonymous FTP site.

The tools used to update websites are often embedded in some editing
application, or a standalone FTP client is used.  The browser components
which do the updating (ie: Netscape Composer and Microsoft FrontPage Express
with the Web Publishing Wizard) are used, but aren't nearly as dominant as
the browsers themselves.

Jan Mikkelsen

---

From: "Jan Mikkelsen" <[email-masked]>
Date: Wed, 20 Oct 1999 22:09:10 -0400

> command or command parameter would be.

Another thought just occurred to me (30 seconds ago, so it might have a few
rough edges):  A version command, which enables particular features.

In order for a client to use HOST, MLST, whatever new feature, it must send
(say): "VER RFC-nnnn" or "VER FTP-n".  Of course there is an overlap (to an
extent) with FEAT, but it might still be a useful way to handle something
like the HOST problem.

A server, when seeing an attempt to use a "new" feature from a client which
hasn't indicated its support, could return "Support not indicated" or
similar.  If a client does indicate its support for a particular feature
level, but doesn't send a host command, a failure could also be indicated.

Jan Mikkelsen

---

From: Robert Elz <[email-masked]>
Date: Wed, 20 Oct 1999 23:16:48 -0400

Date:        Wed, 20 Oct 1999 22:09:10 -0400
    From:        "Jan Mikkelsen" <[email-masked]>
    Message-ID:  <[email-masked]>

  | In order for a client to use HOST, MLST, whatever new feature, it must send
  | (say): "VER RFC-nnnn" or "VER FTP-n".

I don't think that is needed.   There's not a lot of point the client
asking the server before it uses a new command - if the server is old,
it won't understand the "can I use" command, and will reply with an error.
If the server is new, bith commands will work.   If the client just
goes ahead and uses the command it wanted to use in the first place, and
the server doesn't understand it, it will reply with an error - no harm
done.   If the server does understand, it just sends the data the client
wanted (or performs the action, or whatever), and everyone is happy.

FEAT is useful, as it allows the client to discover in one RTT everything
the server can handle.  There's not a lot of point telling the server
a lot about the client, as all it ever does is react to the client's
instructions.   I guess a one command "I can do all of X, Y Z..." could
allow a server to optimise its responses to other commands to suit the
client (perhaps sending that other listing format instead of ls -l in
response to a LIST command - once it knows the client supports it) but
simply having a new command for each new feature (along with the ability
for the client to work out whether or not to use the new command) seems
adequate to me.

wrt HOST - Dan is quite right, it will be basically useless until
clients support it.   That's because of the nature of what is desired
to be achieved, not because of anything specific about the details of
the HOST command.   Only the client knows the string the user entered
which the client then turned into the IP address used to contact the
server.   Without some update to the client, to send that information,
one way or another, virtual FTP hosts cannot happen.

It may be that with the number of ftp client implementations already
in the field, and perhaps never to be updated, adding virtual FTP hosts
may simply be impossible now, however desireable it might be.   It was
possible with HTTP, given that there were only a few browsers around,
and essentially no other tools at the time, and the browsers were still
in such a state of flux that almost everyone updated them monthly.
None of that is true with FTP.

If we are going to continue to discuss HOST, I'd suggest that we decide
to move it out of the MLST draft (and let that get smaller), and make
it a draft of its own, that we can then decide what to do with.  It is
certainly the least mature part of what is currently in MLST.  If we
do that, and someone else would prefer to be the editor of the HOST draft,
that's fine with me...

kre

---

From: Alun Jones <[email-masked]>
Date: Thu, 21 Oct 1999 11:01:32 -0400

>do that, and someone else would prefer to be the editor of the HOST draft,

HOST is so disconnected from the concept of MLST/MLSD, I think it should be
detached from the draft - as you've pointed out, it still has some serious
conceptual flaws to overcome, not the least of which is the estimated
timetable of real-world adoption.  Anyone who's been on here long enough
knows that I've been looking for something that solves the problem that
HOST is aimed at - that of running more than one host on a single IP
address, "just like web servers", as one of my customers put it.  But I
don't want to see issues with HOST holding back the implementation of
MLST/MLSD.

Alun.
~~~~

--
Texas Imperial Software | Try WFTPD, the Windows FTP Server. Find it
1602 Harvest Moon Place | at web site http://www.wftpd.com or email
Cedar Park TX 78613     | us at [email-masked].  VISA / MC accepted.
Fax +1 (512) 378 3246   | NT based ISPs, be sure to read details of
Phone +1 (512) 378 3246 | WFTPD Pro, NT service version - $100.
*WFTPD and WFTPD Pro now available as native Alpha versions for NT*

---

From: Gregory A Lundberg <[email-masked]>
Date: Thu, 21 Oct 1999 11:13:45 -0400

> don't want to see issues with HOST holding back the implementation of

The other commands (MDTM and SIZE) and Stream mode REST should also be
broken out.

Granted MDTM and SIZE have some bearing on MLST.

To me, thhough, all three of these are in the draft more to codify existing
practice than to add new features.

-- 

Gregory A Lundberg              Senior Partner, VRnet Company
1441 Elmdale Drive              [email-masked]
Kettering, OH 45409-1615 USA    1-800-809-2195

---

From: Alun Jones <[email-masked]>
Date: Thu, 21 Oct 1999 11:42:57 -0400

>practice than to add new features.

It would have been nice for these commands to be documented in an RFC
before now - however, I don't think removing them now would speed their
publication significantly.  I do think that leaving HOST in the draft will
delay publication significantly, since it obviously has issues.  Sorry if
anyone got the impression that a conceptual break was the _only_ reason I
have for agreeing that HOST should be broken out.

MDTM, SIZE and REST have been in use for many years, and it would certainly
be nice to be able to point to an RFC, rather than a draft that effectively
says "may change at any time", to new developers.  It's my belief, however,
that we can fix the few remaining quibbles about MLST / MLSD, to get this
thing finalised and published in a timely manner, perhaps faster than we
could edit, submit and review a new pair of drafts.

HOST was added relatively late to the MLST draft, and that was not my
intent when I put my voice behind implementing it - I think it deserves its
own draft and separate consideration, and should neither delay MLST, nor be
rushed through without decent discussion merely to speed MLST's publication
along.

Alun.
~~~~