Subject: Re: A future for the SSH File Transfer Protocol?

From: Jeffrey Altman <[email-masked]>
Date: Thu, 8 Nov 2001 10:51:55 EST

Why should a client ever have to understand a system dependent alias
such as ~fred?

The only entity that should need to know how to map ~fred to what it
points to is the sender of the file.

> 
> I might be the only one, however; nobody has shown much interest in
> answering my clarifying questions or in the extension I proposed (to
> allow user details to be looked up, so a client could understand
> `~fred' and also so that the uid->username mapping in the
> recommended form of the long listing could be done in other contexts
> as well).
> 
> Cheers,
> Simon
> -- 
> Simon Tatham         "What a caterpillar calls the end of the
> <[email-masked]>    world, a human calls a butterfly."
> 

 Jeffrey Altman * Sr.Software Designer      C-Kermit 8.0 Beta available
 The Kermit Project @ Columbia University   includes Secure Telnet and FTP
 http://www.kermit-project.org/             using Kerberos, SRP, and 
 [email-masked]          OpenSSL.  SSH soon to follow.

---

From: Simon Tatham <[email-masked]>
Date: Thu, 08 Nov 2001 16:02:43 +0000

> points to is the sender of the file.

Ah; you think it would be better for servers to understand `~fred'
at the start of a pathname?

Fair enough, I suppose. But the rest of the ethos of SFTP seems to
be that whatever alias or synonym you use to refer to a directory,
you can always call FXP_REALPATH and get the full canonical name. So
in my command-line SFTP client, I don't want to type `cd ~fred' and
then have `pwd' just tell me `~fred'; I'd rather get the same
behaviour I get at the real shell prompt, where I type `cd ~fred'
and `pwd' tells me `/home/fred'.

I suppose that could be done just by making the server understand
`~fred' at the start of a pathname, and then having the client call
FXP_REALPATH("~fred") if it wanted the canonical form; but that
starts to eat into the available characters in filenames, which is
something SFTP has so far been scrupulous in not doing. So instead,
my proposal was to introduce a protocol extension which allowed the
client side to request the home directory of a particular user.

While I was at it, I also added name<->number mappings for users and
groups, on the grounds that it seems silly that a command-line
client can list the textual usernames of file owners by printing out
the long name field, but a GUI client wanting to split all the
details into separate columns has to make do with the numbers.

Cheers,
Simon
-- 
Simon Tatham         "A cynic is a person who smells flowers and
<[email-masked]>    immediately looks around for a coffin."

---

From: "Joseph Galbraith" <[email-masked]>
Date: Thu, 8 Nov 2001 09:20:32 -0700

> details into separate columns has to make do with the numbers.

I'm definitely interested in this.  Can you repost your
proposal?

I would actually like to eliminate the long listing
format (on a per file basis, I think it adds several
hundred percent overhead... on a directory with several
hundred files in it, over a slow link, this is a not
insignificant overhead.)

But to do so, we need to provide a way for the client
to map uid/gid to names.

I'd also like to do something a bit more flexible for
uid/gids -- WinNT, which the platform I'm working on,
doesn't use simple ints for user identifiers.

I'd be tempted to suggest that UIDs be defined as

    string identifier

which would be an opaque data structure to the client.

- Joseph

---

From: Simon Tatham <[email-masked]>
Date: Thu, 08 Nov 2001 16:37:23 +0000

"Joseph Galbraith" <[email-masked]> wrote:

> I'm definitely interested in this.  Can you repost your proposal?

Certainly. It's available as a text file from

  http://www.tartarus.org/~simon/ssh/draft-sftp-userdb-00.txt

> I would actually like to eliminate the long listing format (on a per
> file basis, I think it adds several hundred percent overhead... on a
> directory with several hundred files in it, over a slow link, this
> is a not insignificant overhead.)

Hmm, hadn't thought of that. And compression will only cut it down
by so much...

> But to do so, we need to provide a way for the client to map uid/gid
> to names.

... which would be much more bandwidth-efficient because you only
need do the mapping once for each uid mentioned, and in the average
directory listing there are many more files than uids. Nice idea.

(On the other hand, waiting for the file data to be returned and
then looking up the uids will cost you another round trip. If your
link is high-bandwidth but high-latency, this approach might lose!)

> which would be an opaque data structure to the client.

But suitable for showing to the user, presumably? The ghastly WinNT
security IDs wouldn't be something the average user wanted to see
polluting their file listing :-)

Cheers,
Simon

---

From: "Joseph Galbraith" <[email-masked]>
Date: Thu, 8 Nov 2001 10:06:44 -0700

> link is high-bandwidth but high-latency, this approach might lose!)

True -- on the other hand, you needn't wait until
the directory enumeration is complete to begin
issuing requests to resolve UIDs.  

Alternatively, we could automatically include UID
resolution information as part of the NAMES packet,
avoiding the round trip.  Or perhaps there would be
an additional flag to the READDIR command specifying
whether UID resolution information should be automatically
sent.

Perhaps a more general mechanism for the client to
tell the server what information is important
would be appropriate.

I've also considered a mechanism for the server
to communicate what information is cheaply available
and what information is expensive.  When using
NTFS, for example, security information is costly
to look up (looking up owner / group for each file
can slow down the file list significantly.)

> polluting their file listing :-)

True -- they are pretty ugly.

However, the thing that is suitable for
display is the resolved username.  I would
actually suggets that the identifier string
is a binary blob -- it is only suitable for
sending back to the server in order to
manipulate the security of a file.

Most unix servers would probably put
the four byte uid in it.

- Joseph

---

From: Simon Tatham <[email-masked]>
Date: Thu, 08 Nov 2001 17:18:58 +0000

"Joseph Galbraith" <[email-masked]> wrote:

> Or perhaps there would be an additional flag to the READDIR command
> specifying whether UID resolution information should be
> automatically sent.
> 
> Perhaps a more general mechanism for the client to tell the server
> what information is important would be appropriate.

Ooh. Yes, this _does_ sound good; it also solves your other problem
(once you've included the user database information to make the long
name fields unnecessary, how do you arrange that the long name
fields don't get sent anyway and waste all that effort?).

Perhaps another extended data type, parallel to ATTRS but used for
requests rather than responses? It might look something like this:

  uint32   flags
  uint32   extended_count present only if flag
                          SSH_FILEXFER_ATTR_EXTENDED
  string   extended_type
  ...      more extended_type strings
           so that number of strings equals extended_count

So you can set all the same flags that would be set in the returned
ATTRS, and request particular types of extended data as well (to
preserve extensibility).

Except that of course that doesn't _include_ the long name field,
because it's not part of ATTRS. Bah. Time for an unprincipled hack
to the above, or back to the drawing board?

> However, the thing that is suitable for display is the resolved
> username.  I would actually suggets that the identifier string is a
> binary blob -- it is only suitable for sending back to the server in
> order to manipulate the security of a file.

Oh, I see. Fair enough. So if I wanted to assign ownership of a file
to Fred, I'd do a getuserbyname("fred"), and the binary blob that
came back would be suitable for using in FXP_(F)SETSTAT.

Cheers,
Simon
-- 
Simon Tatham         "I thought I'd put my foot so far into my mouth I
<[email-masked]>    wouldn't be able to sit down without standing up."