Subject: Re: New draft-draft of sftp...

From: Jeffrey Altman <[email-masked]>
Date: Tue, 26 Nov 2002 11:31:13 EST

Darren Moffat wrote:
>
> On Tue, 26 Nov 2002, Jeffrey Altman wrote:
> 
> > Basically the end result is that for each file system on the host the
> > client is going to need to know whether or not the file system is
> > case-sensitive or not.  Plus the client needs to know which characters
> > are treated as special in the file system.  And which are illegal.
> 
> Which makes an assumption that there are sufficient APIs for userspace
> applications to enquire on this.   I know of no such APIs on any system.
> But I could probably tell you off the top of my head the behaviour of
> a few different filesystems.   Encoding this knowlege into an application
> is a bad idea.  Writting a standard which encourages this thus a bad idea.

Exactly.  Which goes directly to reason why globbing must be done on
the system on which the file system is mounted.  It is only on that
system that the appropriate matches can be done by the file system.
Trying to guess what the file system is capable or not capable of will
only produce inaccurate results.

 Jeffrey Altman * Volunteer Developer      Kermit 95 2.1 GUI available now!!!
 The Kermit Project @ Columbia University  SSH, Secure Telnet, Secure FTP, HTTP
 http://www.kermit-project.org/            Secured with MIT Kerberos, SRP, and 
 [email-masked]               OpenSSL.

---

From: Simon Tatham <[email-masked]>
Date: Tue, 26 Nov 2002 17:06:33 +0000

> only produce inaccurate results.

But this is only one side of the story. The other side is the user
expectations, and the security angle.

SCP is a good example of a protocol that does server-side globbing,
and it's one most people on this list will have heard of :-) So
suppose I'm an SCP user and I type a command such as

  scp myserver:[abc]*.* .

to retrieve some files matching a wildcard, and copy them into my
current directory. The server will now send back a list of file
names and file contents, and my client program will create some
files locally.

What's to stop the server sending back a filename which _doesn't_
match the wildcard? Suppose it sends back `/etc/passwd'. Well,
obviously my client can exercise minimal common sense and outlaw any
inbound filename which contains a slash. But if it sends back some
filename which _wasn't_ within the space of possible names I was
expecting, then it can overwrite some local file I didn't want
overwritten. And the client, knowing nothing about the globbing
rules on the server, would have no way to check whether
PASSWORDS.DAT really _did_ match the wildcard I'd specified. (The
example above is carefully chosen; to a Unix user it seems obvious
that PASSWORDS.DAT doesn't match it, but to a VMS user it seems just
as obvious that it does!)

Now suppose I'm talking to a Unix server, so I'm _expecting_ to get
back files beginning with a or b; but suppose my server has been
taken over by a malicious attacker. He can hack the SCP server
process so that it sends back non-matching filenames, and there's
nothing my client software can do to prevent it. So if I want to
protect myself against this type of attack, then I must _always_ do
a wildcard SCP download into an empty directory, so that there are
no existing files that can be accidentally or maliciously
overwritten by the server. I'm fairly sure this is not what most SCP
users do!

There are certainly undeniable reasons why client-side globbing
causes problems. Unfortunately, I consider this to be an undeniable
reason why server-side globbing can _also_ cause problems. Where
does that leave us? On the one hand, we have potentially inaccurate
results; on the other, we have a potential security hazard (although
as far as I know I'm the only SCP implementor who considers it
remotely important). Is there any middle way possible?

Cheers,
Simon
-- 
Simon Tatham         "_shin_, n. An ingenious device for
<[email-masked]>    finding tables and chairs in the dark."

---

From: "Joseph Galbraith" <[email-masked]>
Date: Tue, 26 Nov 2002 15:38:30 -0700

> [snip]
> to retrieve some files matching a wildcard
>
> [snip]
>

Well, I liked my solution :-)

Seriously, though, as your example pointed out,
a VMS user and a unix user are expecting very
different things in terms of how filenames
are globbed.

And as a user, I would be very surprised if globbing
changed dramatically depending on what server
I'm connecting to -- I personally think that what
most users expect is that the globbing rules that
they are used to (the ones of the client OS) will
generaly be applied.

Between this and the potential security issues being
discussed about server side globbing, and the fact
that sftp is written as a file sharing protocol
(like NFS) as opposed to a file transfer protocol
and I definitely believe that globbing belongs in
the client, not the server.

So really, to me, the question is what information
does the client need to get this right.  I haven't had
the opportunity to review the FTP thread about
this topic yet, but that is probably the next step.

- Joseph

---

From: Pontus Skoeld <[email-masked]>
Date: 27 Nov 2002 08:42:11 +0100

"Joseph Galbraith" <[email-masked]> writes:

> Seriously, though, as your example pointed out, a VMS user and a
> unix user are expecting very different things in terms of how
> filenames are globbed.

This is an extremely important point. IMO globbing behaviour shall be
consistent throughout the client, independent of what server it is
connected to. A user that is used to file names being case-sensitive
given the following file list

alittlefile
AnotherFile

probably refers to alittlefile by a*, whereas someone used to
case-insensitive name mean both files. This is a user property (and
should probably be encoded in client behaviour), not a server
property.

Furthermore, I think this is really a non-problem. The server supplies
a list of file names that can be used for globbing. There is no
problem with doing case-sensitive (or case-insensitive) globbing with
those names (the names returned by the server are valid, independent
of how the user the user selects those file names).

Neither are there any new problems introduced by the server above
being able to refer to AnotherFile by the names anotherfile. A server
that does case-sensitive matching might have both AnotherFile and
anotherfile but not return anotherfile in the answers to readdir.

To rephase this; to check for the existence of a file, use
SSH_FXP_STAT, to avoid overwriting it, use SSH_FXF_EXCL. Using the
answers from readdir for this is *wrong*.

I have not yet had time to review the FTP thread, but I'll try to get
around to that ASAP, but I'm not certain how much of that
argumentation will be valid for sftp.

	/Pontus
-- 

Pontus Sköld, see <URL:http://soua.net/> for more information.