Subject: [ftpext] draft-peterson-streamlined-ftp-command-extensions

From: Robert McMurray <[email-masked]>
Date: Mon, 22 Nov 2010 23:01:46 +0000

[Note: Renaming the thread from 'Submitting new drafts'.]

> Anthony Bryan wrote:
> 
> I don't mean to single you out, but
> you showed interest in an CSID/AGNT
> command. Do you have any opinion on
> the other 4 commands in the draft?

Thanks, Anthony.

No worries - I meant to send a better response in the wake of my withdrawn AGNT draft and Mark's current Streamlined FTP Command Extensions draft, but I had been remiss at taking the time to collect my thoughts in a more meaningful way. But since you asked, this seems like as good at time as any. ;-]

CSID - As evidenced by my short-lived AGNT proposal, I think it's very useful to have some sort of client/server discovery command. FEAT does a good job of advertising the list of commands that are supported by a server, but Mark's proposed CSID command sequence provides a much better way for servers and client to identify their features to each other in detail. For security reasons I would probably want the CSID exchange to occur AFTER the client has logged in, but I might be overreacting.

RMDA - This could be a very useful command, and it would parallel the functionality that I'm sure we all typically use in a native operating system environment. In a server implementation I would probably make this command configurable, so administrators would have an option to disable the feature. In which case I would probably hide the command from a FEAT response so it would not be discoverable and have the server return a 500 or 502 reply when RMDA is sent by the client and disabled at the server.

(Note: I mention my thoughts about server implementation details just to open up the discussion as to whether any implementation details should be called out in the draft, or if we should leave that up to the implementers to decide.)

AVBL - This command could be very useful in a client/server world; our FTP server implementation currently returns the available size after a successful login sequence, and the available size is based on server disk quotas and such. That being said, not every FTP client will bubble that information up to the user interface. Having an AVBL command would make it possible for a client to determine whether enough storage space is available at the server BEFORE sending a file, which would be great when you're about to send a 10GB file and you only have 9.99GB of storage space left. ;-]
 
DSIZ - This could certainly be useful, although I would probably want a user to be logged in (not anonymous) before I supported this command in order to reduce the risk of denial-of-service. I'd probably make it configurable so as not to tie up too many server resources for large directory structures; and once again, if the feature were disabled I would probably hide the command from a FEAT response so it would not be discoverable and return a 500 or 502 reply when DSIZ is sent by the client and disabled at the server.

THMB - I have to admit, I don't work on a graphical FTP client, so this command is less useful to me personally, but I can see the merit that it might have to someone that designs a graphical client. That being said, I'm not a big fan of computing the thumbnail dynamically, nor am I fond of allowing the client to specify the thumbnail format and size. I think that it would be better if the server could pre-cache thumbnails for any file type in a server-determined size and image format, and then return the binary data and image format information to the client. For example, most graphically-based operating systems have some method for displaying file icons to a user that is seated at a computer terminal, so an FTP server implementation could simply return that same icon to a client in response to a THMB request.

Please bear in mind, I haven't given a lot of thought as to what the client/server interchange might look like if the server were more in control of the response to a THMB request, but it would probably be sufficient to simply return the image type as part of the response and let the client determine the image dimensions based on the client's capabilities for reading the image data. (As a server guy I like to push any client-friendly processing on the client whenever possible. ;-]) You could borrow an idea from the HASH discussion that we've been having and make the thumbnail image type configurable for the client by using an OPTS command, which might look like the following examples.

In this first example, the server has a pre-configured thumbnail image type of JPG, and may have pre-cached the thumbnails for files; any THMB request with no OPTS specified would use the server's pre-configured thumbnail image type of JPG and image type is specified in the command channel response:

  C> THMB widget.png
  S> 150 JPG Starting thumbnail transfer (1234 Bytes) for widget.png
  S> 226 Transfer complete.

In these next examples a client is asking for different thumbnail image types by using OPTS commands, and the thumbnail image types would still be specified in the command channel responses:

  C> OPTS THMB GIF
  S> 200 OPTS THMB GIF command successful.

  C> THMB widget.psp
  S> 150 GIF Starting thumbnail transfer (1234 Bytes) for widget.psp
  S> 226 Transfer complete.

  C> OPTS THMB PNG
  S> 200 OPTS THMB PNG command successful.

  C> THMB widget.doc
  S> 150 PNG Starting thumbnail transfer (1234 Bytes) for widget.doc
  S> 226 Transfer complete.

In this next example a client is asking for an unsupported image type, so the server would simply return the appropriate error reply:

  C> OPTS THMB FOO
  S> 501 Unsupported thumbnail format.

In this next example a client is asking for the currently-configured thumbnail image type, so the server would return the appropriate response:

  C> OPTS THMB
  S> 200 JPG

You might take this one step further - the server might choose to only support its pre-configured thumbnail image type, thereby allowing pre-caching of all thumbnails in one specific image format. In this example a client is asking for PNG thumbnails, which SHOULD be a supported image type, but the server only supports a pre-configured thumbnail image type for JPG files and returns the appropriate error reply (504?) and the server's pre-configured thumbnail image type is specified in the subsequent command channel response to a THMB request:

  C> OPTS THMB PNG
  S> 504 Specifying thumbnail formats is unsupported.

  C> THMB widget.xls
  S> 150 JPG Starting thumbnail transfer (1234 Bytes) for widget.xls
  S> 226 Transfer complete.

As I said earlier, I hadn't given much thought as to what the client/server THMB interchange might look like if you put the server in control instead of the client, but as a server guy I'd rather see THMB look a little more like the examples that I gave. ;-]

Thanks again!

Robert McMurray
[email-masked]

---

From: "Mark P. Peterson" <[email-masked]>
Date: Mon, 22 Nov 2010 18:32:39 -0600

Hi Robert,

Thank you for taking the time to review the commands.  If you don't mind I would
like to respond to your comments:

CSID:
> For security reasons I would probably want the CSID exchange to occur AFTER the client has logged in, but I might be overreacting.

There is a lot of discussion on the Internet about servers providing "too" much information, especially prior to login.  But having 
worked on both sides of the FTP process, it's usually very easy for the client to identify what brand or flavor the server is. 
There are details which usually identify the server such as the completeness of the FEAT response, the welcome message, responses to 
certain messages, etc.  So, I think it's okay to be available prior to login.  I could change the document to read that it should be 
up to the server's implementation whether or not it's available.

RMDA:
> In a server implementation I would probably make this command configurable, so administrators would have an option to disable the 
> feature.  In which case I would probably hide the command from a FEAT response so it would not be discoverable and have the server 
> return a 500 or 502 reply when RMDA is sent by the client and disabled at the server.

I don't think my document specifies that it isn't configurable.  My implementation in Serv-U is configurable as is almost all other 
FTP commands.  I think this can be left up to the server implementation.  Your possible implementation is exactly how Serv-U 
implements it.  If the feature is disabled, it doesn't appear in the FEAT response, nor is it available through the HELP command.

AVBL:
> ...

Your comments address the reason for its existence.  The client-side implementation in SmartFTP uses the command in this manner.

DSIZ:
> This could certainly be useful, although I would probably want a user to be logged in...

Any information about paths or files requires login in FTP.  I am sure this is the case here as well.  If a server wouldn't want the 
"anonymous" user to have access to this command it could be configured to disallow this command at the server's implementation or 
configuration.  This should be left up to the implementation of the server or administrator of the server.

> I'd probably make it configurable so as not to tie up too many server resources for large directory structures;...

Agreed, the time-consuming nature of the implementation is discussed but configuration by an administrator or server-implementation 
is not.   Do you think such verbiage is required?

THMB:
> I'm not a big fan of computing the thumbnail dynamically, nor am I fond of allowing the client to specify the thumbnail format and 
> size.

That's the point of the command, though.  It's to reduce the amount of data that is required to travel over the wire.  The idea is 
to shorten delivery time to the client, letting the client decide what is best for the client, not the server.  Some clients may 
want very small images, say for a Windows Explorer style listing, others may want larger images (640x480 for example) to show a 
slide show.  I think the client needs to make that determination.  If the server decides to cache these generated images it can.

> I think that it would be better if the server could pre-cache thumbnails for any file type in a server-determined size and image 
> format, and then return the binary data and image format information to the client.

This could waste server-side resources, the client might not even make a request for thumbnail images.

> For example, most graphically-based operating systems have some method for displaying file icons to a user that is seated at a 
> computer terminal, so an FTP server implementation could simply return that same icon to a client in response to a THMB request.

Right, but this isn't really the purpose of this command.  The purpose is to take an original image file and reduce it to a 
specified size, not to return the OS's icon representation for that particular file type (i.e., not SHGetFileInfo() in Windows).

My proposed implementation is fairly straightforward.  If supported the FEAT command replies with THMB and the supported file types. 
When the client wants a thumbnail it asks for it with a single command.  Here is a sample result from FTP Voyager and Serv-U:

PORT 192,168,0,30,199,53
200 PORT command successful.
THMB PNG 98 88 Thomas.JPG
150 Opening thumbnail image transfer BINARY mode data connection (19738 Bytes) for Thomas.JPG.
226 Transfer complete. 19,738 bytes (19,756 compressed to 100.09%) transferred. 19.29 actual KB/sec,  19.28 effective KB/sec.

The client opens a data channel just like any other file transfer, then requests the file using THMB, followed by the output type, 
width, height, and the file name.  The server responds just like a file transfer but also includes the resulting size in 
parentheses.

While designing this command the idea was to first make it simple for both sides, second to make it operate as any other download, 
and third to make it as flexible as possible, allowing the client to decide how the resulting data is to be used and what is needed. 
Of course the server could be configured in any way needed.  For example if the server wanted to limit the size of the thumbnails, 
the server could do that.  Our implementation limits the number of simultaneous threads allowed to create the thumbnails at any 
given time.  I think these details need to be left to the server implementation.

As for using an OPT command to configure an upcoming THMB command for its image type and the thumbnail size, that seems to 
complicate the command.  It's very easy to add these parameters into a single command, eliminating the required handshaking between 
the client and server.  An added benefit is that neither the client nor the server are required to maintain these "options" 
throughout a session, rather all of the information required is sent in a single command.

If you're interested in seeing it in action, download Serv-U and FTP Voyager or SmartFTP.  Login to the server, switch FTP Voyager 
to "Thumbnails" view.  With SmartFTP simply select an image file and a thumbnail image, of dynamic size, will appear in the preview 
pane.

Thank you again for taking the time to provide your feedback.  I look forward to your comments to the above.

Mark P. Peterson - President
http://www.RhinoSoft.com
Voice: +1(262) 560-9627
FAX: +1(262) 560-9628

--------------------------------------------------
From: "Robert McMurray" <[email-masked]>
Sent: Monday, November 22, 2010 5:01 PM
To: <[email-masked]>; "'Anthony Bryan'" <[email-masked]>; <[email-masked]>
Subject: [ftpext] draft-peterson-streamlined-ftp-command-extensions

Thanks, Anthony.

No worries - I meant to send a better response in the wake of my withdrawn AGNT draft and Mark's current Streamlined FTP Command 
Extensions draft, but I had been remiss at taking the time to collect my thoughts in a more meaningful way. But since you asked, 
this seems like as good at time as any. ;-]

CSID - As evidenced by my short-lived AGNT proposal, I think it's very useful to have some sort of client/server discovery command. 
FEAT does a good job of advertising the list of commands that are supported by a server, but Mark's proposed CSID command sequence 
provides a much better way for servers and client to identify their features to each other in detail. For security reasons I would 
probably want the CSID exchange to occur AFTER the client has logged in, but I might be overreacting.

RMDA - This could be a very useful command, and it would parallel the functionality that I'm sure we all typically use in a native 
operating system environment. In a server implementation I would probably make this command configurable, so administrators would 
have an option to disable the feature. In which case I would probably hide the command from a FEAT response so it would not be 
discoverable and have the server return a 500 or 502 reply when RMDA is sent by the client and disabled at the server.

(Note: I mention my thoughts about server implementation details just to open up the discussion as to whether any implementation 
details should be called out in the draft, or if we should leave that up to the implementers to decide.)

AVBL - This command could be very useful in a client/server world; our FTP server implementation currently returns the available 
size after a successful login sequence, and the available size is based on server disk quotas and such. That being said, not every 
FTP client will bubble that information up to the user interface. Having an AVBL command would make it possible for a client to 
determine whether enough storage space is available at the server BEFORE sending a file, which would be great when you're about to 
send a 10GB file and you only have 9.99GB of storage space left. ;-]

DSIZ - This could certainly be useful, although I would probably want a user to be logged in (not anonymous) before I supported this 
command in order to reduce the risk of denial-of-service. I'd probably make it configurable so as not to tie up too many server 
resources for large directory structures; and once again, if the feature were disabled I would probably hide the command from a FEAT 
response so it would not be discoverable and return a 500 or 502 reply when DSIZ is sent by the client and disabled at the server.

THMB - I have to admit, I don't work on a graphical FTP client, so this command is less useful to me personally, but I can see the 
merit that it might have to someone that designs a graphical client. That being said, I'm not a big fan of computing the thumbnail 
dynamically, nor am I fond of allowing the client to specify the thumbnail format and size. I think that it would be better if the 
server could pre-cache thumbnails for any file type in a server-determined size and image format, and then return the binary data 
and image format information to the client. For example, most graphically-based operating systems have some method for displaying 
file icons to a user that is seated at a computer terminal, so an FTP server implementation could simply return that same icon to a 
client in response to a THMB request.

Please bear in mind, I haven't given a lot of thought as to what the client/server interchange might look like if the server were 
more in control of the response to a THMB request, but it would probably be sufficient to simply return the image type as part of 
the response and let the client determine the image dimensions based on the client's capabilities for reading the image data. (As a 
server guy I like to push any client-friendly processing on the client whenever possible. ;-]) You could borrow an idea from the 
HASH discussion that we've been having and make the thumbnail image type configurable for the client by using an OPTS command, which 
might look like the following examples.

In this first example, the server has a pre-configured thumbnail image type of JPG, and may have pre-cached the thumbnails for 
files; any THMB request with no OPTS specified would use the server's pre-configured thumbnail image type of JPG and image type is 
specified in the command channel response:

  C> THMB widget.png

In these next examples a client is asking for different thumbnail image types by using OPTS commands, and the thumbnail image types 
would still be specified in the command channel responses:

  C> OPTS THMB GIF

  C> THMB widget.psp

  C> OPTS THMB PNG

  C> THMB widget.doc

In this next example a client is asking for an unsupported image type, so the server would simply return the appropriate error 
reply:

  C> OPTS THMB FOO

In this next example a client is asking for the currently-configured thumbnail image type, so the server would return the 
appropriate response:

  C> OPTS THMB
  S> 200 JPG

You might take this one step further - the server might choose to only support its pre-configured thumbnail image type, thereby 
allowing pre-caching of all thumbnails in one specific image format. In this example a client is asking for PNG thumbnails, which 
SHOULD be a supported image type, but the server only supports a pre-configured thumbnail image type for JPG files and returns the 
appropriate error reply (504?) and the server's pre-configured thumbnail image type is specified in the subsequent command channel 
response to a THMB request:

  C> OPTS THMB PNG

  C> THMB widget.xls

As I said earlier, I hadn't given much thought as to what the client/server THMB interchange might look like if you put the server 
in control instead of the client, but as a server guy I'd rather see THMB look a little more like the examples that I gave. ;-]

Thanks again!

Robert McMurray
[email-masked]

_______________________________________________
ftpext mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ftpext

---

From: Damin <[email-masked]>
Date: Mon, 22 Nov 2010 23:17:15 -0500

Hi,

On 22 November 2010 19:32, Mark P. Peterson <[email-masked]> wrote:

> > For security reasons I would probably want the CSID exchange to occur
> AFTER the client has logged in, but I might be overreacting.
>
> There is a lot of discussion on the Internet about servers providing "too"
> much information, especially prior to login.  But having
> worked on both sides of the FTP process, it's usually very easy for the
> client to identify what brand or flavor the server is.
> There are details which usually identify the server such as the
> completeness of the FEAT response, the welcome message, responses to
> certain messages, etc.  So, I think it's okay to be available prior to
> login.  I could change the document to read that it should be
I prefer to edge on the side of caution. I don´t see any good reason that
this SHOULD be available before login. Commands available prior to login
should be as limited as possible.

>
> RMDA:
> > In a server implementation I would probably make this command
> configurable, so administrators would have an option to disable the
> > feature.  In which case I would probably hide the command from a FEAT
> response so it would not be discoverable and have the server
> I don't think my document specifies that it isn't configurable.  My
> implementation in Serv-U is configurable as is almost all other
> FTP commands.  I think this can be left up to the server implementation.
>  Your possible implementation is exactly how Serv-U
> implements it.  If the feature is disabled, it doesn't appear in the FEAT
> response, nor is it available through the HELP command.
>
>
AVBL:
> > ...
>
> Your comments address the reason for its existence.  The client-side
> implementation in SmartFTP uses the command in this manner.
>
> DSIZ:
> Any information about paths or files requires login in FTP.  I am sure this
> is the case here as well.  If a server wouldn't want the
> "anonymous" user to have access to this command it could be configured to
> disallow this command at the server's implementation or
> > I'd probably make it configurable so as not to tie up too many server
> resources for large directory structures;...
>
> Agreed, the time-consuming nature of the implementation is discussed but
> configuration by an administrator or server-implementation

> THMB:
> > I'm not a big fan of computing the thumbnail dynamically, nor am I fond
> of allowing the client to specify the thumbnail format and
> > size.
>
> That's the point of the command, though.  It's to reduce the amount of data
> that is required to travel over the wire.  The idea is
> to shorten delivery time to the client, letting the client decide what is
> best for the client, not the server.  Some clients may
> want very small images, say for a Windows Explorer style listing, others
> may want larger images (640x480 for example) to show a
> slide show.  I think the client needs to make that determination.  If the
> server decides to cache these generated images it can.
>
> > I think that it would be better if the server could pre-cache thumbnails
> for any file type in a server-determined size and image
> > For example, most graphically-based operating systems have some method
> for displaying file icons to a user that is seated at a
> > computer terminal, so an FTP server implementation could simply return
> that same icon to a client in response to a THMB request.
>
> Right, but this isn't really the purpose of this command.  The purpose is
> to take an original image file and reduce it to a
> specified size, not to return the OS's icon representation for that
> particular file type (i.e., not SHGetFileInfo() in Windows).
>
> My proposed implementation is fairly straightforward.  If supported the
> FEAT command replies with THMB and the supported file types.
> When the client wants a thumbnail it asks for it with a single command.
>  Here is a sample result from FTP Voyager and Serv-U:
>
> PORT 192,168,0,30,199,53
> 226 Transfer complete. 19,738 bytes (19,756 compressed to 100.09%)
> transferred. 19.29 actual KB/sec,  19.28 effective KB/sec.
>
> The client opens a data channel just like any other file transfer, then
> requests the file using THMB, followed by the output type,
> width, height, and the file name.  The server responds just like a file
> transfer but also includes the resulting size in
> parentheses.
>
> While designing this command the idea was to first make it simple for both
> sides, second to make it operate as any other download,
> and third to make it as flexible as possible, allowing the client to decide
> how the resulting data is to be used and what is needed.
> Of course the server could be configured in any way needed.  For example if
> the server wanted to limit the size of the thumbnails,
> the server could do that.  Our implementation limits the number of
> simultaneous threads allowed to create the thumbnails at any
> As for using an OPT command to configure an upcoming THMB command for its
> image type and the thumbnail size, that seems to
> complicate the command.  It's very easy to add these parameters into a
> single command, eliminating the required handshaking between
> the client and server.  An added benefit is that neither the client nor the
> server are required to maintain these "options"
> If you're interested in seeing it in action, download Serv-U and FTP
> Voyager or SmartFTP.  Login to the server, switch FTP Voyager
> to "Thumbnails" view.  With SmartFTP simply select an image file and a
> thumbnail image, of dynamic size, will appear in the preview
> pane.
>

> No worries - I meant to send a better response in the wake of my withdrawn
> AGNT draft and Mark's current Streamlined FTP Command
> Extensions draft, but I had been remiss at taking the time to collect my
> thoughts in a more meaningful way. But since you asked,
> CSID - As evidenced by my short-lived AGNT proposal, I think it's very
> useful to have some sort of client/server discovery command.
> FEAT does a good job of advertising the list of commands that are supported
> by a server, but Mark's proposed CSID command sequence
> provides a much better way for servers and client to identify their
> features to each other in detail. For security reasons I would
> RMDA - This could be a very useful command, and it would parallel the
> functionality that I'm sure we all typically use in a native
> operating system environment. In a server implementation I would probably
> make this command configurable, so administrators would
> have an option to disable the feature. In which case I would probably hide
> the command from a FEAT response so it would not be
> discoverable and have the server return a 500 or 502 reply when RMDA is
> sent by the client and disabled at the server.
>
> (Note: I mention my thoughts about server implementation details just to
> open up the discussion as to whether any implementation
> AVBL - This command could be very useful in a client/server world; our FTP
> server implementation currently returns the available
> size after a successful login sequence, and the available size is based on
> server disk quotas and such. That being said, not every
> FTP client will bubble that information up to the user interface. Having an
> AVBL command would make it possible for a client to
> determine whether enough storage space is available at the server BEFORE
> sending a file, which would be great when you're about to
> DSIZ - This could certainly be useful, although I would probably want a
> user to be logged in (not anonymous) before I supported this
> command in order to reduce the risk of denial-of-service. I'd probably make
> it configurable so as not to tie up too many server
> resources for large directory structures; and once again, if the feature
> were disabled I would probably hide the command from a FEAT
> response so it would not be discoverable and return a 500 or 502 reply when
> DSIZ is sent by the client and disabled at the server.
>
> THMB - I have to admit, I don't work on a graphical FTP client, so this
> command is less useful to me personally, but I can see the
> merit that it might have to someone that designs a graphical client. That
> being said, I'm not a big fan of computing the thumbnail
> dynamically, nor am I fond of allowing the client to specify the thumbnail
> format and size. I think that it would be better if the
> server could pre-cache thumbnails for any file type in a server-determined
> size and image format, and then return the binary data
> and image format information to the client. For example, most
> graphically-based operating systems have some method for displaying
> file icons to a user that is seated at a computer terminal, so an FTP
> server implementation could simply return that same icon to a
> Please bear in mind, I haven't given a lot of thought as to what the
> client/server interchange might look like if the server were
> more in control of the response to a THMB request, but it would probably be
> sufficient to simply return the image type as part of
> the response and let the client determine the image dimensions based on the
> client's capabilities for reading the image data. (As a
> server guy I like to push any client-friendly processing on the client
> whenever possible. ;-]) You could borrow an idea from the
> HASH discussion that we've been having and make the thumbnail image type
> configurable for the client by using an OPTS command, which
> In this first example, the server has a pre-configured thumbnail image type
> of JPG, and may have pre-cached the thumbnails for
> files; any THMB request with no OPTS specified would use the server's
> pre-configured thumbnail image type of JPG and image type is
> In these next examples a client is asking for different thumbnail image
> types by using OPTS commands, and the thumbnail image types
> In this next example a client is asking for an unsupported image type, so
> the server would simply return the appropriate error
> reply:
>
>  C> OPTS THMB FOO
> In this next example a client is asking for the currently-configured
> thumbnail image type, so the server would return the
> appropriate response:
>
>  C> OPTS THMB
>  S> 200 JPG
>
> You might take this one step further - the server might choose to only
> support its pre-configured thumbnail image type, thereby
> allowing pre-caching of all thumbnails in one specific image format. In
> this example a client is asking for PNG thumbnails, which
> SHOULD be a supported image type, but the server only supports a
> pre-configured thumbnail image type for JPG files and returns the
> appropriate error reply (504?) and the server's pre-configured thumbnail
> image type is specified in the subsequent command channel
> As I said earlier, I hadn't given much thought as to what the client/server
> THMB interchange might look like if you put the server
> in control instead of the client, but as a server guy I'd rather see THMB
> look a little more like the examples that I gave. ;-]
>
> Thanks again!
>
> Robert McMurray
> [email-masked]
>

---

From: "Mark P. Peterson" <[email-masked]>
Date: Tue, 23 Nov 2010 06:52:58 -0600

Hi All,

Is there anyone else that agrees with Damin about CSID?  He says it SHOULD be available only after login.

From: Damin
Sent: Monday, November 22, 2010 10:17 PM
To: [email-masked]
Subject: Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions

Hi,

Hi Robert,

I prefer to edge on the side of caution. I don´t see any good reason that this SHOULD be available before login. Commands available 
prior to login should be as limited as possible.

PORT 192,168,0,30,199,53

Thanks, Anthony.

 C> THMB widget.png

 C> OPTS THMB GIF

 C> THMB widget.psp

 C> OPTS THMB PNG

 C> THMB widget.doc

 C> OPTS THMB FOO

 C> OPTS THMB
 S> 200 JPG

 C> OPTS THMB PNG

 C> THMB widget.xls

Thanks again!

Robert McMurray
[email-masked]

---

From: "William F. Maton" <[email-masked]>
Date: Tue, 23 Nov 2010 08:55:08 -0500 (EST)

> Is there anyone else that agrees with Damin about CSID?  He says it SHOULD be available only after login.

Hi Mark,
 	I'm certainly and old enough salt at systems administration to 
think that identification before authentication or authorization is not a 
good thing.  It makes it easier for the scanners to pick-off what servers 
are running.  I'd like to force them into making the extra effort, since 
it might as well cost them more work and I can monitor the activity 
better.

 	I think standardizing CSID (from the old non-standard CLNT) is a 
good thing though.  It should be left up to the implementor to support it.

 	I've also read he rest of that I-D and have more thoughts on it, 
but that's for another email.

> Mark P. Peterson - President

wfms

---

From: Mat Berchtold <[email-masked]>
Date: Tue, 23 Nov 2010 08:13:37 -0600

Hello Mark ..

>CSID after login
I also think the CSID command should only be available after login. I cannot think of a good reason where a  client with good intentions would need it before. But maybe I'm missing something here?

>Facts
For the facts, I would propose the following facts which would replace the existing Name and Version facts:
VendorName
ProductName
ProductVersion
Where ProductVersion must be in the following format 1*DIGIT "." 1*DIGIT "." 1*DIGIT* "." 1*DIGIT which represents the version as following: major.minor.build.patch

An example:
VendorName: SmartSoft Ltd.
ProductName: SmartFTP
ProductVersion: 4.0.1145.0

>Server/Client fact requirement
The draft proposes that the client must provide the name and version fact where for the server these facts are optional. For equality both parties involved should have equal requirements/rights. My proposal is that all facts are optional. The reasoning is that there is no need for the client to expose its name or version if it only wants to figure out the case sensitivity of the server file system.

Regards,
Mat

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of William F. Maton
Sent: Tuesday, 23 November, 2010 14:55
To: Mark P. Peterson
Cc: [email-masked]
Subject: Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions

> Is there anyone else that agrees with Damin about CSID?  He says it SHOULD be available only after login.

Hi Mark,
        I'm certainly and old enough salt at systems administration to think that identification before authentication or authorization is not a good thing.  It makes it easier for the scanners to pick-off what servers are running.  I'd like to force them into making the extra effort, since it might as well cost them more work and I can monitor the activity better.

        I think standardizing CSID (from the old non-standard CLNT) is a good thing though.  It should be left up to the implementor to support it.

        I've also read he rest of that I-D and have more thoughts on it, but that's for another email.

wfms

---

From: "Mark P. Peterson" <[email-masked]>
Date: Tue, 23 Nov 2010 08:40:14 -0600

Hi Mat,

>CSID after login
Sounds good to me, I don't have a good reason for allowing it before login.

>Facts
I don't have any problem with this either, except for "ProductVersion".  The problem I have with this is that we would be forcing 
venders into a particular version numbering scheme.  Although both you and I and many other vendors use this format, some do not and 
consequently would be unable to follow the specification.

> The draft proposes that the client must provide the name and version...
Agreed.

I would also like to give credit to Mat, that the CSID command was actually Mat's idea.  Prior to preparing this document we 
discussed other commands that we might like to include in this draft.  CSID came out of that discussion.

--------------------------------------------------
From: "Mat Berchtold" <[email-masked]>
Sent: Tuesday, November 23, 2010 8:13 AM
To: "William F. Maton" <[email-masked]>; "Mark P. Peterson" <[email-masked]>
Cc: <[email-masked]>
Subject: RE: [ftpext] draft-peterson-streamlined-ftp-command-extensions

Hello Mark ..

>CSID after login
I also think the CSID command should only be available after login. I cannot think of a good reason where a  client with good 
intentions would need it before. But maybe I'm missing something here?

For the facts, I would propose the following facts which would replace the existing Name and Version facts:
VendorName
ProductName
ProductVersion
Where ProductVersion must be in the following format 1*DIGIT "." 1*DIGIT "." 1*DIGIT* "." 1*DIGIT which represents the version as 
following: major.minor.build.patch

An example:

>Server/Client fact requirement
The draft proposes that the client must provide the name and version fact where for the server these facts are optional. For 
equality both parties involved should have equal requirements/rights. My proposal is that all facts are optional. The reasoning is 
that there is no need for the client to expose its name or version if it only wants to figure out the case sensitivity of the server 
file system.

Regards,
Mat

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of William F. Maton
Sent: Tuesday, 23 November, 2010 14:55
To: Mark P. Peterson
Cc: [email-masked]
Subject: Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions

> Is there anyone else that agrees with Damin about CSID?  He says it SHOULD be available only after login.

Hi Mark,
        I'm certainly and old enough salt at systems administration to think that identification before authentication or 
authorization is not a good thing.  It makes it easier for the scanners to pick-off what servers are running.  I'd like to force 
them into making the extra effort, since it might as well cost them more work and I can monitor the activity better.

        I think standardizing CSID (from the old non-standard CLNT) is a good thing though.  It should be left up to the implementor 
to support it.

wfms

---

From: "Mark P. Peterson" <[email-masked]>
Date: Tue, 23 Nov 2010 08:41:36 -0600

Hi William,

Thank you for taking the time to respond.  I cannot think
of any real reason why CSID shouldn't only be allowed after login.
I will change the document to reflect this.

--------------------------------------------------
From: "William F. Maton" <[email-masked]>
Sent: Tuesday, November 23, 2010 7:55 AM
To: "Mark P. Peterson" <[email-masked]>
Cc: "Damin" <[email-masked]>; <[email-masked]>
Subject: Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions

Hi Mark,

wfms

---

From: Paul Ford-Hutchinson <[email-masked]>
Date: Tue, 23 Nov 2010 15:16:59 +0000

Two comments about the CSID before login question:

1) RFCs define protocols not implementations, so the discussion should be 
about what _can_ happen and how a client/server should respond.
2) I'd like to be 100% certain what payload CSID can carry before banning 
it totally prior to authentication.

Point 1 (the protocol vs implementation thing)
======

We need to say what is and isn't allowed in the protocol, comments about 
implementations are interesting but not really the thrust of an RFC.  In 
this case, we need to state when a client may send CSID; what a server 
should do if the client sends it at the wrong time - and at the right 
time; whether the client should expect that a server might refuse to 
answer a CSID command because it is in a particular state and that it 
might be sensible for it to try again when in a different state in the 
protocol (e.g. logged in).

Also, don't forget that users can log out too - so simply stating after a 
USER command (or some such) isn't really good enough.

Point 2
======

HTTPS is broken because of something similar.  Multi-homed http servers 
cannot operate on HTTPS because the server needs to present the cert to 
the client before the client tells the server which certificate (i.e. 
host) it should be.

I don't know if CSID may contain host specific info  (ext_cfactname and 
local_fact seem to be a bit of a catch all that might get used for 
"hostname" or some such.) but be very wary of this catching people out. At 
the very least, if CSID should not be used for host identification, then 
ban such values from being in these fields.

Perhaps CSID could be permitted to pass different information depending on 
the state of the user's session.  (which brings us back to the protocol vs 
implementation issue - should CSID always pass the same information 
regardless of session state or is the peer allowed to use session context 
to decide what to include - thus meaning that two calls to CSID might be 
allowed to pass different values. In fact - is a client allowed to send 
multiple CSID commands and what should a server do if the information 
passed in the second command does not match that from the first (and does 
REIN need to be involved?)?)

Cheers,
Paul

-- 

Paul Ford-Hutchinson CISSP - Tivoli Security Consultant
IBM UK Ltd. - NHBR - 1PH - North Harbour - Portsmouth - PO6 3AU
IBM Certified Deployment Professional - Tivoli Compliance Insight Manager 
V8.5

Tel +44 (0)7500 078379  (internal: 37269105)

From:
"Mark P. Peterson" <[email-masked]>
To:
"William F. Maton" <[email-masked]>
Cc:
[email-masked]
Date:
23/11/2010 14:45
Subject:
Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions
Sent by:
[email-masked]

Hi William,

Hi Mark,

>> For security reasons I would probably want the CSID exchange to occur 
AFTER the client has logged in, but I might be 
>> overreacting.
>
>
> There is a lot of discussion on the Internet about servers providing 
"too" much information, especially prior to login.  But 
> having
> There are details which usually identify the server such as the 
completeness of the FEAT response, the welcome message, responses 
> to
login.  I could change the document to read that it should 
> be
> I prefer to edge on the side of caution. I don´t see any good reason 
that this SHOULD be available before login. Commands 
> available
>> feature.  In which case I would probably hide the command from a FEAT 
response so it would not be discoverable and have the 
>> server
implementation in Serv-U is configurable as is almost all 
> other
> implements it.  If the feature is disabled, it doesn't appear in the 
FEAT response, nor is it available through the HELP command.
> Any information about paths or files requires login in FTP.  I am sure 
this is the case here as well.  If a server wouldn't want 
> the
> "anonymous" user to have access to this command it could be configured 
to disallow this command at the server's implementation or
> Agreed, the time-consuming nature of the implementation is discussed but 
configuration by an administrator or 
> server-implementation
of allowing the client to specify the thumbnail format 
>> and
>> size.
>
>
> That's the point of the command, though.  It's to reduce the amount of 
data that is required to travel over the wire.  The idea is
> to shorten delivery time to the client, letting the client decide what 
is best for the client, not the server.  Some clients may
> slide show.  I think the client needs to make that determination.  If 
the server decides to cache these generated images it can.
>
>
>> I think that it would be better if the server could pre-cache 
thumbnails for any file type in a server-determined size and image
>
>
> Right, but this isn't really the purpose of this command.  The purpose 
is to take an original image file and reduce it to a
> My proposed implementation is fairly straightforward.  If supported the 
FEAT command replies with THMB and the supported file 
> types.
> parentheses.
>
> While designing this command the idea was to first make it simple for 
both sides, second to make it operate as any other download,
> and third to make it as flexible as possible, allowing the client to 
decide how the resulting data is to be used and what is 
> needed.
> Of course the server could be configured in any way needed.  For example 
if the server wanted to limit the size of the thumbnails,
> As for using an OPT command to configure an upcoming THMB command for 
its image type and the thumbnail size, that seems to
> complicate the command.  It's very easy to add these parameters into a 
single command, eliminating the required handshaking 
> between
> the client and server.  An added benefit is that neither the client nor 
the server are required to maintain these "options"
> to "Thumbnails" view.  With SmartFTP simply select an image file and a 
thumbnail image, of dynamic size, will appear in the 
> preview
> No worries - I meant to send a better response in the wake of my 
withdrawn AGNT draft and Mark's current Streamlined FTP Command
> CSID - As evidenced by my short-lived AGNT proposal, I think it's very 
useful to have some sort of client/server discovery 
> command.
> FEAT does a good job of advertising the list of commands that are 
supported by a server, but Mark's proposed CSID command sequence
> operating system environment. In a server implementation I would 
probably make this command configurable, so administrators would
> have an option to disable the feature. In which case I would probably 
hide the command from a FEAT response so it would not be
> AVBL - This command could be very useful in a client/server world; our 
FTP server implementation currently returns the available
> size after a successful login sequence, and the available size is based 
on server disk quotas and such. That being said, not every
> FTP client will bubble that information up to the user interface. Having 
an AVBL command would make it possible for a client to
> DSIZ - This could certainly be useful, although I would probably want a 
user to be logged in (not anonymous) before I supported 
> this
> command in order to reduce the risk of denial-of-service. I'd probably 
make it configurable so as not to tie up too many server
> resources for large directory structures; and once again, if the feature 
were disabled I would probably hide the command from a 
> FEAT
> response so it would not be discoverable and return a 500 or 502 reply 
when DSIZ is sent by the client and disabled at the server.
> merit that it might have to someone that designs a graphical client. 
That being said, I'm not a big fan of computing the thumbnail
> dynamically, nor am I fond of allowing the client to specify the 
thumbnail format and size. I think that it would be better if the
> server could pre-cache thumbnails for any file type in a 
server-determined size and image format, and then return the binary data
> file icons to a user that is seated at a computer terminal, so an FTP 
server implementation could simply return that same icon to 
> a
> more in control of the response to a THMB request, but it would probably 
be sufficient to simply return the image type as part of
> the response and let the client determine the image dimensions based on 
the client's capabilities for reading the image data. (As 
> a
> HASH discussion that we've been having and make the thumbnail image type 
configurable for the client by using an OPTS command, 
> which
> In this first example, the server has a pre-configured thumbnail image 
type of JPG, and may have pre-cached the thumbnails for
> In these next examples a client is asking for different thumbnail image 
types by using OPTS commands, and the thumbnail image 
> types
> In this next example a client is asking for an unsupported image type, 
so the server would simply return the appropriate error
> As I said earlier, I hadn't given much thought as to what the 
client/server THMB interchange might look like if you put the server
> in control instead of the client, but as a server guy I'd rather see 
THMB look a little more like the examples that I gave. ;-]

wfms

---

From: "Mark P. Peterson" <[email-masked]>
Date: Tue, 23 Nov 2010 13:52:17 -0600

Hi Paul,

Thank you for taking the time to respond.

Point 1
====== 
I agree with your points.  I will clarify the document.

Point 2 
====== 
CSID doesn't contain any host specific information, the intent is only to exchange information about the software between server and client.  As for HOST this is being proposed in a different draft for FTP.  I believe the information passed between the client and server should only apply to the actual software and platform and is independent from the login information.  In other words it should always return the same information.

From: Paul Ford-Hutchinson 
Sent: Tuesday, November 23, 2010 9:16 AM
To: Mark P. Peterson ; [email-masked] 
Subject: Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions

1) RFCs define protocols not implementations, so the discussion should be about what _can_ happen and how a client/server should respond. 
2) I'd like to be 100% certain what payload CSID can carry before banning it totally prior to authentication. 

We need to say what is and isn't allowed in the protocol, comments about implementations are interesting but not really the thrust of an RFC.  In this case, we need to state when a client may send CSID; what a server should do if the client sends it at the wrong time - and at the right time; whether the client should expect that a server might refuse to answer a CSID command because it is in a particular state and that it might be sensible for it to try again when in a different state in the protocol (e.g. logged in). 

Also, don't forget that users can log out too - so simply stating after a USER command (or some such) isn't really good enough. 

Point 2 
====== 

HTTPS is broken because of something similar.  Multi-homed http servers cannot operate on HTTPS because the server needs to present the cert to the client before the client tells the server which certificate (i.e. host) it should be. 

I don't know if CSID may contain host specific info  (ext_cfactname and local_fact seem to be a bit of a catch all that might get used for "hostname" or some such.) but be very wary of this catching people out.  At the very least, if CSID should not be used for host identification, then ban such values from being in these fields. 

Perhaps CSID could be permitted to pass different information depending on the state of the user's session.  (which brings us back to the protocol vs implementation issue - should CSID always pass the same information regardless of session state or is the peer allowed to use session context to decide what to include - thus meaning that two calls to CSID might be allowed to pass different values. In fact - is a client allowed to send multiple CSID commands and what should a server do if the information passed in the second command does not match that from the first (and does REIN need to be involved?)?) 

Cheers, 
Paul 

-- 

Paul Ford-Hutchinson CISSP - Tivoli Security Consultant
IBM UK Ltd. - NHBR - 1PH - North Harbour - Portsmouth - PO6 3AU
IBM Certified Deployment Professional - Tivoli Compliance Insight Manager V8.5

      From:  "Mark P. Peterson" <[email-masked]>  
      To:  "William F. Maton" <[email-masked]>  
      Cc:  [email-masked]  
      Date:  23/11/2010 14:45  
      Subject:  Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions  
      Sent by:  [email-masked] 

--------------------------------------------------------------------------------

Hi William,

Hi Mark,

wfms

---

From: Paul Ford-Hutchinson <[email-masked]>
Date: Tue, 23 Nov 2010 22:40:02 +0000

I understand that HOST is designed to do the job, but extensible crannies 
in protocols tend to get used for all sorts of things they aren't intended 
for.

In fact, it wouldn't surprise me if the review process ends up kicking the 
'extensible' bits of this command into the long grass.  (i.e. the argument 
that it's actually creating interoperability issues rather than preventing 
them - if there's a need for a value - it should be defined syntactically 
and semantically - otherwise it doesn't belong in a protocol spec.)

I suggest that there should be some discussion about what kind of 
information is valid in the CSID command/response and when a client/server 
might expect it to alter (or not).  And if it turns out it is tied to a 
"session" - point out that a "session" can mean a number of things in FTP, 
so that will need to be specified too.

Cheers,
Paul
-- 

From:
"Mark P. Peterson" <[email-masked]>
To:
<[email-masked]>, Paul Ford-Hutchinson/UK/IBM@IBMGB
Date:
23/11/2010 19:55
Subject:
Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions

Hi Paul,

Point 1
====== 
 
Point 2 
====== 
CSID doesn't contain any host specific information, the intent is only to 
exchange information about the software between server and client.  As for 
HOST this is being proposed in a different draft for FTP.  I believe the 
information passed between the client and server should only apply to the 
actual software and platform and is independent from the login 
information.  In other words it should always return the same information.

Point 2 
====== 

Cheers, 
Paul 

-- 

From: 

Hi William,

Hi Mark,

wfms

---

From: "Ángel González" <[email-masked]>
Date: Wed, 24 Nov 2010 00:54:15 +0100

Hello all,

Robert McMurray wrote:
> functionality that I'm sure we all typically use in a native operating
> system environment. In a server implementation I would probably make
> this command configurable, so administrators would have an option to
> disable the feature. In which case I would probably hide the command
> from a FEAT response so it would not be discoverable and have the
> server return a 500 or 502 reply when RMDA is sent by the client and
> disabled at the server.

I don't think a server should do that or be encouraged to. If a server
rejects a RMDA the client SHOULD fall back to a recursive listing and
removing.
As currently defined, on receiving a RMDA the server MUST delete
everything in the subpath the user is allowed to. I think that changing
it to also allow a not remove anything would solve the issue.
If the folder being RMDAed is not deletable by the logged in user, the
server MAY -based both on its configuration and user credentials-
directly reject the command. On the other hand, if it deletes one file
(and can not rollback the operation), it must continue the whole subtree.

About THMB:
The format is restricted to registered image subtypes. It should allow
full MIME types, so an application/x-better-image-format could be used.
And giving that step, always using the full mime for images would seem
more clear.

I also prefer the OPTS syntax, since that would allow larger options and
extensions, such as:

  C> OPTS THMB image/png 60 60 page=2
  S> 200 OPTS THMB PNG command successful.

  C> THMB document.doc
  S> 150 PNG Starting thumbnail transfer
  S> 226 Transfer complete.

for requesting a thumbnail for the second page of the document.

Since the OPTS would be given once at the beginning and then used for many files, it's also important for reducing its continuous repetition.

Robert McMurray wrote:
> That being said, I'm not a big fan of computing the thumbnail dynamically, nor am I fond of allowing the client to specify the thumbnail format and size. I think that it would be better if the server could pre-cache thumbnails for any file type in a server-determined size and image format, and then return the binary data and image format information to the client. For example, most graphically-based operating systems have some method for displaying file icons to a user that is seated at a computer terminal, so an FTP server implementation could simply return that same icon to a client in response to a THMB request.
Note that the server can compute it the first time, then cache the
thumbnails you have requested on previous sessions. Moreover, it could
get heuristics as 'Start prethumbnailing with the current options if it
has requested in order the thumbnails of the first three files in the
folder'.

There should be a note about the potential security issues of the THMB
command. Vulnerabilities processing images are common.

> The draft proposes that the client must provide the name and version fact where for the server these facts are optional. For equality both parties involved should have equal requirements/rights. My…
Sad as it is, it may be useful for implementing client-specific
workarounds in the server.

---

From: Robert McMurray <[email-masked]>
Date: Wed, 24 Nov 2010 04:26:11 +0000

Thanks, Mark.

> Mark wrote:
> data that is required to travel over
> the wire.  The idea is to shorten
> delivery time to the client, letting
> the client decide what is best for
> the client, not the server.

I understood that to be the case, but looking back to Anthony's original question - he had asked which commands were more valuable to me than others. With that in mind, when I considered the THMB command from a server implementation point-of-view as it appears in the current draft, I thought that the current proposal was a little impractical, which would make me lean away from implementing it.

Here's a real-world scenario that illustrates why I think that the current THMB proposal breaks down quickly in a normal use scenario:

Let's say that I have a friend who works in astrophotography, and let's say that he's doing his doctoral work on the Crab Nebula. He books some time at one of the national observatories and takes several series of images of the Crab Nebula that are stitched together into 50 large composite images. (These have very large dimensions, 100K by 100K pixels, and each is over 100MB in size, which are the perfect candidates for thumbnails.) When he publishes his thesis for peer review, he drops the full-sized images in a directory on an FTP site that I manage where his colleagues can access them. Let's say that his thesis garners a modicum of interest on the day of its publication for review - perhaps just 10 users. Each of these users has an FTP client that supports the THMB command as it is currently designed, and each FTP client can use varying thumbnail dimensions. So here's what happens:

Client 1 has the following transaction:

Client01> THMB JPG 100 100 Crab_Nebula_01.jpg
Server00> 150 Starting thumbnail transfer for Crab_Nebula_01.jpg
-         //Note: 5K bytes are transferred over the data channel//
Server00> 226 Transfer complete.

Since the thumbnail was generated dynamically based on the client's input, one of my server's CPUs spikes while it loads the 100MB image into RAM, renders the thumbnail in the user-requested image format and dimensions, and dumps the image on the wire. (Because a client can request any image dimensions and format, my server implementation would probably not cache the thumbnail in memory, but I'll come back to that.) Still, transferring only 5K over the wire is a lot better than transferring 100MB, so we're in agreement that reducing the bandwidth is a good thing.

But I have 9 other clients that are hitting my FTP site and they issue FTP thumbnail requests like the following examples:

Client02> THMB PNG 100 100 Crab_Nebula_01.jpg
Client03> THMB GIF 100 100 Crab_Nebula_01.jpg
Client04> THMB JPG 200 200 Crab_Nebula_01.jpg
Client05> THMB PNG 200 200 Crab_Nebula_01.jpg
Client06> THMB GIF 150 150 Crab_Nebula_01.jpg
Client07> THMB PNG 640 480 Crab_Nebula_01.jpg
Client08> THMB GIF 320 240 Crab_Nebula_01.jpg
Client09> THMB JPG 150 150 Crab_Nebula_01.jpg
Client10> THMB JPG 320 240 Crab_Nebula_01.jpg

So my server has received 10 completely dissimilar thumbnail requests, and all of those requests are just for the first physical image. If each FTP client attempts to download just 20 thumbnails in order to show the first set of thumbnails in each graphical FTP client, I have to dynamically generate 200 different thumbnails. Since these are all 100MB images, I am reducing my bandwidth at the expense of the CPU and RAM resources that are required to process the THMB requests. 

I mentioned earlier that I would probably not cache the thumbnails in RAM because the client can request any thumbnail dimensions, so let's say that my server implementation at least caches thumbnails to disk so I don't have to generate them dynamically for every request. Since each FTP client can request multiple image formats and any image dimensions, I may have to keep track of hundreds of thumbnails for each single physical image. That will quickly start to eat up disk space, so now I have to implement some form of garbage collection to clean up stale thumbnails. But even then, since my friend has 50 physical images and I configure my FTP server implementation to only keep the 10 most-recent thumbnails around, I'm still managing 500 thumbnails for his original 50 physical images.

All in all I find the approach for letting only the client have full control very impractical, and that would prevent me from wanting to implement the THMB command as it is currently documented in the draft. But that being said, I like the idea of having a thumbnail command, just not the way that it's currently proposed. That's why I was suggesting some alternatives, and I'll expound a little on that.

In my last email I had suggested letting the server be a little more in control, and I had suggested that the server could optionally tell an FTP client that the client can't specify the image format. If I use the astrophotography scenario that I just gave, that means that I could have my friend pre-create the 50 thumbnails for the 50 physical images in some fashion where the FTP server implementation would pick them up. (I could use shadow folders, unique thumbnail naming, etc.) This means that there is no spike in CPU or RAM when the 10 FTP clients issue their THMB requests; it also means that I'm only managing 50 static thumbnails. So now the requests could be as simple as the following:

Client01> THMB Crab_Nebula_01.jpg
Server00> 150 Starting thumbnail transfer for Crab_Nebula_01.jpg
Server00> 226 Transfer complete.
Client02> THMB Crab_Nebula_01.jpg
Server00> 150 Starting thumbnail transfer for Crab_Nebula_01.jpg
Server00> 226 Transfer complete.
... etc ...

The reason why I had proposed using the OPTS command was to give the client the level of control that you mentioned in your reply, e.g. "letting the client decide what is best for the client, not the server." If you were to combine your client request concepts with elements of my OPTS-based suggestion, you could create a hybrid of the two approaches that might address all concerns. Here are some examples:

In this example, the client simply asks for the server's current thumbnail configuration:
C> OPTS THMB
S> 200 JPG 100 100

In this example, the client specifies a new thumbnail format:
C> OPTS THMB PNG
S> 200 PNG 100 100

In this example, the client specifies a new thumbnail format and dimensions:
C> OPTS THMB GIF 150 150
S> 200 GIF 150 150

I think that something more of a hybrid approach works better - the client can ask for a file format and dimensions, but the server can still say "no" to a client-specified file format and dimensions when it wants to, but still return a thumbnail when the client asks for it.

C> OPTS THMB PNG 100 100
S> 504 Specifying thumbnail properties is unsupported.

C> THMB widget.png
S> 150 JPG Starting thumbnail transfer for widget.png
S> 226 Transfer complete.

This makes it easier for the server implementation to have a pre-cached collection of thumbnails, especially when given the realistic scenario that I listed above. But if you omit using OPTS command and stick with using a single THMB command, when an FTP client sends a request that your server doesn't want to fulfill for some reason, your only recourse is to fail the whole request. Whereas, if break the process into separate OPTS and THMB commands, you can fulfill a THMB request even if you reject the custom parameters that the client had requested with an OPTS command.

> Mark wrote:
> 
> This could waste server-side
> resources, the client might not even
> make a request for thumbnail images.

This is true, but using the astrophotography scenario once again, I'd rather have 50 5K thumbnail files eating up a tiny fraction of disk space (which is a dirt cheap resource) rather than trying to generate dynamic thumbnails for 50 100MB images and eating CPU and RAM (which are expensive resources).

If my server implementation didn't pre-cache thumbnails, and I configured my server to only allow JPG format thumbnails at 100x100 pixels, it would still be possible to implement some form of in-memory or dynamic to disk caching for subsequent requests, because now the list of variables has been reduced. For example:

Client01> THMB Crab_Nebula_01.jpg
-        //Note: the thumbnail was not//
-        //pre-cached, so the server  //
-        //creates it dynamically and //
-        //caches it to disk after it //
-        //sends the thumbnail to the //
-        //client                     //
Client02> THMB Crab_Nebula_01.jpg
-        //Note: the thumbnail was    //
-        //cached during the previous //
-        //request, so the server can //
-        //send the thumbnail to the  //
-        //client with no additional  //
-        //processing required        //
Client03> THMB Crab_Nebula_01.jpg
... etc ...
Client10> THMB Crab_Nebula_01.jpg

> Mark wrote:
> 
> This isn't really the purpose of this
> command.  The purpose is to take an
> original image file and reduce it to
> a specified size, not to return the
> OS's icon representation for that
> particular file type.

Perhaps that was not the originally-intended purpose for this command, but then I think that you're limiting the usefulness of the command. My suggestion to return an icon may not have been the best example, but limiting the THMB command to just image files is not very useful, since there are other files that would yield beneficial results. For example, video files are typically much larger than images, so why shouldn't video files be able to have unique thumbnails? What about thumbnails for new image types that are introduced later, like SVG files? I would say that it's certainly possible that any server implementation could refuse to send a thumbnail for any file that it chooses, but why shouldn't a server implementation be able to return a thumbnail for any file? I am simply suggesting that limiting the functionality of the THMB command to just images reduces the overall value of the command.

All that being said, as I stated earlier, I like the idea of a THMB command, but at the moment I'm not fond of the current proposal. When I consider that implementers of FTP clients might read about the THMB command in an RFC and start creating FTP clients that can issue requests for thumbnails in any number of image formats and pixel dimensions, I start to back away from this command pretty quickly.

I admit that it would be different if I implemented my own graphical FTP client and my own FTP server, because I could control the THMB interoperability in a way of my choosing - for example, I could define a specific set of thumbnail dimensions and only use JPG format. But since I would only be implementing an FTP server, I don't like the way the odds are stacked. (In some ways this is like writing a function to strip whitespace from text files - if you generate all of your own text files, then you only have to anticipate what you've defined as whitespace. But when you're stripping whitespace from someone else's text files, you have to anticipate 0x20, 0x09, 0xA0, multiple character sets, what to do when you get character codes that you don't recognize, etc.)

Just the same, I'd like to see some form of thumbnail functionality if possible, because the scenario about astrophotography wasn't hypothetical - I actually have a friend that works with astrophotography who generates those types of huge image files; having an effective method to retrieve thumbnails for those large images would be great. But I don't think that the current draft offers an example of an effective method when I consider having to create dynamically-generated thumbnails for a near-infinite number of possible client request parameters.

Thanks again!

Robert McMurray
[email-masked]

---

From: Damin <[email-masked]>
Date: Wed, 24 Nov 2010 00:21:16 -0500

Hi,
I knew this THMB command would be a pain as soon as a saw it.

I think the real problem we´re having with it is that it´s TOO versatile. Do
we really need to let the client specify whatever option they want?
Obvious ways to abuse this would be to exec THMB PNG 100 100, then THMB PNG
100 101, THMB PNG 101,100, and so on...

Perhaps it would be better to simply limit the versatility, the server can
provide certain valid sizes, formats. (somehow...)

In the practical world, we don´t really care that the client gets exactly
the size they want! The point is to give a preview of an image. Say, we
configure a server ONLY to support 500x500 in JPG format. Such an image is
fairly small, easy to transfer, and easy to render. If need-be, the client
can (very quickly, and in their own resources) resize their image for actual
view.
This is a win-win for both the client, and the server. The client gets a
speedy preview, and the server can minimise abuse.

I might also point out that there could be a big problem with this command.
How do we know the aspect ratio of the original image? We can´t really
specify an appropriate thumb size without knowing that information (stretchy
images, anyone?). I think this can be solved with some appropriate commands.

On 23 November 2010 23:26, Robert McMurray <[email-masked]> wrote:

> I understood that to be the case, but looking back to Anthony's original
> question - he had asked which commands were more valuable to me than others.
> With that in mind, when I considered the THMB command from a server
> implementation point-of-view as it appears in the current draft, I thought
> that the current proposal was a little impractical, which would make me lean
> away from implementing it.
>
> Here's a real-world scenario that illustrates why I think that the current
> THMB proposal breaks down quickly in a normal use scenario:
>
> Let's say that I have a friend who works in astrophotography, and let's say
> that he's doing his doctoral work on the Crab Nebula. He books some time at
> one of the national observatories and takes several series of images of the
> Crab Nebula that are stitched together into 50 large composite images.
> (These have very large dimensions, 100K by 100K pixels, and each is over
> 100MB in size, which are the perfect candidates for thumbnails.) When he
> publishes his thesis for peer review, he drops the full-sized images in a
> directory on an FTP site that I manage where his colleagues can access them.
> Let's say that his thesis garners a modicum of interest on the day of its
> publication for review - perhaps just 10 users. Each of these users has an
> FTP client that supports the THMB command as it is currently designed, and
> each FTP client can use varying thumbnail dimensions. So here's what
> happens:
>
> Since the thumbnail was generated dynamically based on the client's input,
> one of my server's CPUs spikes while it loads the 100MB image into RAM,
> renders the thumbnail in the user-requested image format and dimensions, and
> dumps the image on the wire. (Because a client can request any image
> dimensions and format, my server implementation would probably not cache the
> thumbnail in memory, but I'll come back to that.) Still, transferring only
> 5K over the wire is a lot better than transferring 100MB, so we're in
> agreement that reducing the bandwidth is a good thing.
>
> But I have 9 other clients that are hitting my FTP site and they issue FTP
> thumbnail requests like the following examples:
>
> So my server has received 10 completely dissimilar thumbnail requests, and
> all of those requests are just for the first physical image. If each FTP
> client attempts to download just 20 thumbnails in order to show the first
> set of thumbnails in each graphical FTP client, I have to dynamically
> generate 200 different thumbnails. Since these are all 100MB images, I am
> reducing my bandwidth at the expense of the CPU and RAM resources that are
> required to process the THMB requests.
>
> I mentioned earlier that I would probably not cache the thumbnails in RAM
> because the client can request any thumbnail dimensions, so let's say that
> my server implementation at least caches thumbnails to disk so I don't have
> to generate them dynamically for every request. Since each FTP client can
> request multiple image formats and any image dimensions, I may have to keep
> track of hundreds of thumbnails for each single physical image. That will
> quickly start to eat up disk space, so now I have to implement some form of
> garbage collection to clean up stale thumbnails. But even then, since my
> friend has 50 physical images and I configure my FTP server implementation
> to only keep the 10 most-recent thumbnails around, I'm still managing 500
> thumbnails for his original 50 physical images.
>
> All in all I find the approach for letting only the client have full
> control very impractical, and that would prevent me from wanting to
> implement the THMB command as it is currently documented in the draft. But
> that being said, I like the idea of having a thumbnail command, just not the
> way that it's currently proposed. That's why I was suggesting some
> alternatives, and I'll expound a little on that.
>
> In my last email I had suggested letting the server be a little more in
> control, and I had suggested that the server could optionally tell an FTP
> client that the client can't specify the image format. If I use the
> astrophotography scenario that I just gave, that means that I could have my
> friend pre-create the 50 thumbnails for the 50 physical images in some
> fashion where the FTP server implementation would pick them up. (I could use
> shadow folders, unique thumbnail naming, etc.) This means that there is no
> spike in CPU or RAM when the 10 FTP clients issue their THMB requests; it
> also means that I'm only managing 50 static thumbnails. So now the requests
> could be as simple as the following:
>
> The reason why I had proposed using the OPTS command was to give the client
> the level of control that you mentioned in your reply, e.g. "letting the
> client decide what is best for the client, not the server." If you were to
> combine your client request concepts with elements of my OPTS-based
> suggestion, you could create a hybrid of the two approaches that might
> address all concerns. Here are some examples:
>
> I think that something more of a hybrid approach works better - the client
> can ask for a file format and dimensions, but the server can still say "no"
> to a client-specified file format and dimensions when it wants to, but still
> return a thumbnail when the client asks for it.
>
> C> OPTS THMB PNG 100 100
> This makes it easier for the server implementation to have a pre-cached
> collection of thumbnails, especially when given the realistic scenario that
> I listed above. But if you omit using OPTS command and stick with using a
> single THMB command, when an FTP client sends a request that your server
> doesn't want to fulfill for some reason, your only recourse is to fail the
> whole request. Whereas, if break the process into separate OPTS and THMB
> commands, you can fulfill a THMB request even if you reject the custom
> parameters that the client had requested with an OPTS command.
>
> > Mark wrote:
> >
> This is true, but using the astrophotography scenario once again, I'd
> rather have 50 5K thumbnail files eating up a tiny fraction of disk space
> (which is a dirt cheap resource) rather than trying to generate dynamic
> thumbnails for 50 100MB images and eating CPU and RAM (which are expensive
> resources).
>
> If my server implementation didn't pre-cache thumbnails, and I configured
> my server to only allow JPG format thumbnails at 100x100 pixels, it would
> still be possible to implement some form of in-memory or dynamic to disk
> caching for subsequent requests, because now the list of variables has been
> reduced. For example:
>
> Perhaps that was not the originally-intended purpose for this command, but
> then I think that you're limiting the usefulness of the command. My
> suggestion to return an icon may not have been the best example, but
> limiting the THMB command to just image files is not very useful, since
> there are other files that would yield beneficial results. For example,
> video files are typically much larger than images, so why shouldn't video
> files be able to have unique thumbnails? What about thumbnails for new image
> types that are introduced later, like SVG files? I would say that it's
> certainly possible that any server implementation could refuse to send a
> thumbnail for any file that it chooses, but why shouldn't a server
> implementation be able to return a thumbnail for any file? I am simply
> suggesting that limiting the functionality of the THMB command to just
> images reduces the overall value of the command.
>
> All that being said, as I stated earlier, I like the idea of a THMB
> command, but at the moment I'm not fond of the current proposal. When I
> consider that implementers of FTP clients might read about the THMB command
> in an RFC and start creating FTP clients that can issue requests for
> thumbnails in any number of image formats and pixel dimensions, I start to
> back away from this command pretty quickly.
>
> I admit that it would be different if I implemented my own graphical FTP
> client and my own FTP server, because I could control the THMB
> interoperability in a way of my choosing - for example, I could define a
> specific set of thumbnail dimensions and only use JPG format. But since I
> would only be implementing an FTP server, I don't like the way the odds are
> stacked. (In some ways this is like writing a function to strip whitespace
> from text files - if you generate all of your own text files, then you only
> have to anticipate what you've defined as whitespace. But when you're
> stripping whitespace from someone else's text files, you have to anticipate
> 0x20, 0x09, 0xA0, multiple character sets, what to do when you get character
> codes that you don't recognize, etc.)
>
> Just the same, I'd like to see some form of thumbnail functionality if
> possible, because the scenario about astrophotography wasn't hypothetical -
> I actually have a friend that works with astrophotography who generates
> those types of huge image files; having an effective method to retrieve
> thumbnails for those large images would be great. But I don't think that the
> current draft offers an example of an effective method when I consider
> having to create dynamically-generated thumbnails for a near-infinite number
> of possible client request parameters.
>
> Thanks again!
>
> Robert McMurray
> [email-masked]

---

From: "Mark P. Peterson" <[email-masked]>
Date: Wed, 24 Nov 2010 06:53:32 -0600

Hi Paul,

Thank you for providing your thoughts on CSID.  In response to your comments I would like to say that CSID is designed to do nothing more than provide information about the server software, version, vendor, and platform; and for the client the server software, version, and vendor.  The information provided by both sides should not change regardless of whether or not a user is logged in, into any particular domain or host.  It can be extended if needed, however the idea behind it is that it returns these constant values simply used to help identify both sides of the connection.

Do you think this needs to be clarified?  If so, where and what verbiage would you like to see?

From: Paul Ford-Hutchinson 
Sent: Tuesday, November 23, 2010 4:40 PM
To: Mark P. Peterson 
Cc: [email-masked] 
Subject: Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions

I understand that HOST is designed to do the job, but extensible crannies in protocols tend to get used for all sorts of things they aren't intended for. 

In fact, it wouldn't surprise me if the review process ends up kicking the 'extensible' bits of this command into the long grass.  (i.e. the argument that it's actually creating interoperability issues rather than preventing them - if there's a need for a value - it should be defined syntactically and semantically - otherwise it doesn't belong in a protocol spec.) 

I suggest that there should be some discussion about what kind of information is valid in the CSID command/response and when a client/server might expect it to alter (or not).  And if it turns out it is tied to a "session" - point out that a "session" can mean a number of things in FTP, so that will need to be specified too. 

Cheers, 
Paul 
-- 

      From:  "Mark P. Peterson" <[email-masked]>  
      To:  <[email-masked]>, Paul Ford-Hutchinson/UK/IBM@IBMGB  
      Date:  23/11/2010 19:55  
      Subject:  Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions 

Hi Paul, 

Point 1 
====== 
  
Point 2 
====== 

Point 2 
====== 

Cheers, 
Paul 

-- 

Hi William,

Hi Mark,

wfms

---

From: "Mark P. Peterson" <[email-masked]>
Date: Wed, 24 Nov 2010 07:42:16 -0600

Hi Robert,

I'm not exactly sure how to reply to your message I will try without quoting, if that makes any sense.  I understand the problem in 
your example, however I think that the problem leads to the implementation of a good FTP server vs. one that isn't so good.  In the 
case of your server I can see a cache implementation that would solve the problem, reduce the amount of required disk space, and 
provide the client with varying sizes of images as I propose with this command.  To solve the real-world problem you've outlined I 
would:

* Add a limit in the server, either hard-coded or user configurable, to identify the maximum dimensions of a thumbnail, let's use 1k 
x 1k for an example.
* Design a thumbnail cache which identifies when an image was placed into the cache, when it was last accessed, the thumbnail 
dimensions, etc.
* When a THMB command arrives for a particular file, first identify if it's in the cache.  If it is in the cache, use the file from 
the cache and create a thumbnail image and provide it to the client.  If not in the cache, create a thumbnail of the maximum allowed 
size, add it to the cache, then return the requested image to the client.
* Maintain the cache.

This simplistic outline does require an additional transformation to get the thumbnail image, however the horsepower required to 
perform this operation is now trivial, for example taking that 1k x 1k image down to 100 x 100.  If another request comes in for 200 
x 200 for that same image, again the CPU usage is very small, yet the amount of data transferred remains low and not limited.

My point to the above example is that we are talking about the implementation.  I realize implementations cannot be ignored but in 
your example an efficient simple solution does exist while not limiting the client to a fixed thumbnail size.  The implementation I 
propose, above, could be extended to support multiple image sizes in addition to a single image size, whatever is needed by the 
server.  For example, if the server received a request for a 256 x 256 thumbnail, the server could decide to create a 512 x 512 
cache image, then create the 256 x 256 sized image to send to the client.  Instead of maintaining a cache of specific sizes it could 
use reasonable stratifications from which thumbnails can be created.

If the server decides it wants to attempt to pre-cache images it could, and it could be triggered by the first thumbnail image 
requested in a directory.  Again, I think this would be an implementation detail that would be at the server author's design.

Regarding thumbnail types.  I agree with you, thumbnails of videos, PDFs, Word documents, text files, and others are all reasonable 
uses for this command.  I don't think my document prevents different file types from being supported, I think it's quite extensible.

--------------------------------------------------
From: "Robert McMurray" <[email-masked]>
Sent: Tuesday, November 23, 2010 10:26 PM
To: "'Mark P. Peterson'" <[email-masked]>; <[email-masked]>
Subject: RE: [ftpext] draft-peterson-streamlined-ftp-command-extensions

Thanks, Mark.

> the client, not the server.

I understood that to be the case, but looking back to Anthony's original question - he had asked which commands were more valuable 
to me than others. With that in mind, when I considered the THMB command from a server implementation point-of-view as it appears in 
the current draft, I thought that the current proposal was a little impractical, which would make me lean away from implementing it.

Here's a real-world scenario that illustrates why I think that the current THMB proposal breaks down quickly in a normal use 
scenario:

Let's say that I have a friend who works in astrophotography, and let's say that he's doing his doctoral work on the Crab Nebula. He 
books some time at one of the national observatories and takes several series of images of the Crab Nebula that are stitched 
together into 50 large composite images. (These have very large dimensions, 100K by 100K pixels, and each is over 100MB in size, 
which are the perfect candidates for thumbnails.) When he publishes his thesis for peer review, he drops the full-sized images in a 
directory on an FTP site that I manage where his colleagues can access them. Let's say that his thesis garners a modicum of interest 
on the day of its publication for review - perhaps just 10 users. Each of these users has an FTP client that supports the THMB 
command as it is currently designed, and each FTP client can use varying thumbnail dimensions. So here's what happens:

Since the thumbnail was generated dynamically based on the client's input, one of my server's CPUs spikes while it loads the 100MB 
image into RAM, renders the thumbnail in the user-requested image format and dimensions, and dumps the image on the wire. (Because a 
client can request any image dimensions and format, my server implementation would probably not cache the thumbnail in memory, but 
I'll come back to that.) Still, transferring only 5K over the wire is a lot better than transferring 100MB, so we're in agreement 
that reducing the bandwidth is a good thing.

So my server has received 10 completely dissimilar thumbnail requests, and all of those requests are just for the first physical 
image. If each FTP client attempts to download just 20 thumbnails in order to show the first set of thumbnails in each graphical FTP 
client, I have to dynamically generate 200 different thumbnails. Since these are all 100MB images, I am reducing my bandwidth at the 
expense of the CPU and RAM resources that are required to process the THMB requests.

I mentioned earlier that I would probably not cache the thumbnails in RAM because the client can request any thumbnail dimensions, 
so let's say that my server implementation at least caches thumbnails to disk so I don't have to generate them dynamically for every 
request. Since each FTP client can request multiple image formats and any image dimensions, I may have to keep track of hundreds of 
thumbnails for each single physical image. That will quickly start to eat up disk space, so now I have to implement some form of 
garbage collection to clean up stale thumbnails. But even then, since my friend has 50 physical images and I configure my FTP server 
implementation to only keep the 10 most-recent thumbnails around, I'm still managing 500 thumbnails for his original 50 physical 
images.

All in all I find the approach for letting only the client have full control very impractical, and that would prevent me from 
wanting to implement the THMB command as it is currently documented in the draft. But that being said, I like the idea of having a 
thumbnail command, just not the way that it's currently proposed. That's why I was suggesting some alternatives, and I'll expound a 
little on that.

In my last email I had suggested letting the server be a little more in control, and I had suggested that the server could 
optionally tell an FTP client that the client can't specify the image format. If I use the astrophotography scenario that I just 
gave, that means that I could have my friend pre-create the 50 thumbnails for the 50 physical images in some fashion where the FTP 
server implementation would pick them up. (I could use shadow folders, unique thumbnail naming, etc.) This means that there is no 
spike in CPU or RAM when the 10 FTP clients issue their THMB requests; it also means that I'm only managing 50 static thumbnails. So 
now the requests could be as simple as the following:

The reason why I had proposed using the OPTS command was to give the client the level of control that you mentioned in your reply, 
e.g. "letting the client decide what is best for the client, not the server." If you were to combine your client request concepts 
with elements of my OPTS-based suggestion, you could create a hybrid of the two approaches that might address all concerns. Here are 
some examples:

I think that something more of a hybrid approach works better - the client can ask for a file format and dimensions, but the server 
can still say "no" to a client-specified file format and dimensions when it wants to, but still return a thumbnail when the client 
asks for it.

C> OPTS THMB PNG 100 100

C> THMB widget.png

This makes it easier for the server implementation to have a pre-cached collection of thumbnails, especially when given the 
realistic scenario that I listed above. But if you omit using OPTS command and stick with using a single THMB command, when an FTP 
client sends a request that your server doesn't want to fulfill for some reason, your only recourse is to fail the whole request. 
Whereas, if break the process into separate OPTS and THMB commands, you can fulfill a THMB request even if you reject the custom 
parameters that the client had requested with an OPTS command.

> make a request for thumbnail images.

This is true, but using the astrophotography scenario once again, I'd rather have 50 5K thumbnail files eating up a tiny fraction of 
disk space (which is a dirt cheap resource) rather than trying to generate dynamic thumbnails for 50 100MB images and eating CPU and 
RAM (which are expensive resources).

If my server implementation didn't pre-cache thumbnails, and I configured my server to only allow JPG format thumbnails at 100x100 
pixels, it would still be possible to implement some form of in-memory or dynamic to disk caching for subsequent requests, because 
now the list of variables has been reduced. For example:

> OS's icon representation for that

Perhaps that was not the originally-intended purpose for this command, but then I think that you're limiting the usefulness of the 
command. My suggestion to return an icon may not have been the best example, but limiting the THMB command to just image files is 
not very useful, since there are other files that would yield beneficial results. For example, video files are typically much larger 
than images, so why shouldn't video files be able to have unique thumbnails? What about thumbnails for new image types that are 
introduced later, like SVG files? I would say that it's certainly possible that any server implementation could refuse to send a 
thumbnail for any file that it chooses, but why shouldn't a server implementation be able to return a thumbnail for any file? I am 
simply suggesting that limiting the functionality of the THMB command to just images reduces the overall value of the command.

All that being said, as I stated earlier, I like the idea of a THMB command, but at the moment I'm not fond of the current proposal. 
When I consider that implementers of FTP clients might read about the THMB command in an RFC and start creating FTP clients that can 
issue requests for thumbnails in any number of image formats and pixel dimensions, I start to back away from this command pretty 
quickly.

I admit that it would be different if I implemented my own graphical FTP client and my own FTP server, because I could control the 
THMB interoperability in a way of my choosing - for example, I could define a specific set of thumbnail dimensions and only use JPG 
format. But since I would only be implementing an FTP server, I don't like the way the odds are stacked. (In some ways this is like 
writing a function to strip whitespace from text files - if you generate all of your own text files, then you only have to 
anticipate what you've defined as whitespace. But when you're stripping whitespace from someone else's text files, you have to 
anticipate 0x20, 0x09, 0xA0, multiple character sets, what to do when you get character codes that you don't recognize, etc.)

Just the same, I'd like to see some form of thumbnail functionality if possible, because the scenario about astrophotography wasn't 
hypothetical - I actually have a friend that works with astrophotography who generates those types of huge image files; having an 
effective method to retrieve thumbnails for those large images would be great. But I don't think that the current draft offers an 
example of an effective method when I consider having to create dynamically-generated thumbnails for a near-infinite number of 
possible client request parameters.

Thanks again!

Robert McMurray
[email-masked]

---

From: "Mark P. Peterson" <[email-masked]>
Date: Wed, 24 Nov 2010 07:58:06 -0600

Hi Ángel,

RMDA
-------

I will add this to the document.

THMB
-------
>  It should allow full MIME types, so an application/x-better-image-format could be used.

I cannot find a reason to disagree, so I agree that following a standard for
identifying the thumbnail type should be used.  I could use your help here
clarifying my document.

> I also prefer the OPTS syntax, since that would allow larger options and
extensions, such as:...

I prefer specifying the parameters in the command for two primary reasons.

1) By requiring a client to use OPTS to specify output type and resolution
the client and server must exchange this information in a separate command.
Commands require a handshake and consequently some latency between the
server and client for the exchange.  Depending on many different factors this
latency, even on a fast connection, can be noticeable.  If the client wants different
sized images frequently, let's say for every thumbnail image, it would be
even more noticeable.  The bandwidth required to specify 10 bytes or less as a
parameter to the command is trivial by comparison even on the slowest
connections.

2) By requiring a client to use OPTS to identify the parameters, both the client
and the server must maintain the parameters for this command, more-or-less
as a state.  I'm a big fan of keeping things very simple.

3) End-user support is easier when parameters are specified with the command.
When end-users or customers have problems they'll often provide a portion of a
log to identify what is going on with the communication between the server and
client.  Technicians analyzing the log may not have all of the required information
regarding the THMB command.  With the parameters being issued with the
command analysis is far simpler and accurate.

Could you give me some examples or how you would like to see this worded
and where you would like to see this placed in the document?

--------------------------------------------------
From: "Ángel González" <[email-masked]>
Sent: Tuesday, November 23, 2010 5:54 PM
To: <[email-masked]>
Cc: "Robert McMurray" <[email-masked]>; <[email-masked]>; <[email-masked]>; <[email-masked]>
Subject: Re: [ftpext] draft-peterson-streamlined-ftp-command-extensions

Hello all,

About THMB:

  C> THMB document.doc

Since the OPTS would be given once at the beginning and then used for many files, it's also important for reducing its continuous 
repetition.

Robert McMurray wrote:
> That being said, I'm not a big fan of computing the thumbnail dynamically, nor am I fond of allowing the client to specify the 
> thumbnail format and size. I think that it would be better if the server could pre-cache thumbnails for any file type in a 
> server-determined size and image format, and then return the binary data and image format information to the client. For example, 
> most graphically-based operating systems have some method for displaying file icons to a user that is seated at a computer 
> terminal, so an FTP server implementation could simply return that same icon to a client in response to a THMB request.

Mat Berchtold wrote:
> that there is no need for the client to expose its name or version if it only wants to figure out the case sensitivity of the 
> server file system.

---

From: "Ángel González" <[email-masked]>
Date: Wed, 24 Nov 2010 21:36:58 +0100

> parameter to the command is trivial by comparison even on the slowest
Sure, but why would the client want differently sized images frequently?
I think the client would only need to state it once, perhaps with a
change on
window resize. The number of thumbnails on a typical case would be much
larger.

> as a state.  I'm a big fan of keeping things very simple.
There is a trade off here. Although it is tempting to not store anything
at the server
for this, I still consider that using OPTS is a better route.

> command analysis is far simpler and accurate.
The server could repeat the options in the 150 response if it is deemed
needed for
user support.

> and where you would like to see this placed in the document?

Doing a bit of googling for image vulnerabilities:
http://en.wikipedia.org/wiki/Windows_Metafile_vulnerability
http://blogs.techrepublic.com.com/itdojo/?p=892
http://www.sophos.com/pressoffice/news/articles/2004/09/va_critical16sep04.html
http://www.kb.cert.org/vuls/id/680620
http://www.kb.cert.org/vuls/id/238678
http://www.libpng.org/pub/png/libpng.html
http://secunia.com/advisories/27213/
http://hypersecurity.blogspot.com/2010/03/cve-2010-0188-adobe-reader-tiff.html

Vulnerabilities in image processing are quite bad, because it is easy to
make a client
render an image.THMB adds image rendering to ftp servers, which have a
great
exposure to the public.
These vulnerabilities often allow arbitrary code execution, but even a
DoS would be
bad for a ftp server. Most importantly, an attacker must not be able to
get LocalSystem/root
access on a server by just uploading a malicious image. Not even if the
fault lies on
some OS-provided library used for the thumbnailing.

I suppose the appropiate section is the security one, with a note to it
from THMB.
I don't know if it should be introduced by noting how thumbnailing adds
a significant
amount of code and how image vulnerabilities appear from time to time.

The main piece could be similar to this:
> Server implementations should be prepared in advance to mitigate
> vulnerabilities in
> the libraries used for thumbnailing. Servers are encouraged to perform
> all thumbnailing
> operations in a low-privilege process isolated from the server one.

Best regards

---

From: "Ángel González" <[email-masked]>
Date: Wed, 24 Nov 2010 21:42:23 +0100

Damin wrote:
> I might also point out that there could be a big problem with this
> command. How do we know the aspect ratio of the original image? We
> can´t really specify an appropriate thumb size without knowing that
> information (stretchy images, anyone?). I think this can be solved
> with some appropriate commands.

Knowing the image size is much easier than knowing how to create a
thumbnail for it. Note that the library used for thumbnailing may
already take that into account.

When given the command "THMB PNG 20 40 foo.gif" a conformant could just
shell out executing "|convert |foo.gif|-resize 20x40 thumb.png|"