Subject: Re: solving the SFTP text mode issue

From: Jeffrey Altman <[email-masked]>
Date: Fri, 10 May 2002 18:03:04 EDT

> > I don't think the MacOS X problem can be solved even
> > on the server side, because while the server can
> > determine who create the file (or at list a 4 byte
> > tag representing the creator), it can't know what the
> > creator uses for new-line.  I'd love to be wrong
> > on this, and learn that MaxOS X stores an attribute
> > that tells what the newline convention is.
> 
> As far as I know, there's no global convention for storing
> end-of-line representation. A counter-examples of sorts, is
> BBedit, which will default to any of the big-three end-of-line
> representations, as configured, and optionally store that
> fact in an application-specific resource.
> 
> Ugly hacks might be required to determine end-of-line representation
> (or character encoding, or text vs. binary files) in the general case.
> 
> (It seems to be that this is one more reason to keep any ugly hacks
> on the platform where the problem exists, (rather put lots of varients
> into a wire protocol for a text mode) and to make support for a text mode 
> transfer optional.)
> 

You do not need to have a bit that indicates the end of line format
for a file.  This can be determined along with the classification of
character set by performing a file scan prior to the transfer. For
example in C-Kermit 8.0 the output of DIR /XFERMODE for an arbitrary
directory might look like:

      1024  2000-09-15 03:46:16  .rnd (B)
      1156  2001-09-21 11:36:08  adm3.txt (T)(7BIT)
    219417  1999-09-11 11:31:34  all-escapes.txt (T)(7BIT)
     40960  1998-12-12 09:16:46  apartm~1.doc (B)
       262  1997-12-11 15:38:28  as400.txt (T)(7BIT)
    147728  1999-11-18 12:04:00  ASYCFILT.DLL (B)
       194  1999-09-10 12:25:22  authtest.ksc (T)(7BIT)
     28745  1998-06-17 00:00:00  AUTOLAYT.DLL (B)
      4559  1999-05-19 16:59:20  autotelnet.ksc (T)(7BIT)
        18  2000-03-29 16:26:50  badprint.ksc (T)(7BIT)
       531  1998-12-28 10:10:26  books.txt (T)(7BIT)
       832  2001-10-19 18:02:40  bounce.bat (T)(7BIT)
      2150  2001-03-10 08:53:56  console.reg (T)(UCS2LE)
      4252  1999-12-07 20:07:12  COPYING.TXT (T)(7BIT)
      3595  1999-12-08 17:19:52  crypto.html (T)(7BIT)
       307  1999-10-13 14:42:24  url.c (T)(7BIT)
     13398  2000-09-03 12:58:12  utf-8-demo.txt (T)(UTF8)
     20263  2000-09-03 12:58:04  utf-8-test.txt (B)
       801  1995-10-23 18:29:28  validate.cmd (T)(7BIT)

This is all computed by performing scans of the files at
runtime. C-Kermit then allows you to set default character sets to use
for 7-bit and 8-bit files as well as specifying a transfer character
set.  The transfer character-set would be UTF8 or some other Unicode
Transform.  For details see

  http://www.kermit-project.org/ckermit80.html#x4

 Jeffrey Altman * Sr.Software Designer      Kermit 95 1.1.21  available now!!!
 The Kermit Project @ Columbia University   SSH plus Telnet, FTP and HTTP
 http://www.kermit-project.org/             secured with Kerberos, SRP, and 
 [email-masked]                OpenSSL.

---

From: [email-masked] (Albert Lunde)
Date: Fri, 10 May 2002 21:59:42 CDT

> character set by performing a file scan prior to the transfer. For

File scans _are_ an ugly hack compared to reliable file attributes...
in my sordid past I've written one for determing character encoding and other
attributes of files under CDC NOS, with 6/12 bit character encodings.
;)

Anyway, with a single over-the-wire text encoding, or as few varients
as possible, most of this stuff doesn't need to be in the wire
protocol, or in interoperable clients on another OS.

--
    Albert Lunde          [email-masked] (new address)
                          [email-masked] (old address)