Subject: Ftp-WG: ftp.ipswitch.com:21 vs. Ftp rfc & draft-ietf

From: Pat LaVarre <[email-masked]>
Date: Fri, 18 Jan 2002 18:17:38 -0500

> > P.S. As yet ... replies 500 to FEAT
> Try anon login here: ftp://ftp.ipswitch.com
...
> Ftp-WG: FEAT UTF8 does NOT mean \r and \n ok in
pathnames?

Good fun, thank you.  [Please everyone tell me offline
who feels instead I should have sent this
person-to-person.]  I think I see ...

Per rfc1123, but differing from ftp.microsoft.com,
less and more than one command may appear in a packet:

        C> quote NOOP%0D%0ASYST%0D%0A
        s> 200 noop
        s> 215 UNIX

        C> quote NOOP%0D%0ASYST
        s> 200 noop
        C> quote %0D%0A
        s> 215 UNIX

Vs. rf2640 of LANG before "authentication" and FEAT
before LANG, this FtpServer replies 530 to FEAT until
after authentication:

        C> FEAT
        s> 530 Please login with USER and PASS
        ...

Vs. rfc2640, in an area not covered by rfc959, this
FtpServer liberally accepts LF as meaning CR LF:

        C> quote CWD /%0ANOOP%0D%0A
        s> 250 CWD successful
        s> 200 noop
        s> 215 UNIX

Vs. rfc2640, this FtpServer conservatively accepts CR
NUL as meaning ignore all further commands:

        C> quote CWD /%0D%00NOOP%0D%0A
        C> quote QUIT%0D%0A
        C> quote QUIT%0D%0A
        ...

Vs. rfc959, this FtpServer conservatively accepts NUL
as meaning ignore all further commands:

        C> quote CWD /%00NOOP%0D%0A
        C> quote QUIT%0D%0A
        C> quote QUIT%0D%0A
        ...

Thanks for sharing.    Pat LaVarre

__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/

---

From: "Gregory A Lundberg" <[email-masked]>
Date: Sat, 19 Jan 2002 16:20:29 -0500

> less and more than one command may appear in a packet:

Where does this "packet" junk come from?  Packets are immaterial.  The only
time packets are important is when your supporting FTP and a clueless
vendor's software is assuming packets have some special importance or are
under the control of the sending application.

The FTP is a language specification.  As such, a number of things are
_intentionally_ left unspecified.

Let's examine some byte sequences.  For my discussion, assume each byte
sequence example begins at the "start of a new line" state.

    CWD\040PATHNAME\r\n

        The correct parse yeild the patameter PATHNAME.

    CWD\040PATH\r\0NAME\r\n

        The sequence \r\0 is an escaped \r, therefore the correct parse
        for the parameter yields PATH\rNAME.

    CWD\040PATH\r\nNAME\r\n

        The sequence \r\n terminates the command, therefore the correct
        parse yields the parameter PATH and leaves NAME\r\n in the input
        stream for subsequent parsing.

    CWD\040PATH\nNAME\r\n

        The correct parse for the parameter yeilds PATH\nNAME.  On the
        control connection, a bare \n has no special meaning.

    CWD\040PATH\rNAME\r\n

        The correct parse yields the parameter PATH\rNAME.  The \r,
        however, was not followed by either \0 or \n and is technically
        a syntax error.  Client authors SHOULD NOT send this sequence
        and MUST be prepared for it to be rejected as a syntax error.
        The intention of the FTP, however, is that the control
        connection be usable by a human using the TELNET protocol.  Here
        the principle "be liberal in what you accept" comes into play.
        We have a choice of issuing an error response due to the syntax
        error or accepting the parameter; the server SHOULD (liberally)
        accept it and SHOULD NOT issue an error response.

        This desire to support a user's use of TELNET leads to the other
        problem: all TELNET commands are preceeded by IAC (\0377).  So
        that character must also be escaped.

    CWD\040PATH\0377\0377NAME\r\n

        The correct parse yields the parameter PATH\0377NAME.  The TELNET
        protocol specifies that IAC IAC represents an escaped IAC to be
        treated as data and not a TELNET command.

    CWD\040PATH\0377\0375\001NAME\r\n

        The correct parse yeilds the parameter PATHNAME.  The sequence
        IAC DO ECHO (\0377\0375\0001) is a TELNET command.  Whether
        (and how) the FTP server honors this command is unspecified; it
        MAY honor the command (injecting IAC WILL ECHO into the return
        data stream for the control connection), MAY recognize and
        refuse the command (injecting IAC WONT ECHO), or MAY ignore the
        command (making no response at all).

    CWD\040PATH\0377\0375NAME\r\n

        The correct parse yeilds the parameter PATHAME (note the N is
        missing).  This is because the TELNET protocol requires WILL
        WONT DO and DONT to be followed by a single-byte option code.
        The fact that IAC DO 'N' is not a valid TELNET command is
        immaterial.  Why assume the N is part of the TELNET command,
        and not data?  Because, by the rule of being liberal in what
        we accept, we accept ANY single-byte following IAC DO to be
        part of some (possibly new) TELNET option negotiation.  To
        assume otherwise would break compatability on the control
        connection with future TELNET options.

        (Note that I chose IAC DO ECHO specifically.  It shows _why_
        the FTP SHOULD NOT honor and implement all of the TELNET
        options.  To do so would imply that characters received by the
        FTP server following acceptance of the option would be echoed
        on the return data stream.  This would make the responses
        difficult to separate from command echos.  The server has no
        way of knowing whether the client can handle the echoed
        characters.  By the principle of being conservative in what
        we send, the server MAY accept IAC DO ECHO and refuse the
        option by sending IAC WONT ECHO, or MAY ignore IAC DO ECHO,
        but the server SHOULD NOT accept the option by sending IAC
        WILL ECHO.)

    CWD\040PATH\0377NAME\r\n

        The correct parse yields the parameter PATHAME (again, the N
        is missing).  The TELNET protocol requires IAC to be followed
        by a command-code byte.  The rule of being liberal in what we
        accept applies here just as it did for IAC DO 'N'.  We don't
        know that 'N' is not a new, unknown TELNET command and must
        accept it as such.

The discussion so far has assumed only ASCII characters.  The addition of
RFC 2640 complicate matters only slightly.  Prior to RFC 2640, the fact that
TELNET commands have a higher precidence could safely be ignored.

Servers implementing 2640, however, MUST interpret the data stream using the
precidence order:

 - TELNET (highest precidence)
 - Character set recognition (ie., UTF8)
 - End-of-line recognition (ie, \r\0 and \r\n)

The reason is that character set encoding (such as UTF8) add "shift state".
While shifted, bit sequences which appear identical to end-of-line sequences
are NOT end-of-line sequences.

As an example of a simple character encoding scheme (which my server could
advertize as FEAT X-CHARACTER-LITERAL), I will use matching single quote (')
characters to "shift" into my alternate character set.  For this character
encoding scheme we'll use ANSI/ISO C character literal formatting, accepting
all bytes as data, except \ and '; following all requirements for ANSI/ISO C
character literals but accepting all invalid \ escape sequences by dropping
the \ (just as many compilers do).  (Remember, however, that I am already
using C notation, so if I want to show the \ character as appearing "on the
wire" I will note it as \\.)

    CWD\040'\040PATH\r\0NA\r\n'ME\r\n

        The correct parse yields the parameter '\040PATH\r\0NA\r\n'ME.
        This is because the character set "shift" state has higher
        precidence than end-of-line recognition.

        Note that the parse accepts the \0 as data.  This does NOT
        violate the rules stated in RFC 2640.  At this point, we only
        have a "parameter" token.  The \0 may, or may not, represent
        an invalid character because the token has not yet been
        converted into a pathname.  The conversion from token to
        pathname is unspecified (and implementation defined).  The
        restriction in RFC 2640 is that the _pathname_ (the token,
        after conversion) MUST NOT contain \0 is in deference to the
        problems faced by ANSI/ISO C programs when dealing with this
        character.  Note that RFC 2640 specifies the use of UTF8, and
        that \0 will NEVER appear in any "shifted" character encoding
        under UTF8 except the single, unshifted (ASCII) character NUL.

        My hypothetical server, therefore, would accept the above
        as the command-token CWD followed by the parameter-token
        '\040PATH\r\0NA\r\n'ME.  During sematic interpretation, it
        would find the parameter-token for CWD is a pathname and
        attempt a conversion.  At this point, my server could issue
        an error response since the character \0 appears in a pathname
        in violation of RFC 2640.  By the rule of being liberal in
        what we accept, however, if the implementation can handle \0
        in a pathname token, my server could accept it and proceed to
        attempting to execute the CWD action with the (now converted)
        pathname.

    CWD\040'\040PATH\377\375\001NA\r\n'ME\r\n

        The correct parse here is '\040PATHNA\r\n'ME.  This is because
        the TELNET protocol has higher precidence than character
        set recognition.

    CWD\040'\040PATH\r\0NA'\r\0ME\r\n

        The correct parse here is '\040PATH\r\0NA'\rME.  The second \r\0
        occurred in the unshifted state, an is therefore interpreted at
        the end-of-line recognition level of precidence.

Prior to RFC 2640, the precidence rules were immaterial.  The two sets of
rules did not intersect and there was no ambiguity.

The only possible issue is whether RFC 2640 (or the MLST draft) should
exlicity state the precidence rule.

My feeling is the fact that the precidence rule MUST exist is obvious to
anyone seriously setting out to implement the FTP, and there is no need in
the MLST draft to explictly state so.  Perhaps, if we ever do a complete
replacement for RFC 959, merging it and all subsequent RFCs into a single
document, we could add the explicit statement in deference to those who,
after being hit with a 2x4, refuse to acknowledge the presence of a 2x4
because it was stated to be present.