Subject: Fwd: I-D ACTION:draft-ietf-fax-esmtp-conneg-03.txt

From: Dave Crocker <[email-masked]>
Date: Mon, 05 Aug 2002 08:47:53 -0700

Folks,

The SMTP Conneg draft has been revised, with the primary change being 
addition of two Appendices.  One discusses use of the option when there are 
multiple recipients.  The other discusses use of the option when there is 
relaying.

d/

>A New Internet-Draft is available from the on-line Internet-Drafts 
>directories.
>This draft is a work item of the Internet Fax Working Group of the IETF.
>
>         Title           : SMTP Service Extension for Content Negotiation
>         Author(s)       : K. Toyoda, D. Crocker
>         Filename        : draft-ietf-fax-esmtp-conneg-03.txt
>         Pages           :
>         Date            : 02-Aug-02
>
>This document defines a content negotiation service
>extension for SMTP [ESMTP1, ESMTP2] whereby an SMTP
>client may request information about content
>capabilities of the target device or system that is
>serviced by an SMTP server.  The SMTP server may report
>the target's content capabilities back to the client.
>This process emulates a classic facsimile start-of-
>session capabilities negotiation, although it can be used
>for a broad range of email-based scenarios.  This service
>extension is primarily intended for 'direct', one-hop,
>originator/recipient SMTP transfers, although relayed
>scenarios through multiple SMTP servers are permitted.
>
>A URL for this Internet-Draft is:
>http://www.ietf.org/internet-drafts/draft-ietf-fax-esmtp-conneg-03.txt

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

---

From: Keith Moore <[email-masked]>
Date: Mon, 05 Aug 2002 15:19:46 -0400

[bcc'ed to iesg - on the assumption that they might not want to see the 
resulting replies]

> multiple recipients.  The other discusses use of the option when there is 

I think the two appendices help slightly.  However they also introduce 
new problems not present in the previous draft.  Also, the new version of 
the document fails to correct a number of problems (some of them serious) 
in the previous version.

In particular,

- it is still overbroad, claiming applicability for general email
  when it really works only for a limited subset of content 
  which has an obvious ordering relation for "quality".   
  perhaps illustrating this, all of the examples are for fax.

- it still permits unauthorized parties to make arbitrary and
  unspecified alterations to mail.  

- worse, the new version permits unauthorized parties to make
  arbitrary assertions of recipient capabilities in the absence
  of explicit information.

- the new version also permits multiple conversions to take place,
  potentially degrading the message considerably more than a
  single conversion.

- it still mandates use of particular reply and status codes in cases
  where they are not always appropriate

- it still has the server reporting status codes for conditions that
  are best detected on the client

- it still allows SMTP servers to make assertions of recipient capabilities
  without actual knowledge of those capabilities.

- I still think overloading of RCPT is a pessimal way to do this
  since the command will often need to be issued twice anyway -
  either because of a mixture of recipient capabilities, or because
  the conversion could not be performed on-the-fly.

recommendations for near-term publication:  

- this feature should be renamed and the document reworded to make it 
  clear that it is fax-specific rather than general purpose.  

- misleading text should be fixed, unworkable recommendations should be
  removed.

- the document should be published as experimental rather than on the 
  standards track, since it doesn't meet the requirements for proposed 
  standard.

things that need to be fixed before the document should be considered
as standards-track:

- needs to be examined for a wide range of content

- needs to have a well-defined protocol by which senders can request
  conneg "convert-or-fail" semantics on a per-message, and probably
  per-bodypart, basis.   if they are not requested, such semantics
  must not be applied.

- needs to explicitly specify which conversion algorithms are permissible
  and how new conversion standards may be added, since there can be 
  multiple ways to perform a conversion and some are worse than others.

- needs to state that conneg features are only advertised with the 
  recipient's consent and control

- needs to state exactly which parties are authorized to perform
  conversion.

- needs to prevent multiple conversions.

some details are below.  not everything above can be seen from a close-up
view, so the list of details below should be considered separately from
the overall problems listed above.

> 1.   INTRODUCTION
> 
>      When a data source and a receiver have interactive access to 
>      each other, the receiver often informs the source of its 
>      capabilities, to permit optimized performance or functionality for 
>      the interaction.  Classic telephone-based facsimile is an example, 
>      as are voice over IP and ESMTP, among Internet applications.  The
>      store-and-forward nature of Internet mail is usually assumed to 
>      preclude such capabilities exchanges, although the sender in a 
>      store-and-forward scenario could benefit from knowing precise 
>      details about the receiver.  In some configurations, direct 
>      email-based interactions -- with the originating ESMTP client 
>      and the destination ESMTP server able to have direct TCP 
>      connect -- are possible, such as over an intranet. In addition 
>      an end-to-end exchange can ESMTP for hop-by-hop enforcement.   

nit: s/can/can use/ ???

problem: this proposal doesn't provide any way to "enforce" content
negotiation past a single hop.  there is no requirement that the CONNEG
option be relayed from one hop to another.  even if there were, it would
be problematic - it would require earlier SMTPs to advertise capabilities
of recipients for which they did not have information, and it would 
require (and trust) later SMTPs to perform arbitrary and undefined 
conversions on content to satisfy recipient-specific constraints.

>      This document defines an SMTP-based service extension [ESMTP1, 
>      ESMTP2] for content negotiation, whereby an SMTP client may 
>      request information about content capabilities of the target 
>      device or system that is serviced by an SMTP server.  

nit: the capability request is on a per-recipient-mailbox basis rather
than a 'target device or system' basis.  of course being able to request
per-recipient-mailbox capabilities is sufficient to request target
device capabilities when all recipients accessed via a single SMTP server 
have the same capabilities.  but the fact that it's a per-recipient
capability makes the proposal more complex to handle for the sender,
and it would be better to make this clearer from the start.

> The SMTP 
>      server may report the target's content capabilities back to the 
>      client. This process can be used to emulate a classic facsimile
>      start-of-session capabilities negotiation, as well as being used 
>      for other email-based services.  The extension is primarily 
>      intended for "direct" SMTP transfers, although relayed scenarios 
>      are permitted through a series of SMTP servers and are discussed 
>      in Appendix B.
> 
> 
> 2.   CONVENTIONS
>      
>      In examples, "C:" and "S:" indicate lines sent by the
>      client and server respectively.
>      
>      The key words "MUST", "MUST NOT", "SHOULD", "SHOULD
>      NOT", and "MAY" in this document are to be interpreted
>      as defined in "Key words for use in RFCs to Indicate
>      Requirement Levels" [KEYWORDS].
> 
> 
> 
> 3.   CONTENT NEGOTIATION SERVICE EXTENSION
>      
>      (1)  The name of the SMTP service extension is
>           "Content_Negotiation"
> 
>      (2)  The EHLO keyword value associated with this extension
>           is "CONNEG"
> 
>      (3)  A parameter using the keyword "CONNEG" is added to the
>           RCPT command
> 
>      (4)  The server responds with a report of the content
>           capabilities of the device or system that embodies the
>           target RCPT address.
> 
> 
> 
> 4.   CONNEG PARAMETER TO RCPT
>      
> 4.1  Parameter
> 
>      Keyword:
>           
>           CONNEG
>      
>      Argument:
>           
>           REQUIRED
>             The client requires support for the capability. If the 
>             target does not support the CONNEG parameter, the target 
>             MUST reject the RCPT command with a 504 reply.
> 
>             If the target can not support the capability due to a 
>             temporary problem, it MUST reject the RCPT command with 
>             a 404 reply. 

nit:  It is not clear whether it is desirable for the server to give 
CONNEG errors priority over all other sources of error.   For instance,
if a recipient address is invalid, reporting *that* probably results in
a more useful error message than reporting that the CONNEG feature is
not supported for that particular recipient.  

suggestion:  reporting of errors with the CONNEG=REQUIRED option SHOULD 
only occur if the RCPT command would otherwise have a successful result.  
If the RCPT command would fail for other reasons, the error and status
codes should reflect the nature of that failure.  In most cases this will
result in a more useful error message.   An reasonable exception might 
exist if the server checks several potential error conditions concurrently, 
and it determines that the command would fail the CONNEG=REQUIRED criteria
before it determines that the command would fail due to other reasons.

>           OPTIONAL
>             The client requests the target to use the capability. If 
>             the target can not support the capability at this time, the 
>             target MUST process the address and message as if the 
>             requested CONNEG capabilities had not been specified.
>             
>           If the argument does not exist, the default is "REQUIRED".
>           
>           When a capability is REQUIRED by the client but can not 
>           currently be supported by the target, an error response will 
>           have significant performance impact to overall SMTP 
>           processing.  Use of the OPTIONAL parameter will ensure high 
>           SMTP performance.

nit: s/ensure high SMTP performance/facilitate better SMTP performance/

even better would be to say that REQUIRED SHOULD only be used when the
sender has specified that the message delivery should fail unless it
is known that the recipient has the necessary capability.

even better would be to omit REQUIRED entirely, since it increases the
complexity of the server implementation and the protocol, for essential
zero benefit to the client.  REQUIRED doesn't remove the need for the
client to detect errors due to content negotiation, since it is always
possible that the server/recipient does not support any way of handling
the content at all, or even if such conversion might be possible, the 
client does not have the capability to perform that specific conversion.  

> 4.2 Client Action
>           
>      If the server issued a 250-CONNEG, as part of its
>      EHLO response for the current session, the client MAY
>      issue the CONNEG parameter with RCPT.
>           
>      If the client issues the CONNEG parameter with
>      RCPT, then it MUST honor the capabilities specified
>      in the CONNEG RCPT reply, and transform data that is
>      sent, so that the server can accept the data. The
>      client SHOULD transform the data to the "highest" level
>      of capability of the server.

problem: the client cannot be compelled to perform transformations that
are impossible or for which the client lacks the capability.  no client
can reasonably convert video to audio. few clients could convert audio
or fax to text, and then only if the audio or fax contained words.
if taken literally this provision would prevent any client from asserting 
CONNEG.

suggestion: rather than saying "MUST honor" say that the client
MUST NOT send forward which is inconsistent with the recipient's
stated capabilities.  the client SHOULD, if capable, perform 
transformations which are likely to preserve the recipient's ability
to use most of the information in the content.  if the client is unable
to perform such transformations, it MUST bounce the message.

(there probably needs to be some interaction with critical content
here - it might be acceptable for a SMTP to discard conntent marked
as non-critical rather than bouncing the message.  but under no
circumstances should the message be delivered to the recipient when
the recipient has stated via CONNEG that he/she cannot receive it)

>      If the server rejects the RCPT command with a 404 reply, the 
>      client may later reissue the RCPT with the CONNEG parameter
>      in a different SMTP session.

nit: presumably "may" should be capitalized here.

>      If the server returns an EHLO 250 code without CONNEG 
>      capabilities, the client MUST NOT issue a CONNEG parameter 
>      with RCPT.
> 
>      Methods of using of this option with multiple addressees, for 
>      the same content, are discussed in Appendix A.
> 
>      
> 4.3 Server Action
>           
>      If the client specifies "CONNEG=REQUIRED" in the RCPT, 
>      but the server does not support the CONNEG parameter, the server
>      MUST reject the RCPT command with a 504 reply. 
>      
>      If the server supports the CONNEG parameter, but can not return 
>      the recepient's capability temporarily, the server MUST reject the 
>      RCPT command with a 404 reply. For example, if the server gets 
>      the capability information from a directory, but its connection is 
>      offline, the server MUST reject the RCPT command with a 404 
>      reply. 
> 
>      If the client specifies "CONNEG=OPTIONAL" in the RCPT, 
>      but the server does not support the CONNEG parameter or can not 
>      return the recipient's capability temporarily, the server MUST 
>      process the address and message as if the requested CONNEG
>      capabilities had not been specified.
> 
>      Regardless of the value of the parameter, if the server does 
>      support the CONNEG parameter and the address is acceptable, 
>      then it MUST issue a 250 reply, followed by the capabilities
>      of the server that is specified by  the RCPT address.
>      Successful responses to CONNEG RCPT requests will always be
>      multiple SMTP lines.  The first line is the normal RCPT response,
>      and subsequent lines beginning with the exact string
>      "250-CONNEG " and "250 CONNEG " are the CONNEG responses.
>      The last line begins with "250 CONNEG ".  
> 
>      If the SMTP server supports ENHANCEDSTATUSCODES [RFC1893], the 
>      response strings for a success are "250-2.1.5 CONNEG" and 
>      "250 2.1.5 CONNEG". 

nit: I think you want these to be "250-2.1.5 CONNEG " and "250 2.1.5 CONNEG ".

nit: it is out of scope for this document to specify the status codes for
a successful RCPT command.

>      The response strings for indicating a
>      permanent failure are "504-5.3.3 CONNEG" and "504 5.3.3 CONNEG". 
>      The response strings for a temporary failure are "404-4.3.3 
>      CONNEG" and "404 5.3.3 CONNEG".

problem: this overloads existing status codes for new error conditions.
there needs to be new status codes for "sender insisted on not delivering
the message unless it knew for sure that the recipient could deal with
it and the recipient's capabilities could not be obtained".

>      All CONNEG-capable clients and CONNEG-capable servers MUST
>      be able to successfully process CONNEG lines that are up to 512
>      characters long, as required by RFC2821. If the length of CONNEG
>      lines is greater than 512 characters, the server MUST insert
>      line breaks and make next CONNEG line. 

problem: you need to explain more about how this is done.  e.g. can
"<CR><LF>250-2.1.5 CONNEG " be inserted in the middle of a token?
are spaces at the end of a reply line significant?

>      The contents of the capability listing MUST conform to 
>      the specifications in "Content Feature Schema for Internet Fax". 
>      [RFC2879]

problem: this restriction seems inappropriate if the content is not a fax.

> 
> 
> 5.   SYNTAX
>      
>      Command with "CONNEG":
>       "RCPT TO:" ("<Postmaster@" domain ">" / "<Postmaster>"
>                    / Forward-Path) (SP "CONNEG =" ("REQUIRED" /
>                  "OPTIOANL") CRLF

nit: OPTIONAL is misspelled

nit: "CONNEG =" should be "CONNEG="

nit: "<Postmaster@" domain ">" is redundant with Forward-Path

problem: grammar excludes other RCPT options

suggestion: the RCPT command syntax should not be formally described 
in this document; it is adequately described elsewhere and CONNEG
doesn't change the syntax.

>      Reply:
>       ( ("250-" CRLF) *("250-CONNEG" capability CRLF)
>         ("250 CONNEG" capability CRLF) )/
>       ( ("250-2.1.5" CRLF) 
>           *("250-2.1.5 CONNEG" capability CRLF)
>            ("250 2.1.5 CONNEG" capability CRLF) )/
>       ("504" CRLF) /
>       ("504 5.3.3" CRLF) /
>       ("404" CRLF) /
>       ("404 4.3.3" CRLF) /
> 
>       capability  =  <<as per [RFC2879]>>

problem: this grammar excludes the possibility of other kinds of 
responses besides CONNEG.

problem: need to specify how line wrapping works.

radical suggestion: 

1. server encodes the recipient's conneg string in quoted-printable, with 
the line length limits and wrapping rules per the MIME spec.

2. server then prefixes each of the resulting quoted-printable lines with 
 
"250-2.1.5 CONNEG "

(perhaps adding 2.1.5 even if ENHANCEDSTATUSCODES are not supported)

3. client collects all lines from the response that begin with
"250-2.1.5 CONNEG ", removes that prefix from each line, and 
processes the resulting set of lines as a quoted-printable encoded 
file in order to obtain the conneg string for the recipient.

> 
> 6.   EXAMPLES
> 
> 6.1  Success Response
> 
>      An example of ESMTP sequence with successful RCPT response
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED

nit: should be "CONNEG=REQUIRED"

>      S: 250-<[email-masked]> recipient ok
>      S: 250 CONNEG (&(image-file-structure=TIFF-minimal)
>      S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
>      S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
>      S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
>      S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
>      S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
>      S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
>      S: (ua-media=stationery) )

nit: example shows inappropriate line wrapping, without explanation.

>      C: DATA
>      
>      S: 354 okay, send data
>      
>      C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
>           Per:
>           (  image-file-structure=TIFF-minimal
>              dpi=400
>              image-coding=JBIG
>              size-x=2150
>            )
>         >>

nit: example shows invalid content-type

nit: example assumes fax.

>      S: 250 message accepted
>      
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 6.2  Long Response
> 
>      An example of successful RCPT response when the length of 
>      capability is greater than 512 characters. 
> 
>      S: 250-2.1.5<[email-masked]> recipient ok
>      S: 250-2.1.5 CONNEG (&(image-file-structure=TIFF-minimal) ...
>      S: 250-2.1.5 CONNEG   .....
>      S: 250 2.1.5 CONNEG   (color=Binary)

okay, that helps, but it still doesn't show whether line wrapping
can occur in the middle of a conneg token.

> 
>      An example of succssful RCPT response when CONNEG-capable 
>      server supports ENHANCEDSTATUSCODES.
> 
>      S: 250-2.1.5<[email-masked]> recipient ok
>      S: 250 2.1.5 CONNEG (&(image-file-structure=TIFF-minimal)
>      S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
>      S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
>      S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
>      S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
>      S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
>      S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
>      S: (ua-media=stationery) )

nit: example assumes fax.

> 
> 6.3  Failure Response
> 
>      An example of ESMTP sequence with parmanent failure RCPT 
>      response. 
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED

nit: should be "CONNEG=REQUIRED"

>      
>      S: 504 <[email-masked]> recipient ok
> 
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 6.4  Temporary Failure Response
> 
>      An example of an ESMTP sequence with temporary failure RCPT 
>      response when the value of parameter is "REQUIRED":
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED

nit: should be "CONNEG=REQUIRED"

>      
>      S: 404 <[email-masked]> recipient ok
> 
>      C: QUIT
>      
>      S: 221 goodbye
>      .
>      .
>      .
>      retry according to implementation
> 
> 
> 6.5  Temporary Failure with Optional handling
> 
>      An example of an ESMTP sequence with temporary failure RCPT 
>      response when the value of parameter is "OPTIONAL":
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = OPTIONAL

nit: should be "CONNEG=OPTIONAL"

>      
>      S: 250 <[email-masked]> recipient ok
>      
>      C: DATA
>      
>      S: 354 okay, send data
>      
>      C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
>           Per "A Simple Mode of Facsimile Using Internet Mail"
>           RFC2305
>         >>

nit: example assumes fax

>      
>      S: 250 message accepted
>      
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 
> 7.   IANA CONSIDERATIONS
>      
>      On publicatiom of this document by the RFC Editor, the IANA 
>      shall register the Content_Negotiation ESMTP extension defined 
>      in section 3.
> 
> 
> 
> 8.   SECURITY CONSIDERATIONS
>      
>      This ESMTP option calls for a respondent to disclose
>      its capabilities.  Mechanisms for determining the
>      requestor's authenticated identity are outside the
>      scope of this specification.  It is intended that this
>      mechanism permit disclosure of public information;
>      hence there is no particular need for security
>      measures.
>      
>      However there is nothing to prevent disclosure of
>      sensitive information that should receive restricted
>      distribution.  It is, therefore, the responsibility of
>      the disclosing ESMTP server to determine whether
>      additional security measures should be applied to the
>      use of this ESMTP option.
> 
>      A man-in-the-middle attack might change the capabilities reported 
>      for a given recipient. For example: Suppose the sender knows the 
>      recipient has the ability to view color documents so they mark 
>      some things in red in what is otherwise a black and white 
>      document. But someone interferes with the returned capabilities, 
>      indicating that the recipient only supports black and white. The 
>      document is duly downgraded, with the result that the recipient 
>      doesn't see what the sender marked. 
>      
>      An indirect exposure can occur when the report of a capability 
>      implies use of specific software.  If that software is known to 
>      have security weaknesses, the capabilities report effectively 
>      advertises the associated opportunity to exploit the security 
>      weakness.
>      
>      For target SMTP servers that require security mechanisms to be in 
>      force at the start of the session, the target SHOULD refrain from 
>      including the CONNEG parameter in an EHLO response until the 
>      requisite security mechanisms are in force.
> 
>      For digitally signed content, the use of this option poses a
>      special challenge.  Digitally signing content relies on that
>      content to be in a particular form.  Use of this option changes
>      that form.  Hence an SMTP client that uses this option on
>      digitally signed content MUST be able to recompute the digital
>      signature of the content. 
> 
> 
> 
>  9.  ACKNOWLEDGEMENTS
>      
>      Graham Klyne provided useful suggestions to an earlier
>      draft.
> 
> 
> 
> 10.  NORMATIVE REFERENCES
>      
>      [ESMTP1]  Klensin, J., Freed, N., Rose, M., Stefferud,
>                E. and D. Crocker, "SMTP Service Extensions",
>                RFC 1869, November 1995
>      
>      [ESMTP2]  Klensin, J., "Simple Mail Transfer Protocol",
>                RFC 2821, April 2001.
>      
>      [RFC2879] McIntyre, L. and G. Klyne, "Content Feature
>                Schema for Internet Fax", RFC 2531, August
>                2000
> 
>      [RFC2305] Toyoda, K., Ohno, H., Murai, J. and  D. Wing, 
>                "A Simple Mode of Facsimile Using Internet Mail", 
>                RFC 2305, March 1998.
> 
> 
> 
> 11.  AUTHORS' ADDRESSES
>      
>      Kiyoshi Toyoda
>      Matsushita Graphic Communication Systems,Inc
>      2-3-8 Shimomeguro, Meguro-Ku
>      Tokyo 153  JAPAN
>      
>      +81.3.5434.7161
>      [email-masked]
>      
>      
>      Dave Crocker
>      Brandenburg InternetWorking
>      675 Spruce Drive
>      Sunnyvale, CA  94086  USA
>      
>      +1.408.246.8253
>      [email-masked]
> 
> 
> 
> 12.  FULL COPYRIGHT STATEMENT
>      
>      Copyright (C) The Internet Society (2001).  All Rights
>      Reserved.
>      
>      This document and translations of it may be copied and
>      furnished to others, and derivative works that comment
>      on or otherwise explain it or assist in its
>      implementation may be prepared, copied, published and
>      distributed, in whole or in part, without restriction
>      of any kind, provided that the above copyright notice
>      and this paragraph are included on all such copies and
>      derivative works.  However, this document itself may
>      not be modified in any way, such as by removing the
>      copyright notice or references to the Internet Society
>      or other Internet organizations, except as needed for
>      the purpose of developing Internet standards in which
>      case the procedures for copyrights defined in the
>      Internet Standards process must be followed, or as
>      required to translate it into languages other than
>      English.
>      
>      The limited permissions granted above are perpetual and
>      will not be revoked by the Internet Society or its
>      successors or assigns.
>      
>      This document and the information contained herein is
>      provided on an "AS IS" basis and THE INTERNET SOCIETY
>      AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL
>      WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT
>      LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
>      HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
>      WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
>      PARTICULAR PURPOSE.
> 
> 
> 
> APPENDIX
> 
> A.   USAGE FOR MULTIPLE RECIPIENTS
> 
>      ESMTP permits specification of multiple recipients for the same
>      content.  This option specified here can produce information that
>      different recipients have different content capabilities.  How 
>      this differential information is used will depend upon the 
>      requirements of the sender.  This specification does not mandate 
>      particular choices. Three scenarios are possible:
> 
>      (1) Single Recipient
>      
>      For each recipient, conduct a separate ESMTP session.  This
>      ensures that each content transfer can be tailored to the
>      capabilities of each recipient.  This also consumes the largest
>      amount of bandwidth and the largest number of cross-network SMTP
>      protocol exchanges
> 
>      (2) Common Content
> 
>      For a single ESMTP session, issue RCPT commands that obtain
>      content capabilities information for each recipient.  With the
>      DATA command, send the best content form that can be processed by
>      ALL of the recipients.  Some recipients will receive content that
>      is below their best capabilities.  However this approach also 
>      consumes the least bandwidth and has the fewest cross-network 
>      protocol exchanges

problem: this scenario does not work because the sender has no way to know a 
priori whether it is possible to produce a common form that is acceptable
to all recipients.  so any sender that sends multiple RCPT commands in a
single session MUST be prepared to terminate the session (either via RSET
or QUIT) and retry sending only to those recipients that DO have a common 
format.

suggestion: delete scenario 2, as it is not workable. 
 
>      (3) Partial Batching
>      
>      This scenario begins the same as the "Common Content" scenario.
>      Content capabilities information is obtained during a single ESMTP
>      session, with all of the RCPT commands issued together.  The
>      difference for this scenario is that the client SMTP then
>      terminates the session with an RSET and begins one or more new,
>      separate sessions.  Addressees are processed in batches, according
>      to the similarity of their capabilities.  This option balances
>      consumption of bandwidth with optimization of content, at the cost 
>      of a terminated session. 
> 
> 
> B.   SCENARIOS FOR RELAYING
> 
>      ESMTP is a direct transfer mechanism, using a single TCP 
>      connection.  It supports Internet mail store-and-forward through 
>      such characteristics as global addressing and ESMTP enforcement of 
>      global features.  

nit: I'm not at all sure what this last sentence means.

> Use of the ESMTP Conneg option is 
>      straightforward when the originating ESMTP client is able to 
>      directly connect to the destination ESMTP server.  The nature of 
>      this option is to return address-specific information that will 
>      affect content transmission.  Hence the use of this option in the 
>      presence of the in-direct effects of store-and-forward is not 
>      obvious.
>      
>      This Appendix discusses some styles of use for the ESMTP Conneg 
>      option when messages are being relayed.  The Appendix is intended
>      only to provide discussion.  It is neither intended to be 
>      exhaustive nor restrictive.  Other scenarios are likely and 
>      encouraged.
>      
>      
> B.1  Relay Server Modes
> 
>      The key challenge for use of the ESMTP Conneg option is the 
>      requirement the receiving ESMTP server be able to return 
>      capabilities on behalf of a target addressee.  This requirement 
>      translates into two, basic styles of operation for the receiving 
>      ESMTP server:
>      
>      (1) Server Knows Addressee Capabilities
>      
>      Here the receiving server has the necessary details about an 
>      addressee, at the time the sending server issues the RCPT 
>      response.  Typically, this information will be obtained through a 
>      direct, real-time query mechanism, either to a directory 
>      containing addressee information or to a service run directly on 
>      behalf of the individual addressee and possibly on the addressee's 
>      system.  
>      
>      One form of query to the addressee's system is to stack a 
>      cascading sequence of ESMTP sessions and RCPT commands 
>      together, all the way to the destination ESMTP server. Although 
>      theoretically possible, this attempt to turn a multi-hop scenario 
>      into a real-time, pseudo-direct query is not practical.  It 
>      will most likely result in response delays for the RCPT command 
>      that are not acceptable, particularly over the public Internet.
>      
>      (2) Server Ensures Conversion
>      
>      Fundamental to the use of a capabilities exchange is the 
>      requirement that the receiver of capabilities information be able 
>      to convert content into a form that is more capable than the 
>      default form, when the receiver indicates that it can support the 
>      superior form.  That is, use of this option presumes that the 
>      sender is holding a "more capable" form of the content and will 
>      map it to a "less capable form" if the receiver does not support 
>      the superior form.  

problem: there is inherently no way for the sender to know in advance 
whether it can convert the message in a form that is  suitable for the
recipient.

problem: for most kinds of content there is not an obvious ordering
which determines "more capable" or "less capable".  fax images containing
text are not inherently more or less capable than text - each has different
benefits and drawbacks.  HTML is not inherently more or less capable than 
PDF.   it depends on how the recipient will use the message.  so the 
assumption by which relaying is justified is not valid.

>      Hence another style of relaying configuration is to have a relay 
>      SMTP respond to an RCPT capability query by indicating that it 
>      supports the most capable form.  The sender will pass the best 
>      version it can.  The receiver has then taken responsibility for 
>      performing later conversions, as necessary, to the next hop in the 
>      sequence.  That is, the receiving ESMTP server inherits the same 
>      level of responsibility already being held by the sending ESMTP 
>      client.

problem: this is unworkable unless the SMTP server already knows the
recipient's capabilities, since without knowing those capabilities
the SMTP server cannot be sure that it can convert the message to
something the recipient can accept.  and if the SMTP server already
knows the recipient's capabilities, there is no need for it to lie 
about them.

problem: this introduces the possibility of multiple conversions.  multiple 
conversions can cause considerable lossage due to interference between the 
conversions - e.g. aliasing when audio or image sampling rates are changed, 
interference caused when artifacts in the the output of one audio codec 
appear in  the input of another.

problem: this provision almost certainly allows SMTPs to make assertions
about recipient capabilities without the permission of the recipient.

> B.2  Some End-to-End Scenarios
> 
>      This section suggests methods of using ESMTP Conneg for achieving 
>      an end-to-end service that uses knowledge of recipient 
>      capabilities for modifying the content or the handling of the 
>      content.  This section is intended only to explore possible 
>      scenarios.  Others are feasible and likely. The choice of scenario 
>      to support will depend upon particular service policies chosen for 
>      a relay.  This specification provides no constraint or guidance 
>      about which policies to choose.
> 
>      (1) Basic Tranfer betweeen Organizations
>      
>      Again noting that alternative configurations and support 
>      environments are permitted, a simple example of combining the two 
>      modes of receiving server style can be helpful.  The scenario to 
>      consider is sending from one organization's email system to 
>      another organization's email system, across the Internet.  
>      
>      This scenario presumes two, independent email services, one for 
>      each organization.  In this simple example, the user has a local 
>      ESMTP server and it talks with the organization's Internet email 
>      gateway.  Hence there are four ESMTP servers.  The originating 
>      server takes the message from the originating user.  It then 
>      relays the message to the originating organization's ESMTP 
>      gateway.  This gateway, in turn, relays the message to the 
>      destination organization's gateway, which finally delivers it to 
>      the recipient's server.
>      
>      In this simple example, the originating ESMTP server includes a 
>      CONNEG option on the RCPT command 

problem: for the originating server to include CONNEG on outgoing mail
without any per-message authorization from the sender presumes that it 
is never desirable to deliver mail unless the recipient capabilities are 
known.  this is a radical departure from current practice and is probably 
not feasible.  

if the example assumes that the organizations have a mutual agreement to 
support CONNEG for all recipients and the outgoing server only asserts 
CONNEG when sending to recipients at that organization, this needs to be
stated explicitly.

>      to the originating 
>      organization's Internet gateway and the gateway always responds 
>      that it supports the highest capabilities.  It is then given the 
>      most capable form of the content.
>      
>      The originating gateway then performs a RCPT CONNEG exchange with 
>      the destination organization's gateway.  This gateway can operate 
>      in the same "Ensure Conversion" mode as the originating gateway, 

problem: no it cannot, at least not without more assumptions than are stated.

>      or it can perform a real-time query about the addressee's 
>      capabilities.  The former mode defers resolving recipient 
>      capabilities until the final step.  The real-time query requires 
>      both that the query mechanism be timely, in order to avoid RCPT 
>      response delays, and that it be accurate, with correct information 
>      about the addressee.
>      
>      The final exchange is, of course, "direct".  The recipient's ESMTP 
>      server is presumed to have easy access to the necessary 
>      capabilities information.
> 
>      (2) Integration with Multipart/Alternative
>      
>      Basic support for Conneg will typically involve a Conneg client 
>      that has a single, high-quality version of content and then maps 
>      is "down" to the best quality that can be supported by the 
>      responding Conneg server.  That is, the sending system will do a 
>      conversion.  In another mode of operation the Conneg client 
>      already holds a number of different mappings of content at 
>      different levels of quality.  The client will use the Conneg 
>      response to choose the content quality that is "closest" to the
>       capabilities of the receiver.
>      
>      MIME's Multipart/Alternative provides a means for a content 
>      originator to send a fixed set of content quality choices.  An 
>      ESMTP relay in the sequence can then choose to use Conneg as a 
>      means of selecting one of the alternatives to transmit, rather 
>      than transmitting the full set.  Such a scenario is particularly 
>      useful when the communication path changes from high-bandwidth to 
>      low.
>      
>      (3) Content Staging on Retrieval Server
>      
>      Rather than converting content to a lower quality, an ESMTP Conneg 
>      client might choose to use a Conneg response for choosing not to 
>      send the content directly.  It might, instead, remove the content 
>      from the message and store the content on a retrieval server (for 
>      access through HTTP, FTP or the like.)  It can then put a citation 
>      into the message, which points to the stored content.
>      
>      For the retrieval server, access security, as well as life-time 
>      management of the content on the retrieval server, should be 
>      considered. A full discussion of these considerations is out of 
>      the scope of this memo."
> 
>      (4) Alternate Addressing
>      
>      If an ESMTP Conneg client has access to multiple addresses for the 
>      same recipient, it might use Conneg to determine which is most 
>      capabable for particular content and send the content to that one.
>      
>      This scenario requires a number of infrastucture features for 
>      which there currently are no standards.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 06 Aug 2002 01:21:23 -0700

Keith,

Responding to those items that I can:

>   which has an obvious ordering relation for "quality".

Please feel free to provide real-world examples of usage that is likely to 
be attempted and that will not work properly.

>      perhaps illustrating this, all of the examples are for fax.

You believe that making examples be for something other than fax will 
materially affect the adequacy of the normative specification?

>   unspecified alterations to mail.

Sorry, no.  It makes no alterations to the power of intermediaries.  It 
gives them better information, not different capabilities.

In any event, the "permissions" assigned to intermediaries are a matter of 
administration policy, not technical specification.  If an administrative 
authority does not want to permit use of this option, it will not use it.

>   arbitrary assertions of recipient capabilities in the absence

That is not what the discussion says.  It describes a way of having the 
receiving SMTP server take on the responsibilities for matching 
capabilities that already reside with the sending SMTP server.

Remember that these servers about which you are so suspicious already have 
"control" over the semantics of the recipient's address.  You can't get 
more powerful than that.

And, by the way, please remember that the appendix is not normative.  It 
does not change the specification.  It "permits" and "prohibits" nothing.

>   without actual knowledge of those capabilities.

What is the problem this causes?  How does it make this service fail?

>- I still think overloading of RCPT is a pessimal way to do this

A number of separate analyses of the design choices suggest pretty strongly 
that a separate command is actually worse than an RCPT parameter.

A separate command changes the SMTP state machine.  That is not a minor 
thing.  An example of the impact of this is probable failure in getting 
through some firewalls.

>   since the command will often need to be issued twice anyway -

Your use of the word "often" is contrary to the likelihoods expected by others.

>   clear that it is fax-specific rather than general purpose.

There is nothing in the normative text that is fax specific.

>- misleading text should be fixed, unworkable recommendations should be

Please offer whatever text changes you think helpful.

>   standards track, since it doesn't meet the requirements for proposed

You have so far offered no details of failure scenarios that warrant the 
fears you are expressing.

>- needs to be examined for a wide range of content

Again, you are offering such a generic suggestion, there is no way to know 
what problems you fear or how to satisfy them.

>   per-bodypart, basis.   if they are not requested, such semantics

So, SMTP is not appropriate to standardize without having submit and pop 
also specified at the same time?

That is a barrier that is equivalent to the one you wish to impose before 
standardizing the current option.  It means that you believe that an entire 
suite of specifications must be offered at the same time, before any one of 
the components may be standardized.  This is contrary to typical IETF practise.

>   multiple ways to perform a conversion and some are worse than others.

Conneg is not new to this option.  The requirement that you wish to impose 
has not been imposed on other uses of Conneg.  It appears that you are 
confusing a carriage scenario with a conversion mechanism.

>   recipient's consent and control

If you have text to suggest for the draft, please offer it, for working 
group review.  (It would have been nice to have had that text offered 
earlier.  The danger is that there will be a running stream of 
non-normative modifications of this nature.)

>- needs to state exactly which parties are authorized to perform

Huh?

>- needs to prevent multiple conversions.

What does that mean?  What problems are you concerned about?

>negotiation past a single hop.

You believe that there does not already exist a requirement for relays to 
enforce content quality?  You believe that the concern about enforcing 
content quality is new to this option?

>CONNEG errors priority over all other sources of error.

The nature of any complex system is that there can be multiple errors at 
approximately the same time.

Hence your concern for prioritization of error reporting is not specific to 
this option.

>can reasonably convert video to audio.

Please provide a detailed example of Conneg use that could lead to the 
problem of needing to convert video to audio.

>to all recipients.

Feel free to provide a real-world example of being unable to produce a 
common form within the context of conneg capabilities information.  You 
have raised this concern before and we have, so far, seen nothing concrete 
to substantiate the concern.

d/

----------

---

From: Keith Moore <[email-masked]>
Date: Tue, 06 Aug 2002 14:13:00 -0400

> be attempted and that will not work properly.

I have already done so, on multiple occasions.  Perhaps we have different 
ideas of what is 'likely' or what is 'real-world'.  To me it seems likely 
and consistent with the real world that senders will send around a wide
variety of content, that recipients will want to specify what kinds of 
content they are willing to accept to keep from having their mailboxes
cluttered with useless content, and that senders will at least occasionally 
want to specify that the message should not be delivered unless the recipient 
is willing to accept the content.  All of which is fine.  But the assumption 
that some intermediate MTA can then be expected to "transform" the sender's 
content into something that the recipient is willing to accept - that does
not seem either fine, likely, or consistent with the real world.

At any rate, I don't think it's acceptable for an extension to cause
delivery failure or unrequested munging of messages even in 'unlikely' 
cases - when whether the extension is employed isn't under control of 
the sender.

> materially affect the adequacy of the normative specification?

I think that the examples being all fax illustrates that other cases 
have not been adequately considered.  Because as soon as you start to
consider other cases of how SMTP is used to send content, you find 
that the existing proposal doesn't suffice.
 
> gives them better information, not different capabilities.

No, it's not just giving better information - because the SMTP client is 
*required* to do something with that information.  And there are words 
and examples in the current proposal that have MTAs doing transformations 
without any authorization from the sender or recipient to do so.

> authority does not want to permit use of this option, it will not use it.

I disagree that it is appropriate to assign such authority to administrators.
Certainly the transformations will not be performed (i.e. the software will
not be installed) without the administrators' consent, but the authority
to transform a particular message needs to be given by the user, not the
administrator.  The administrator is rarely in a position to know whether
the transformation is appropriate.

> capabilities that already reside with the sending SMTP server.

I think you have that backwards, or something.  It doesn't make any 
sense to me.

> "control" over the semantics of the recipient's address.  You can't get 

Yes, but the function of SMTP intermediaries has generally been limited to
relaying of the message intact - not performing arbitrary transformations on 
the message.  We've accepted that minor transformations might be necessary
in a gateway situation, but we've never blessed the idea that arbitrary MTAs 
should change the content of messages within the SMTP world.

> does not change the specification.  It "permits" and "prohibits" nothing.

Perhaps it is only misleading or confusing, but it still affects the 
quality of the document.
 
> What is the problem this causes?  How does it make this service fail?

If the message is not delivered intact when sender did not request
transformations and the receiver did not request transformations,
that's a failure.  SMTP isn't supposed to mung messages.
 
> that a separate command is actually worse than an RCPT parameter.

A lot of that analysis depends on how you assume that this is used - and 
in particular whether it is used only by special-purpose devices or if 
it's used between UAs in general.   I don't share those assumptions.

> thing.  An example of the impact of this is probable failure in getting 

A separate command doesn't have to change the state machine in the sense
that it adds either new states or new state transitions -  and I'd argue
that the command should not do so.  I'll grant that firewalls might have 
trouble with a new command.  But I've also seen firewalls have trouble 
with new parameters, so I don't know how much the existence of firewalls 
favors a new parameter over a new command. 

And RCPT with CONNEG most definitely changes the state machine of the sender,
even in cases where the sender sends at most one RCPT per session, because
there is at least new one state transition necessary for "cannot adapt
content to recipient's capabilities".  And in general MTAs will need to deal
with the multiple RCPT cases, which adds a fair amount of complexity to the
sender's state machine.

> Your use of the word "often" is contrary to the likelihoods expected by 

Yes, I understand that, though I think "some others" would be more accurate.
People will disagree about what they think are probable usage scenarios.  

> There is nothing in the normative text that is fax specific.

The expectation that it's possible to downgrade to a common format is 
specific to fax.  It might work with voice mail assuming no proprietary 
codecs were used, but that seems like a stretch.  Also, the assumptions 
about how this will be used, that governed design decisions like 
overloading RCPT vs a new command, seem to be heavily based on fax.
 
> Please offer whatever text changes you think helpful.

To some degree I have tried to do so already, though perhaps not bracketed
as such.  But I don't think we're quite to the point we can solve the 
problems by mere wordsmithing. 

> fears you are expressing.

That's false. I've done so on multiple occasions.

> what problems you fear or how to satisfy them.

Well, I've already stated them on multiple occasions.  And when the problem
is one of a failure to consider some important factor early in the design 
phase (as this seems to be), the fix is to reconsider that factor. 
Until that is done, all protocol fixes will be suspect.

> also specified at the same time?

I didn't say anything about the mechanism, and in fact I expect that 
senders would specify transformation permissions in message and body part 
headers.  I don't know if it's necessary to standardize the mechanisms 
by which recipients specify conneg parameters, but I do think it's 
necessary that recipients consent to such rather than having some 
unrelated other party make assertions on their behalf.

> the components may be standardized.  This is contrary to typical IETF 

I think you have this backwards.  It's contrary to IETF practice to 
standardize a protocol that has known technical omissions - which this
proposal certainly does.  Your claim that it's appropriate to advance
an incompete specification because 'an entire suite of specifications
must be offered at the same time' is baseless and not supported by
either the process standards or by 'typical IETF practise'.  

> confusing a carriage scenario with a conversion mechanism.

The difference here, I believe, is that with CONNEG over SMTP arbitrary
SMTP servers are both allowed and expected *as a matter of protocol 
conformance* to perform conversions without consent of either sender or 
receiver.
 
> non-normative modifications of this nature.)

I could offer text on this one specific point, but there are other points
for which offering text seems counterproductive.  In general, it seems 
premature to offer text when we don't even have a vague agreement on what 
specific mechanisms (e.g. for indicating consent) should look like.
 
> What does that mean?  What problems are you concerned about?

this is mentioned below in the detailed comments.
 
> content quality is new to this option?

I don't know what it means to "enforce content quality", partially since
I think it's the job of MTAs to "maintain content integrity" and partially
since the MTA has no notion of what "quality" means to the recipient.   
I believe there is now a requirement for relays to relay mail intact - in 
the current world, any relay that doesn't do so is nonstandard.  The proposed 
extension tries to change that.

> Hence your concern for prioritization of error reporting is not specific to 

No, but the proposal says that it MUST take precedence, and that seems 
overbroad.  I did attempt to suggest better wording.
 
> problem of needing to convert video to audio.

Someone sends mail with a video attachment to a recipient that advertises
that it can only accept audio.  Somebody (whether the sender or an 
intermediate MTA) adds the CONNEG option for that recipient.  The conversion 
is not possible, yet it is required by the current spec.
 
> to substantiate the concern.

As far as I can tell, this is only because you believe that nobody will try 
to use CONNEG except in those cases that you've anticipated or where it has
been observed to work well.   It's not as if I haven't supplied concrete 
examples as to how things can fail, but you dismiss them as not 'real-world'.  
In my "real world" users' interests and habits are quite diverse and are 
not so predictable.

At any rate, making things work in your idea of the real world is not 
sufficient.  Email must function properly as a matter of design, so that
if the specifications are followed then predictable and useful behavior 
is assured.  Speculation about what users are likely to do and what 
administrators are likely to impose is not sufficient.

Keith