Subject: CORRECTION: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard

From: The IESG <[email-masked]>
Date: Mon, 01 Jul 2002 17:34:24 -0400

[Please note the corrected four week last call date, due to the 
upcoming 54th IETF.  Sorry for any confusion.]

The IESG has received a request from the Internet Fax Working Group to 
consider SMTP Service Extension for Content Negotiation 
<draft-ietf-fax-esmtp-conneg-02.txt> as a Proposed Standard.  

The IESG plans to make a decision in the next few weeks, and solicits
final comments on this action.  Please send any comments to the 
[email-masked] or [email-masked] mailing lists by July 29, 2002.

Files can be obtained via 
http://www.ietf.org/internet-drafts/draft-ietf-fax-esmtp-conneg-02.txt

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 01:53:12 -0400

[referencing draft-ietf-fax-esmtp-conneg-02.txt]

executive summary:  I recommend  against approval of this document
withut extensive changes.  It's not well thought-out (except for the
rather narrow application of SMTP to a one-recipient fax gateway)
and it seems likely to degrade SMTP interoperabliity if widely
adopted.   I do suggest some changes which would make it less
harmful, but it still needs considerable work to be of general 
applicability.

1. layering problems

this functionality is a poor fit for SMTP because SMTP is the 
mail *transport* protocol, and as such should not be dealing 
with content at all, which is a user agent function.

I admit that (by design) there is no opportunity for e2e interaction
at the user agent level in the current protocols, and also that any 
mechanism that might be used to advertise user agent capabliities 
(e.g. rescap) introduces the opportunity for that mechanism to get 
out-of-sync with the actual capabilities/preferences of the recipient.
but what's to stop this mechanism from getting out-of-sync also?
it seems like this extension has very limited applicability - it will 
only work reliably when the SMTP server and user agent or gateway are
very closely tied and when the recipient always uses the same UA
or UAs of very similar capabilities.  (certainly not true of me!)

one can argue that extending SMTP to allow it to perform negotiation on 
behalf of the user agent is no worse than putting that negotiation in a 
separate protocol.  but if we're going to extend SMTP to perform functions 
on behalf of the user agent, it needs to be made very explicit that this 
negotiation is being done on behalf of the user agent - and in particular
that the negotiation being requested represents the preferences of the 
*user*  (i.e. the sender and/or recipient) rather than some third party.

IIRC, this is the same thing that was insisted on for the OPES stuff - 
the negotiation/transformation needs to have explicit consent from, and 
be under control of, at least one of the ultimate endpoints
(sender or receiver) and it seems to me that it's appropriate to apply
the same policy for email.

2. protocol design problem

for transformations that should be under control of specific recipients,
the protocol is poorly designed.  in particular there seems to be no
ability to apply content-transformation on a per-recipient basis,
to handle the case where different transformations are acceptable to
differnet recipients.  again, this drastically limits the applicability
of the protocol (it won't work well unless all recipients at that
SMTP server have the same capability set, which is very unlikely for
most conventional SMTP servers)

of course it's not easy to get SMTP to handle different client-side
transformations for different recipients, since SMTP is designed to 
transfer the exact same message body (modulo received fields) to each 
of a set of recipients.  this is a different model than used by
phone-based facsimile protocols which expect there to be only one
recipient of any given transmission, so it's not surprising that
the fax-style negotiation doesn't fit well into SMTP.

in order to make client-side negotiation a clean fit into SMTP, it is 
necessary for the client to first query the server for the capabilities 
of each intended recipient, then to deduce how many total transformations 
are necessary, then to send one or more transformed versions of the message.
for instance, if recipients A, B, E, and F could accept one transformation 
of the message and recipients B, C, and D could could accept a higher 
quality transformation of the message, the client might attempt to send the 
high-quality version to B, C, and D and then send the lower quality 
version to A, E, and F.

but as currently designed the proposed extension doesn't have the 
necessary protocol hooks to permit this - at least not without
a considerable amount of ugliness.  overloading the RCPT 
command to return capabilities breaks the RCPT command and forces
the client to do RSET in order to send different versions to 
different recipients.

3. timing

this proposal could not have come at a worse time.

putting content negotiation in the transport protocol is likely 
to result in a significant increase in anomolous behaviors
from the mail system which are difficult to diagnose and fix -
and the proposal is being introduced at a time when the mail system 
is already suffering from serious reliabliity problems (often due 
to ill-conceived measures for blocking spam).  at this time,  
*any* proposal for adding new functionality to SMTP should be viewed 
with strong reservations, unless it somehow offers an improvement
in reliability.  a proposal that significantly increases the complexity
of SMTP error handling seems unlikely to do that.

4. naming and applicability 

the protocol claims to be a general-purpose content-negotiation scheme
but actually has "fax" assumptions embedded very deeply. 

first of all, it assumes that there is a linear ordering relationship
for document suitability to the recipient.  while this might be arguable 
for facsimile images (even this is debatable given orthogonal attributes 
like resolution and color), it's certainly not the case for MIME
email in general.

5. recommendations

a.  I strongly recommend that you redesign this extension so that, instead
of overloading the RCPT command, you define a separate command to
request a recipient's capabilities.  e.g.

RCAP <recipient-addr>

would return the conneg capability associated with that recipient,
if available.  there would need to be separate indications
for permanent error (perhaps if the recipient addr were invalid),
temporary error (e.g. can't do the database lookup),
no capabliities available (perhaps not an error, just no information),
and a return of capabilities.

one nice side-benefit is that it doesn't change the error handling
model, because the proposal mentioned above doesn't affect 
the actual message handling commands at all.  (however it does
add a few round trip delays)

b. the idea that the sender should send the "best" or "highest
quality" version needs to be relaxed or generalized, since there's
currently no clear notion of what this means. (and thus no way to 
know whether an implementation conforms to these criteria).  

c. this extension needs to be made less fax-specific.
it needs to be examined with some realistic non-fax-specific
test cases, and it needs non-fax conneg examples also.

d. there needs to be a good way for the recipient to specify
capabilities to the SMTP server, at least for any SMTP server
that can serve arbitrary UA recipients.  probably this means another
extension along with SMTP AUTH.

still, this is a big can of worms, because there's never been
any ability of a UA to specify something to its inbound SMTP
servers before - there's presently not even a good way for
a UA to know which server(s?) should be given this information...

e. make it absolutely clear that such transformations are only to be
made with explicit authorization by the sender and/or recipient.

(a dedicated smtp-to-printer hardware device would be considered 
to have explicit authorization simply because it was purchased
for such a purpose, but a SMTP server acting on behalf of a user
using a general-purpose MUA would need such authorization) 

f. the document needs to consider the effect of message forwarding
on the returned string - if a recipient has his mail forwarded
elsewhere it's not clear what conneg string should be returned.

g. the document alludes to the ability to work through intermediaries
but this isn't explained anywhere, and it needs to be explained.

6. specific problems

> 4.2 Client Action:
>           
>      If the server issued a 250-CONNEG, as part of its
>      EHLO response for the current session, the client MAY
>      issue the CONNEG parameter with RCPT-TO.
>           
>      If the client issues the CONNEG parameter with
>      RCPT-TO, then it MUST honor the capabilities specified
>      in the CONNEG RCPT-TO reply, and transform data that is
>      sent, so that the target can accept the data. The
>      client SHOULD transform the data to the "highest" level
>      of capability of the target.

this incorrectly assumes that (a) there is only one target when
the client can easily specify multiple RCPT commands, and
(b) there is a clear notion of "highest level of capability"
for the target.

also why is the term "target" used rather than the more traditional
"recipient" or "mailbox"?

>      If the server rejects the RCPT-TO command with a 404 reply, the 
>      client may later reissue the RCPT-TO with the CONNEG parameter.

presumably in the context of a different SMTP session?
it's not clear...

> 4.3 Server Action:
>           
>      If the client specifies "CONNEG = REQUIRED" in the RCPT-TO, 
>      but the target does not support the CONNEG parameter, the target
>      MUST reject the RCPT-TO command with a 504 reply. 

here there is a potential conflict between various extensions.
a RCPT command may fail for any number of reasons.  if any of those
reasons is a permanent fatal error it should be sufficient for 
the server to report any one of those errors.  it's not reasonable
for this extension to pre-empt other error notificaitons.
(it may be that other extensions have similar wording, in which case
they're similarly flawed)

>      If the target support the CONNEG parameter, but it can not return 
>      the recepient's capability temporarily, the target MUST reject the 
>      RCPT-TO command with a 404 reply. For example, if the target get 
>      the capability information from a directory, but its connection is 
>      offline, the target MUST reject the RCPT-TO command with a 404 
>      reply. 

similarly: if any permanent failure is detected in the RCPT command, 
the appropriate 5xx response SHOULD probably be returned in preference
to any 4xx response.  for instance if the recipient's mailbox has
been administratively disabled then a 5xx response should be returned
rather than a 404 response which would mean "ask me later what kind
of media types  this recipient can accept"... whereupon the client
would do so only to (eventually) find out that it couldn't deliver 
the message to the recipient anyway.

>      If the client specifies "CONNEG = OPTIONAL" in the RCPT-TO, the 
>      target MUST process the address and message as if the requested 
>      CONNEG capabilities had not been specified.

this doesn't seem to make sense as written.  is "CONNEG = OPTIONAL"
really supposed to be interpreted the same as omission of the 
CONNEG parameter?

also, I don't think there should be spaces around the = sign -
I don't think this is legal syntax for SMTP extensions.

>      Regardless of the value of the parameter, if the target does 
>      support the CONNEG parameter, then it MUST issue a 250 reply, 

again, there could be other reasons for permanent or temporary failure
of the RCPT command, and the assertion of a CONNEG parameter should 
not pre-empt the reporting of such failures.

>      followed by the capabilities of the target that is specified by 
>      the RCPT-TO address. Successful responses to CONNEG RCPT-TO 
>      requests will always be multiple SMTP lines.  The first line 
>      is the normal RCPT-TO response, and subsequent lines beginning 
>      with the exact string "250-CONNEG " and "250 CONNEG " are 
>      the CONNEG responses. The last line begins with "250 
>      CONNEG".  

I wonder here - assuming that this can of worms does not get closed
back up, and there might later be demand for other mechanisms
to request recipient capabilities (e.g. since this one is not very general),
might there be multiple kinds of strings emitted in the RCPT response?
e.g. CONNEG and CONNEG2 and FINAL_SOLUTION_CONNEG?
and if so should this document insist that CONNEG be last?

>      If the SMTP server supports ENHANCEDSTATUSCODES, the response 
>      strings for a success are "250-2.1.5 CONNEG" and "250 2.1.5 
>      CONNEG". The response strings for indicating a permanent 
>      failure are "504-5.3.3 CONNEG" and "504 5.3.3 CONNEG". 
>      The response strings for a temporary failure are "404-4.3.3 
>      CONNEG" and "404 5.3.3 CONNEG".

are these codes already defined in 1893?  (I didn't look) 

I don't think it's appropriate to dictate extended status codes
since their entire purpose is to provide more precision than
the SMTP reply codes.  there are various sources of permanent
failure and various sources of temporary failure and the set
of status codes is extensible independently of SMTP reply codes.

>      All CONNEG-capable clients and CONNEG-capable servers MUST
>      be able to successfully process CONNEG lines that are up to 512
>      characters long, as required by RFC2821. If the length of CONNEG
>      lines is greater than 512 characters, the server MUST insert
>      line breaks and make next CONNEG line. 

CONNEG *what* lines?  command lines? response lines?  both?

>      The contents of the capability listing MUST conform to
>      [RFC2506], [RFC2533] and [RFC2534], and MAY be used to emulate
>      services, such as the facsimile start-of-session capabilities
>      negotiation as described in "Content Feature Schema for
>      Internet Fax". [RFC2879]
> 
> 
> 
> 
> 5.   SYNTAX
>      
>      Command with "CONNEG":
>       "RCPT TO:" ("<Postmaster@" domain ">" / "<Postmaster>"
>                    / Forward-Path) (SP "CONNEG =" ("REQUIRED" /
>                  "OPTIOANL") CRLF

the space doesn't belong after CONNEG, and OPTIONAL is misspelled. 

> 
>      Reply:
>       ( ("250-" CRLF) *("250-CONNEG" capability CRLF)
>         ("250 CONNEG" capability CRLF) )/
>       ( ("250-2.1.5" CRLF) 
>           *("250-2.1.5 CONNEG" capability CRLF)
>            ("250 2.1.5 CONNEG" capability CRLF) )/
>       ("504" CRLF) /
>       ("504 5.3.3" CRLF) /
>       ("404" CRLF) /
>       ("404 4.3.3" CRLF) /

it's not appropriate to put literal response codes in the grammar,
particularly the extended status codes which might actually need to
vary more than is indicated here.

much better to have something like:

reply: initial-reply *(conneg-reply/other-reply) final-reply

initial-reply: reply-code "-" [ status-code ] CRLF

conneg-reply: reply-code "-" [ status-code " " ] "CONNEG" capability CRLF

other-reply: reply-code "-" [ status-code ] *char CRLF

final-reply: reply-code " " [ status-code ] CRLF 

> 
>       capability = <<as per [RFC2506], [RFC2533], [RFC2534]
>                     and [RFC2879] >>
> 
> 
> 
> 6.   EXAMPLE
> 
>      An example of ESMTP sequence with successful RCPT-TO response
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED
>      
>      S: 250-<[email-masked]> recipient ok
>      S: 250 CONNEG (&(image-file-structure=TIFF-minimal)
>      S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
>      S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
>      S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
>      S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
>      S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
>      S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
>      S: (ua-media=stationery) )

this needs clarification - it looks like the sender is sending
multiple lines without a 250 or CONNEG on each line.
it would probably be better to break up this example into
several SMTP lines and also to explain that any of those lines 
can be longer (up to XXX bytes) than shown in the example.

>      
>      C: DATA
>      
>      S: 354 okay, send data
>      
>      C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
>           Per:
>           (  image-file-structure=TIFF-minimal
>              dpi=400
>              image-coding=JBIG
>              size-x=2150
>            )
>         >>

this is confusing - it makes it appear that TIFF-FX
is the only content-type acceptable.  

also TIFF-FX is not the name of the content-type,
presumably it's application/tiff-fx or some such.

>      
>      S: 250 message accepted
>      
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
>      An example of successful RCPT-TO response when the length of 
>      capability is greater than 512 characters. 
> 
>      S: 250-2.1.5<[email-masked]> recipient ok
>      S: 250-2.1.5 CONNEG (&(image-file-structure=TIFF-minimal) ...
>      S: 250-2.1.5 CONNEG   .....
>      S: 250 2.1.5 CONNEG   (color=Binary)
> 
> 
>      An example of succssful RCPT-TO response when CONNEG-capable 
>      server supports ENHANCEDSTATUSCODES.
> 
>      S: 250-2.1.5<[email-masked]> recipient ok
>      S: 250 2.1.5 CONNEG (&(image-file-structure=TIFF-minimal)
>      S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
>      S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
>      S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
>      S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
>      S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
>      S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
>      S: (ua-media=stationery) )

again, this response is confusing because it's line wrapped

> 
> 
>      An example of ESMTP sequence with parmanent failure RCPT-TO 
>      response. 
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED
>      
>      S: 504 <[email-masked]> recipient ok
> 
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 
>      An example of an ESMTP sequence with temporary failure RCPT-TO 
>      response when the value of parameter is "REQUIRED":
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = REQUIRED
>      
>      S: 404 <[email-masked]> recipient ok
> 
>      C: QUIT
>      
>      S: 221 goodbye
>      .
>      .
>      .
>      retry according to implementation
> 
> 
> 
>      An example of an ESMTP sequence with temporary failure RCPT-TO 
>      response when the value of parameter is "OPTIONAL":
> 
>      S: 220 ifax1.jp IFAX
>      
>      C: EHLO ifax1.jp
>      
>      S: 250-ifax1.jp
>      S: 250-DSN
>      S: 250 CONNEG
>      
>      C: MAIL FROM:<[email-masked]>
>      
>      S: 250 <[email-masked]> sender ok
>      
>      C: RCPT TO:<[email-masked]> CONNEG = OPTIONAL
>      
>      S: 250 <[email-masked]> recipient ok
>      
>      C: DATA
>      
>      S: 354 okay, send data
>      
>      C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
>           Per "A Simple Mode of Facsimile Using Internet Mail"
>           RFC2305
>         >>
>      
>      S: 250 message accepted
>      
>      C: QUIT
>      
>      S: 221 goodbye
> 
> 
> 
> 7.   IANA CONSIDERATIONS
>      
>      On publicatiom of this document by the RFC Editor, the IANA 
>      shall register the Content_Negotiation ESMTP extension defined 
>      in section 3.
> 
> 
> 8.   SECURITY CONSIDERATIONS
>      
>      This ESMTP option calls for a respondent to disclose
>      its capabilities.  Mechanisms for determining the
>      requestor's authenticated identity are outside the
>      scope of this specification.  It is intended that this
>      mechanism permit disclosure of public information;
>      hence there is no particular need for security
>      measures.
>      
>      However there is nothing to prevent disclosure of
>      sensitive information that should receive restricted
>      distribution.  It is, therefore, the responsibility of
>      the disclosing ESMTP server to determine whether
>      additional security measures should be applied to the
>      use of this ESMTP option.

seems like you should at least mention SMTP AUTH. 

>      A man-in-the-middle attack might change the capabilities reported 
>      for a given recipient. For example: Suppose the sender knows the 
>      recipient has the ability to view color documents so they mark 
>      some things in red in what is otherwise a black and white 
>      document. But someone interferes with the returned capabilities, 
>      indicating that the recipient only supports black and white. The 
>      document is duly downgraded, with the result that the recipient 
>      doesn't see what the sender marked. 

the STARTTLS command could be useful in providing integrity assurance.

>      An indirect exposure can occur when the report of a capability 
>      implies use of specific software.  If that software is known to 
>      have security weaknesses, the capabilities report effectively 
>      advertises the associated opportunity to exploit the security 
>      weakness.
>      
>      For target SMTP servers that require security mechanisms to be in 
>      force at the start of the session, the target SHOULD refrain from 
>      including the CONNEG parameter in an EHLO response until the 
>      requisite security mechanisms are in force.

---

From: Dave Crocker <[email-masked]>
Date: Tue, 02 Jul 2002 00:23:43 -0700

At 01:53 AM 7/2/2002 -0400, Keith Moore wrote:
>withut extensive changes.  It's not well thought-out

What additional thinking out are you looking for?

>  (except for the rather narrow application of SMTP to a one-recipient fax 
> gateway)

In fact the motivation for it is not necessarily a gateway, but rather any 
recipient environment that conforms to this profile, including usage that 
is wholly retained among Internet-capable participants.

For that matter, the mechanism applies on a per-recipient basis, so what 
prompts the claim that it works only for a single recipient?

>and it seems likely to degrade SMTP interoperabliity if widely adopted.

How will that happen?  How is it likely?

>with content at all, which is a user agent function.

The construct of using SMTP to verify support of a higher level feature is 
not all that different from DSN.  What is the practical problem with 
applying it here?

>but what's to stop this mechanism from getting out-of-sync also?

The end-to-end asynchrony and latency for this mechanism certainly do 
provide an opportunity for a recipient's capabilities to change, after it 
has advertised a particular set.  However the key point to note is that 
there is no mediating storage/directory service, so the advertisement has a 
degree of directness not possible through any other mechanism.

For recipients concerned about such volatility, all they have to do is 
refrain from advertising CONNEG support.  For senders concerned about such 
volatility, they should use CONNEG=Required.

>or UAs of very similar capabilities.  (certainly not true of me!)

1.  The specification already highlights that it is tailored for rather 
'direct' scenarios.

2.  Each user can decide what to advertise.  If they vary their UA they can 
choose to advertise a safe and lesser set of capabilities.  In any event, 
it is their choice.  The mechanism does not impose any problems that are 
not inherent in use of multiple UAs.  However it DOES permit relatively 
direct advertising of capabilities, which is currently not feasible.

>but if we're going to extend SMTP to perform functions
>*user*

In what way does that affect the technical specification?

That is, how is the current specification deficient, with respect to your 
concern?

>in particular there seems to be no
>differnet recipients.

We must be reading different specifications.  Choice of enforcement of this 
option is per-recipient.  Each recipient may generate a capabilities 
response if they wish a different form of the content.

What, exactly, are you looking for that is missing?

>of a set of recipients.

It sounds as if you are assuming that later transmission of content that is 
revised according to capabilities information is required (or expected) to 
be sent using that same address list.

That is like requiring that all BCC copies are sent along with the TO and 
CC copies.

In other words, there is not such requirement.  It is OK to do it and it is 
OK not to.

Yes, the current CONNECT option is intended to send the same content to all 
recipients, since that is the style of SMTP.  There is nothing preventing 
later sending an alternative content form to selected recipients.

Interestingly, this would not actually create any additional round-trips or 
additional content transfers.

>  this is a different model than used by
>recipient of any given transmission,

You might want to take a look at fax broadcasting services.  The last-hop 
telephone portion behaves as you describe, but only after a 'generic' 
posting process.

>  so it's not surprising that the fax-style negotiation doesn't fit well 
> into SMTP.

Actually, the proposed scheme permits content transmission tailoring that 
is considerably more flexible than is typical in the real fax world.

>are necessary, then to send one or more transformed versions of the message.

Ahh.  I see.  You want to insist on an interaction scenario that is known 
to be unacceptable, since it is guaranteed to introduce too much delay 
before there is any utility at all.

>   overloading the RCPT

If that is what the client chooses to do, then yes.  However there are 
other courses of action available.  At the least, note that the content is 
not sent until the client has CONNEG responses about all of the recipients.

>to ill-conceived measures for blocking spam).

How does use of the CONNEG mechanism affect reliability or accountability?

>   at this time,
>in reliability.

I was not aware that we had any problems being attributed to SMTP 
options.  Please provide some citations, so we can compare their details to 
the current proposal.

>but actually has "fax" assumptions embedded very deeply.

The fax world is the basis for the idea, yes.  However the specification 
has been done so as to be independent of fax, thereby using an approximate 
model that is known to be useful from the fax world and then applying it 
for general enhancement of email.

>for document suitability to the recipient.

Ok.  I give up.  I have no idea what "linear ordering relationship for 
document suitability" means.

>request a recipient's capabilities.  e.g.

In spite of the proffered benefits of a separate command, I fail to see how 
it produces any real benefits.

It certainly does not make anything simpler, and it certainly vastly 
increases relay delay, due to nearly doubling the round-trip exchanges 
during an SMTP session that uses the option.

>model,

How does the current proposal change the error handling "model"?

>quality" version needs to be relaxed or generalized,

It already has been relaxed.  Note the SHOULD rather than MUST.

>  since there's currently no clear notion of what this means.

Yet we have lived with that notion in multipart/alternative for some years.

>c. this extension needs to be made less fax-specific.

What, specifically, makes it fax specific, and what changes are you 
specifically seeking that will be less fax-specific?

>test cases, and it needs non-fax conneg examples also.

Test cases are always a good thing.  Feel free to offer some that satisfy 
your non-faxiness goal.

>extension along with SMTP AUTH.

You want to require that that mechanism be in place before the current 
specification can be advanced?  I think the phrase "chicken and egg" works 
here.  You create a deadly embrace for reaching any utility.

>a UA to know which server(s?) should be given this information...

Think hard.  What is likely to drive creation of such a 
mechanism?  Requiring that it be built in the abstract or looking for it to 
be built in reaction to actual demand?

>made with explicit authorization by the sender and/or recipient.

This sort of parental direction in IETF specifications probably does no 
harm, but it is difficult to imagine that it does much benefit.

>elsewhere it's not clear what conneg string should be returned.

There are all sorts of implications in forwarding.  In general, forwarding 
that is not simply SMTP relaying is outside of current IETF 
specification.  Imposing a requirement about it here, without demonstrating 
actual problems, is a tad extreme.

>but this isn't explained anywhere, and it needs to be explained.

All it says is that it works with SMTP relaying, just as one would expect 
and SMTP extension tow work.

>...

It does not make the assumption you suggest.  In fact, I can't imagine how 
you came to the assessment that it makes that assumption.

>...

It is probably a good idea that we not try to fix SMTP (yet again, given 
the protracted drums effort.)  And it certainly is not a good idea to 
require that the current specification fix something this vague when no 
SMTP-related specification is required to.

>really supposed to be interpreted the same as omission of the

I think that the text that is in the first paragraph of that section ("but 
the target does not support the CONNEG parameter") got deleted during 
revision.  In general, those first 3 paragraphs are dealing with lack of 
support, whereas the following paragraphs are dealing with presence of support.

>also, I don't think there should be spaces around the = sign -

ack.

>might there be multiple kinds of strings emitted in the RCPT response?

Please indicate what capabilities are missing from this capabilities 
mechanism that would make it more general and for which there is a 
demonstrated need.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 10:33:32 -0400

> What additional thinking out are you looking for?

see below.
 
> prompts the claim that it works only for a single recipient?

if different recipients have different capabilities, the sender has
to either send a single message with a format that is suitable for
all recipients, or it has to do a RSET and resend the RCPT commands
separately for each version of the message that it sends.  nowhere
in the document do I see any mention of this, and furthermore it
complicates error handling.  so I assume that it wasn't adequately 
considered.

> How will that happen?  How is it likely?

because the proposal complicates the SMTP state machine (especially when 
you consider the multiple recipient case), it adds additional
failure modes, and the error handling is not well described.
 
> not all that different from DSN. 

now that's a bizarre statement.  DSN is a per-mta feature.  it is either
supported or not supported by an MTA.  it doesn't modify content.  
DSN doesn't reveal anything about a recipient's user agent. 

> degree of directness not possible through any other mechanism.

Dave, the document itself recognizes the possibility of a directory
service, even listing a specific error code for this case, so your statement 
is facially false.  Furthermore it must be recognized that the vast majority
of MTAs do not interact directly with user agents at all, so the 
"degree of directness" you claim is completely bogus.

> refrain from advertising CONNEG support.  

in other words, what you are saying is that this extension has very
narrow applicabliity - it is not  usable on the vast majority of
message transfer agents, but only on those that are tightly coupled
with an MUA or gateway that applies to all recipients.  

so why are we complicating SMTP with an extension that only applies
to a very narrow case?  is support for fax really so important 
that we have to extend SMTP in such a way that it's only usable for
fax, and in such a way that fax-capable MTAs are a completely different
class than other MTAs?

> For senders concerned about such 

the volatility isn't really the issue I'm concerned about - the issue 
is the lack of a standard mechanism for communication of capability
sets from the UA (or the user) to the inbound MTA; and more generally, 
the difficulty of using the CONNEG mechanism to support users who
use multiple UAs with different capabilities.  at least, the latter
scenario doesn't seem to have been considered.

> 1.  The specification already highlights that it is tailored for rather 

true, but it also says that indirect ones are possible.

> 2.  Each user can decide what to advertise.  

HOW?  there's no mechanism defined that allows them to do so.

> If they vary their UA they can 

because if third parties make unauthorized modifications to messages
then the end-to-end integrity of the message is destroyed.

is that technical enough for you?

> What, exactly, are you looking for that is missing?

reread my earlier message, and see above.

> be sent using that same address list.

well, by the time the sender knows what the recipients' capabilities are,
the receiver already thinks it has the recipient list, and it is expecting
a single messgae to be sent to all of those recipients.

maybe you need to read the draft you claim to have co-written?

> later sending an alternative content form to selected recipients.

so you're suggesting that multiple copies of the message be sent to those
recipients?  

> additional content transfers.

that's about the most bizarre thing I've ever seen you write...

> telephone portion behaves as you describe, but only after a 'generic' 

perhaps, and I'm not familiar with these.  but my guess is that they don't
do per-recipient negotiation on earlier hops.

> before there is any utility at all.

PIPELINING would make the delay negligible.  and it's far preferable to 
overloading the RCPT command.
 
> not sent until the client has CONNEG responses about all of the recipients.

perhaps you'd like to describe those "other courses" which allow the
client to meet the requirement that it deliver the highest quality 
content to each recipient?  or for that matter, that allow the client
to satisfy mutually conflicting needs of recipients?  (e.g. one recipient
will only take gif while another will only take jpeg)

> How does use of the CONNEG mechanism affect reliability or accountability?

it complicates error handling.

> options.  Please provide some citations, so we can compare their details to 

the problems are with SMTP in general.  if you want citations, go read
delivery failure notices from a few large mailing lists.

> for general enhancement of email.

It's evident that not very much thought has been invested in making
the non-fax case work.

> document suitability" means.

it means that the proposal assumes there is a notion of "higher quality" 
vs "lower quality" versions of a message that can be determined for all 
messgaes and that is obvious to an implementor.

> during an SMTP session that uses the option.

it gets you out of the "oops I don't really want to send this message to
that recipient because that recipient needs a different format of the
message" trap.

as far as round trips, RCAP RCAP RCAP RCPT DATA RCPT DATA RCPT DATA
is slightly better than RCPT RCPT RCPT RSET RCPT DATA RCPT DATA RCPT DATA

but you really ought to pipeline these things anyway.  

> How does the current proposal change the error handling "model"?

by introducing new error conditions that require different handling
than existing errors.

> It already has been relaxed.  Note the SHOULD rather than MUST.

it's still a conditional requirement of the specification that
is so vaguely defined as to be unimplementable except perhaps
for the narrow fax case.  different implementors could easily interpret
the SHOULD in different ways even absent any conditions that would allow
them to disregard the SHOULD.  in other words, it doesn't produce 
predictable behavior.

> Yet we have lived with that notion in multipart/alternative for some years.

in retrospect we might consider mp/alternative a bug.
I don't think citing it adds weight to your argument.

> Test cases are always a good thing.  Feel free to offer some that satisfy 

hey, you're the one making the proposal, seems like it's your job to
make sure it really has the applicability that you claim it does.  

I don't really think the proposal provides enough benefit to make it
worthwhile even if it's done right. but rather than rejecting it out 
of hand (and recognizing that the fax community wants this badly)
I'm trying to offer some constructive criticism that would make it 
less objectionable.

> here.  You create a deadly embrace for reaching any utility.

I think it's called 'having a complete specification'.   I didn't
say anything about having the mechanism deployed, but I do think it
needs to be defined.

> be built in reaction to actual demand?

no argument there - but my point is that this is breaking new ground, 
and as such there are many issues that need to be considered that we
haven't thought of before.

> harm, but it is difficult to imagine that it does much benefit.

tell that to the OPES people who were trying to push for web 
proxies that would modify content without either the provider's
or consumer's consent.     the last thing I want is an ISP 
advertising content capabilities on my behalf without my authorization -
and I can think of some large ones that would do exactly that.

I used to think we could rely on each layer to maintain the appropriate
degree of transparency for that layer without our demanding that it be so.
history has proven me wrong.  it needs to be specified.
 
> actual problems, is a tad extreme.

apparently you'd rather just deploy a new protocol extension without 
even bothering to think about how it affects the infrastructure.

let's just say I disagree.

> and SMTP extension tow work.

how does it work with SMTP relaying if there's no defined way to 
relay the capabilities?  either you leave it to the last hop,
which (while fine with me) makes the statement kind of trivial,
or you do something really ugly...  again, a bit of clarification
of this statement wouldn't hurt.

> you came to the assessment that it makes that assumption.

well, I can't imagine how you came to a number of the assessments you
are claiming.  perhaps we live in different universes.

> SMTP-related specification is required to.

so we should always write new specifications as poorly as existing ones?

this wasn't so much of a problem when we didn't have so many extensions -
but the more extensions we have the more problems like this we introduce .

> mechanism that would make it more general and for which there is a 

actually this one is the least of my concerns about the document.
if the other problems can be fixed this one is easy...
but the fix for this depends to some degree on how you fix the other
problems, so I'll defer making suggestions for now.

Keith

---

From: John C Klensin <[email-masked]>
Date: Tue, 02 Jul 2002 19:45:19 -0400

IESG,

I want to partially agree with Keith -- this proposal makes me
very nervous and I believe the IETF should share that
nervousness.  Since Keith and Dave are more than capable of
discussing the various issues that have been identifed to the
point of clarity, I want to take a slightly different cut at the
problem and tentatively suggest a different approach.

For me, the root problem here is that the proposal seems to
simultaneously say:

	* The MTA here knows a good deal about the MUAs
	associated with each of the recipients.
	
	* The MTA is able to support a relay function.

It appears to me that, even ignoring layering violations, having
both of those things be true requires either a good deal of
negotiation among a set of relays and the final target host
--going beyond the "next relay will take it and accept
responsibility" model that some SMTP extensions use, e.g., to
support non-7bit-transport-- or there has to be some magic.  I
don't see either in a quick reading of the document.

More generally, I think we should hesitate about SMTP extensions
whose purpose is to support a particular application, without
understanding all of the implications of generalizing them.  I
think this is quite similar to one of Keith's points.

But I believe it has been a characteristic of much of the fax
work so far that relaying is not anticipated (this draft, in
repeatedly commenting about "direct" connections, reinforces
that belief).  Indeed, we have text floating around that
suggests that Internet Fax setups be configured to avoid relays,
just as we have had similar suggestions about EDI-over-email
applications.  Sending a virtual fax to a fax broadcast device
is another matter, of course: to the degree to which any action
takes place in the message content, we've traditionally thought
about that as a UA function, with the broadcast device acting as
a receiving SMTP, delivering the message to the relevant UA (no
matter how automated), which then initiates a completely new
SMTP connection (or set of connections).  Or we have modelled
the broadcast device as a fax-capable SMTP gateway, with some
different mail transport on the far side, which completely
changes the rules.

This is all leading to a constructive suggestion:  Unless there
is a clear need for an SMTP relay function that follows SMTP
rules --e.g., the relay MTA doesn't look at message content at
all-- it may be time for a new SMTP extension/ capability
announcement to deal with these specialized applications that
use SMTP as a message transport but that really aren't email in
the normal sense.  

One version of such an extension would be "NORELAY".  An SMTP
server advertising it would guarantee, if the client requested
that option (presumably on MAIL or a separate command -- this
makes no sense on a pre-recipient basis) to either deliver a
message or bounce it, but, under no circumstances to relay it
off to another SMTP server.  Of course, a server advertising
that extension (or not) could refuse relaying even if the client
didn't request the option: we never require servers to relay.

In the presence of such an option, content negotiation that
would require that the server know about its local MUAs becomes,
IMO, somewhat more plausible.  We have always permitted
originating and delivery MTAs more flexibility than intermediate
ones.

In this particular case, if there was no relaying and the
capabilities of the receiver were really server capabilities,
rather than MUA ones (as would be the case for a dedicated
Internet fax device or a distribution system that simulated
one), the whole picture could be considerably simplified by
treating capability negotiation on a per-SMTP-connection basis,
rather than a per-recipient one.

Is it possible that something of that nature would make sense?

-------------

A few other issues that Keith didn't seem to pick up on (to my
surprise):

(i) So far, it has been a basic assumption of the ESMTP
environment that a server advertising a particular capability
does not (I'd have to go back and look, but I think perhaps MUST
NOT) reject an attempt to request the desired capability.   I
understand the reasons here, given the general model, but
returning a 504 to "RCPT TO:<baz@bar> CONNEG" (implying
"REQUIRED") would seem to violate that principle.

(ii) In order to facilitate efficient relaying and processing of
messages with large numbers of recipients, we permit, indeed
almost encourage, SMTP receivers to accept RCPT commands without
verification and then to prepare bounce messages if needed.  I
strongly suspect that there are conforming MTAs that can behave
no other way.  It seems to me from this specification that a
server that advertises CONNEG must not perform this types of
deferred operation, at least with regard to that parameter, but
must handle errors within the SMTP transaction.   If that
requirement actually exists, I believe it is very important that
the document point it out.

Also, if relaying is permitted, then either the first relay in
the chain has to have complete knowledge about what will be
permitted downstream (and the document doesn't give any
indication as to how it would find that out), or emailed
rejection messages become an absolute requirement (at least
unless no response is issued to any RCPT TO until the final
delivery MTA has generated a 250 code and returned that along
the line -- I suppose that would be possible, but it would
require yet other SMTP extensions to specify the behavior and it
would probably be a performance disaster).

(iii) I can't reconcile the last statement of section 4.2:

	If the server returns an EHLO 250 code without CONNEG
	capabilities, the client MUST work as "a Simple Mode of
	Facsimile Using Internet Mail" [RFC2305].

With the assertion that this is a general facility, not limited
to fax.   If it is a duck, let's call it a duck and be done with
it.  And, if fax is sufficiently special that we need things
like this, perhaps it is time to add something like NORELAY, or
a FAX extension, to make it clear that what is happening is not
normal SMTP.

(iv) Like Gregory Neil Shapiro, I'm not sure what a "target" is
and I'm not sure the term is used consistently.  E.g., in 4.2,
it appears as if the "target" is not the receiving SMTP server,
but is rather the mailbox (and, presumably, MUA) listed in the
RCPT command.  But 4.3 says "... the target MUST reject..."
which strongly implies an SMTP server action.

(v) Just as an editorial matter, but, in order to avoid
unnecessary confusion, it would probably be helpful if the
document adopted terminology consistent with RFC 2821 (and 821).
E.g., there is no such command as "RCPT-TO", the name of the
relevant command is "RCPT".

--------

To paraphrase advice Dave has given on many other occasions,
let's divide this set of issues up and solve the problems that
are well-defined and that we know how to solve and leave the
rest until there is a demonstrated need.  The problem here, as I
understand it --both generally and after reading the
introduction to this draft-- is fax and fax-like
start-of-session 
capabilities negotiation in a direct connection environment.
Let's think about using SMTP extension mechanisms, if necessary,
to specify/require that case and just avoid generalizations that
require us to deal with all of the poorly-understood (or at
least unspecified in this document)	relaying, intermediary
server, and other issues. 

     john

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 21:14:56 -0400

a few brief comments on John's comments, not all of which have to do with
the conneg-over-smtp draft:

- I see two possible paths to "fixing" this proposal

	1. do the necessary protocol fixes to make it generally 
           applicable for email

	2. make clear that it has only limited applicability
	   for smtp-to-fax gateways, either supporting a
	   single recipient or where all recipients have the
	   same capabilities, and where relaying and forwarding
	   were specifically disallowed.

	   part of making this clear would be to clearly label the extension -
	   not "CONNEG" or "Content_Negotiation" but FAXNEG or "Fax_Negotiation"

	   however if pursuing this path it would still be necessary to 
	   leave room for VOICENEG or MIMENEG or whatever, and this still
	   probably means NOT overloading RCPT.
	
I strongly prefer alternative #1, because I believe it leads to a less
complex and less costly end-state.  In other words I'd far rather have
a single SMTP extension that makes sense in an integrated messaging world 
that can also be used by specialized services, than to have separate 
extensions for (for lack of a better term) "workstation mail", internet 
fax, voice mail,  etc.

I do however recognize that there is probably pressure to do this 
soon rather than to wait until there can be consensus from 
all of the various parties.  However, for this same reason I doubt 
that the #2 approach should result in a standards-track document.

- I also feel compelled to say that waiting until IETF Last Call to get 
public review of any SMTP extension from the wider SMTP community 
is a very questionable strategy.  This statement could be generalized 
by replacing "SMTP" with any other protocol name that has a significant
number of implementations and/or installed base.

- Regarding the idea for a NORELAY extension: I think we would do well
to encourage more direct delivery for all kinds of email.  The email 
system seems to be becoming more fragile all the time, thanks in large
part to spam and viruses and (often poorly chosen or misconfigured) 
countermeasures for these.    The more hops that a message has to 
travel, the lower the probability that it will be delivered.

However it's not immediately clear to me how a NORELAY extension helps -
will senders (fax or otherwise) really be willing to request it?

- In a similar vein, I think we should reconsider the traditional 
encouragement for SMTP receivers to accept RCPT commands without
verification.  The reason is that the mail system  is now considerably
burdened by spam that is often sent to invalid recipient addresses,
and which often has an invalid return address.  Such messages
(and the resulting bounces) clutter up the mail queues and postmaster 
mailboxes where they're generally discarded because most postmasters
don't have time to read them any more.  Early detection of invalid
recipient addresses allows the message to be immediately rejected
rather than bounced, reducing the load on the receiving MTA.

What we really should do is encourage MTA implementors to provide
the necessary hooks to make possible early detection of bad recipient
addresses.

- regarding failures due to requesting CONNEG

> "REQUIRED") would seem to violate that principle.

perhaps more to the point, I think the current semantics are broken.

if the server cannot supply a conneg string for a particular
recipient (because it does not have one or because the sender
lacks permission or whatever) the only reasonable behavior is
for the server to report that fact (which should NOT be a 
server-reported error)

at this point the client can either fail to deliver the message
(because it's not willing to downgrade that far) or it can 
assume that the recipient has only minimal capabliities (e.g. 
simple mode) and do the required downgrade to that mode.

but in the unwilling-to-downgrade case it's the CLIENT's job 
to assert the error, not the server's job.  the client may have
to assert the error in any cases, since it may not be possible
to reasonably downgrade the message to suit the recipient's
capabliities even if the recipient does support CONNEG.
having the server assert the error just makes things more complex
without adding any real functionality.

- Also, unless the client is acting directly on behalf of the
recipient (which is not required by the draft) the client
is being asked to make a convert-or-fail decision without
any reasonable (sender-supplied or standardized) criteria as 
to what kinds of conversions are acceptable.  

This seems like another major omission in the current spec.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Tue, 02 Jul 2002 18:45:32 -0700

Keith,

>         1. do the necessary protocol fixes to make it generally

Given that we DID make this generally applicable for email, someone still 
needs to list the way(s) the current specifications is deficient in that 
regard.  What capabilities or scenarios need to be support that cannot be 
with the current specification?

>            for smtp-to-fax gateways,

Perhaps you missed the portion of my response that pointed out that it is 
not limited to that use and, in fact, that is not even its primary intended 
use.  If there is something in the specification that indicates otherwise, 
please point it out.

>            were specifically disallowed.

There is a difference between being optimized for single-user and/or 
single-hop scenarios, versus prohibiting multi-user and/or multi-hop 
scenarios.  As nearly as I can tell, you are missing that difference.

>            leave room for VOICENEG or MIMENEG or whatever,

If you would supply some sort of functional description for these, it might 
be possible to understand how the current specifications fails to satisfy 
them.  I have no idea what you are intending, and therefore cannot even 
guess at the deficiencies you are implying.

>recipient (which is not required by the draft)

Please specify what you mean by "acting directly on behalf of the 
recipient" and how the current specification fails to provide the necessary 
technical specification.

If the specification contained whatever language about "acting directly on 
behalf" you are seeking, how would it change the normative technical 
specification?

>  the client

You again seem to be missing the fact that that "omission" predates CONNEG 
and has been part of Internet mail for 8 years.

d/

----------

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 22:06:10 -0400

> with the current specification?

go back and reread my earlier messages.

> use.  If there is something in the specification that indicates otherwise, 

I've done so.  go back and reread my earlier mssages.

> scenarios.  As nearly as I can tell, you are missing that difference.

the current document leaves relaying and forwarding undefined.
this needs to be fixed.  it might be a simple fix, but it's still an
omission.

> guess at the deficiencies you are implying.

I'm not going to waste the time to try to specify something which I'm
clearly convinced is a Bad Idea.  

> recipient" and how the current specification fails to provide the necessary 

I've already done so, by saying that the recipient needs a standard way
to set the conneg string that is returned.  

> and has been part of Internet mail for 8 years.

I'm not aware of a content-specific convert-or-fail decision that has to be 
made as part of Internet mail, much less one that has been around for 8 years.

Keith

---

From: [email-masked]
Date: Tue, 02 Jul 2002 23:55:12 -0400

> guess at the deficiencies you are implying.

I think that's Keith's POINT - that this has gotten to "Last Call" and there's
no understanding on the part of the supporters what the "voice" or "mime"
communities want, or whether this can/should be extended to address their
concerns as well.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Wed, 3 Jul 2002 16:40:36 -0400

At 18:45 -0700 on 07/02/2002, Dave Crocker wrote about Re: Last Call: 

>and has been part of Internet mail for 8 years.

Fixing/Avoiding that "Omission" it seems to me to be the 
job-of/use-for Multi-Part/Alternate. The Sender creates all the 
Permutations/Combinations they want and sends the total package. It 
is then the job of the Recipient to wade through the mess and select 
the versions that it wants to accept for display/processing.

CONNEG appears at its root to be an attempt to avoid the "wasted" 
bandwidth and processing time/effort to create a M-P/A MIME message 
most of whose content the Recipient is going to ignore after receipt. 
It also removes the requirement that the Recipient is even capable of 
parsing a M-P/A MIME message to locate the "correct" version of FAX 
which it can understand/process.

I apologize if I am stating the obvious but sometimes it is needed to 
do so to refocus the discussion (the "Forest for the Trees" 
Phenomenon).

---

From: Dave Crocker <[email-masked]>
Date: Wed, 03 Jul 2002 14:27:58 -0700

At 04:40 PM 7/3/2002 -0400, Robert A. Rosenberg wrote:
>bandwidth and processing time/effort to create a M-P/A MIME message most 
>of whose content the Recipient is going to ignore after receipt. It also 
>removes the requirement that the Recipient is even capable of parsing a 
>M-P/A MIME message to locate the "correct" version of FAX which it can 
>understand/process.

Hmmm.  As I recall, the specification was not developed with this goal in 
mind.  However I think that you are pointing out a trade-off in the use of 
ESMTP/CONNECT and Multipart/Alternative that is largely correct.

(I say largely rather than completely because there are some feature 
combinations that define so many combinatorials, they could not be 
considered for M/A but are straightforward for CONNEG.)

Still, you do raise an interesting perspective.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

---

From: Dave Crocker <[email-masked]>
Date: Wed, 03 Jul 2002 19:12:12 -0700

At 09:31 PM 7/3/2002 -0400, John C Klensin wrote:
>   Would you
>expect the client to work its way through those combinatorics in
>real time,

yes.

no matter what the details are, computing the intersection is not that 
difficult and the computation of such issues does not take that long.  No 
matter what the computations are.

As to the question of real-time generation of the chosen transform(s), that 
bridge was crossed as soon as the client went down the path of CONNEG.  In 
other words, if the client wants to do ESMTP/CONNEG, it has better be 
programmed to do the necessary real-time transforms.

>Alternately, the client could notice that the different
>recipients had different capabilities (_that_ ought to be fairly
>easy in real time, especially if each of them announced its
>capabilities in the same order and using the same syntax --
>which the spec doesn't seem to require)

doesn't use the same syntax?

as to order:  were we doing data processing of millions of records, fixed 
position formats might make sense.  We don't do it for message or content 
headers, why should we do it for these attributes?

>  but the potentially-long SMTP connection
>times (in the RSET, figure out the formats and build them while
>the SMTP connection is still open, and then send them
>separately) or abandoning the connection and starting over

and RSET is markedly inefficient, yes.  that's clear.

my personal expectation is that a client will either be designed to send 
the least common denominator of the address list response capabilities, or 
it will send the addresses one per DATA and tailor each.

the alternative that has been the subject of such delightful speculation, 
here, with the RSET required, strikes me as worth very little of any 
implementor's time.

>Just trying to understand the expected situation...

The spec is pretty simple, and it is mostly based on a set of previous work 
involving CONNEG.

The situations for the spec's use are also pretty simple.

The actual uses will no doubt have some surprises, because new features 
always have surprises.

So what *I* would like to understand is where this is going.

There are some small writing errors in the specification that have been 
noted and will be changed.  There are some suggestions for clarification 
and/or guidance that seem pretty reasonable, to.

As to the rest of the discussions about the thread, I have so far failed to 
detect any analysis that discloses a failure in the specification, of the 
type: "it won't work".

So, as I say, what *I* would like to understand is where this is going.

d/

----------

---

From: Dave Crocker <[email-masked]>
Date: Wed, 03 Jul 2002 22:51:45 -0700

At 01:41 AM 7/4/2002 -0400, John C Klensin wrote:
>   With a
>sufficiently complex (non-binary and non-orthogonal) capability
>set, I suspect the computation might even turn out to be
>NP-hard.

With enough thrust, a pig can fly, but that is not a relevant concern for 
building an airline.

The mechanism here provides a means of communicating arbitrary capabilities 
data.  It is one of multiple ways to communicate that data.

If you have a concern about the nature and interactions of the data, then 
that concern needs to be pursued separately from the communication mechanism.

It might also be worth noting how thoroughly abstract your concern is.  In 
fact it is so abstract that it does not qualify to call it unfounded.

On the average, such generic concerns fail to facilitate forward progress 
or protocol improvement.  Quite the contrary.

>A few more constraints, or some explicit guidance that a client
>that gets sufficiently confused is quite justified in just
>giving up, would go a long way here.

Or more thrust.

d/

----------

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Thu, 4 Jul 2002 13:58:59 -0400

At 14:27 -0700 on 07/03/2002, Dave Crocker wrote about Re: Last Call: 

>Still, you do raise an interesting perspective.

Thank you. I am looking at the issue from a overview perspective 
(IOW: How does CONNEG work vs. How would M-P/A work). Since I was not 
involved in the development of CONNEG, I do not know the intent of 
the developers - All I can comment on is the effect of its operation 
in comparison with other methods and state what the EFFECT of the 
choices made are (while taking the effect as the intent even though 
this linkage is probably spurious).

As to the multiple combinations, that was covered by my " 'wasted' 
... processing time/effort". Beyond a certain point, the generation 
of multiple versions (just to be complete) when only one (or a 
limited number) are going to ultimately be used is an exercise in 
futility.

The advantage of CONNEG is that you get the information to generate a 
full list of the different combinations you must produce. The issue 
is what you do then. Do you generate the requisite number of 
combinations and send them out as the minimal number of message 
copies (lumping together those addresses that can share a copy - even 
if that copy is not an exact match to the full capabilities of the 
receiver as in the case of sending a B&W Fax to a Color Capable 
Recipient).

IMO, the ultimate aim (even if this is not practical) is to deliver 
to each recipient the same version of the Fax as they would have 
received IF they were the only addressee while at the same time 
generating only the minimum number of copies to allow 
multi-addressing. IOW: A one copy (/limited number of copies) LCD 
"solution" is not the "Best" solution since it will produce a Fax 
that is not the same as would have been delivered if there were no 
other addresses.

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Jul 2002 15:15:10 -0400

> received IF they were the only addressee 

I think this is a reasonable and practical goal.  I don't think that 
recipient A should experience a difference in message quality 
(or worse, fail to receive the message at all) based on whether or 
not you cc'ed recipient B.

> while at the same time
> multi-addressing.

I think this is an implementation decision.  It should be *possible*
to do this (and it is), but it shouldn't be required of implementations.

here are a couple of trivial examples of the effect of a "downgrade to 
LCD format" implementation strategy.  

assume:

recipient A accepts only HTML and plain text
recipient B accepts only PDF and plain text

an HTML message sent only to A gets delivered as HTML, but if B is cc'ed 
(or bcc'ed) on the message then both A and B get plain text (if the 
client knows how to convert HTML to plain text).  A is accustomed
to receiving HTML from B and is surprised that there's an information
loss.

more seriously, a PDF message sent only to B gets delivered as PDF, but 
if A is cc'ed on the message then the message is bounced for both A and 
B because there's no good way to downgrade PDF to a format acceptable 
to B (or the client doesn't know how).  

So far I have not been able to identify conditions for which downgrading 
to LCD is a viable implementation strategy.  My conclusion is that
implementations that support CONNEG and that attempt to deliver to
mutliple recipients in one transaction MUST be prepared to do RSET and 
split up the delivery if it becomes evident that delivering the same 
version of the message to all recipients is not appropriate.

Keith

---

From: Graham Klyne <[email-masked]>
Date: Thu, 04 Jul 2002 23:17:25 +0100

I am pretty confident this is not the case with CONNEG feature 
matching.  At worst, I think it's polynomial in the size of the feature 
expression, and typically closer to linear.

The worst-case arises when expressions contain repeated conjunctions of 
disjunctions which lead to a multiplicity of conjunctions when reduced to 
the disjunctive normal form.

#g
--

>set, I suspect the computation might even turn out to be

-------------------
Graham Klyne
<[email-masked]>

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Fri, 5 Jul 2002 12:45:34 -0400

At 15:15 -0400 on 07/04/2002, Keith Moore wrote about Re: Last Call: 

>to do this (and it is), but it shouldn't be required of implementations.

Bad phrasing on my part (I had intended the minimal message count to 
be a target not a cast-in-stone requirement). If you must degrade 
into one message per address mode - so be it - BUT I think that if 
you are going to do so, you are cheaping out by not trying to add 
some minimal logic to run the chain of pending copies to see if any 
can be satisfied by the current version. I'm not asking for a full 
matrix analysis (which is the "Best"/"Optimum" Method) - just take 
them in order of the addresses and run the pending chain. This will 
probably not create the minimal count (except by luck) but can still 
reduce the count somewhat.

>to receiving HTML from B and is surprised that there's an information

No - This is a standard M-P/A situation and should be sent to both as 
such (just as if the original message was generated by the MUA and 
packaged as M-P/A [which is the normal action of most of the MUAs I 
know unless the user has overridden the send both option).

>to B (or the client doesn't know how).

This is a more complex case. It would/could be solvable by a Smart 
MUA that acted the same as the HTML/Plain case by sending/creating as 
M-P/A with Text/Plain and application/PDF versions. The "Best" 
solution is if the MUA could also deliver a HTML copy (which would be 
packaged between the Text and the PDF in the stacking order).

Note: I am assuming a 2 stage MUA where the messages are created and 
queued for delivery and upon connect, the MUA can drop out (if it 
wants) unneeded alternatives (instead of doing the transforms in real 
time). I see no reason that the M-P/A creation (with optional 
deletion prior to transmit) could not be a viable alternative for 
real-time transforms (at least in the case where the number of 
different variations is limited/minimal such as in the above case or 
in the case of a Fax where you must convert Color to B&W and/or 
support 2 or 3 resolutions). I am working in a Blue Sky mode with 
these comments and I'm just tossing out a best functionality case 
which must/will be modified by implementation and real-world 
constraints.

>version of the message to all recipients is not appropriate.

Note: I am offering all of the above ideas/processing methods as 
suggestions for possible listings of ways to reach the goal of 
delivering the message to the recipient as if in a one-to-one mode 
while trying to minimize the overhead that is associated with 
one-to-one mode by leveraging the processing overhead of the message 
generation/uploading for additional addressees. As someone who has 
not been associated with the evolution of this effort I think my "new 
perspectives" can, at times, point out methodologies that might not 
be obvious those who have participated in its design. OTOH, I may be 
totally of base and my ideas may have already been raised and 
considered <g>.

---

From: Keith Moore <[email-masked]>
Date: Fri, 05 Jul 2002 14:08:28 -0400

> know unless the user has overridden the send both option).

I don't follow your reference to multipart/alternative.  The source
message in the example is HTML only.  And I don't think anyone has 
proposed that the SMTP client performing a conversion should use 
multipart/alternative to include both the old and the versions -
if nothing else, the fax stuff that is driving this has had the
ability to use multipart/alternative for 10+ years and they obviously
don't consider it sufficient for their purposes.

Also, B might not want HTML, not even in a multipart/alternative - 
and it seems quite reasonable to me that B would specify this 
(since I don't want it either).

The fact that many MUAs generate multipart/alternative with text+html
by default should not be cited as if it were a desirable practice.
 
> packaged between the Text and the PDF in the stacking order).

senders' MUAs are irrelevant to this discussion, unless they also perform
conversion based on SMTP CONNEG responses (which would reqiure them
to connect directly to recipients' ultimate MTAs - not likely)

the scenario is this - some MTA gets a message, it looks at the recipient
capabilities, and decides that it needs to convert the message to make
it acceptable to the recipients.  the point of my examples is to demonstrate
that there will be cases where there is no single message format that
is acceptable to all recipients, which implies that any MTA which attempts 
to deliver to more than one recipient at a time (and which uses CONNEG)
must be prepared to handle the case where different behavior is applied 
to different recipients - and this further requires RSET and/or 
close-connection-and-retry logic.
(even with a close RSET should be strongly recommended)

Keith

---

From: Dave Crocker <[email-masked]>
Date: Fri, 05 Jul 2002 16:13:53 -0700

At 01:58 PM 7/4/2002 -0400, Robert A. Rosenberg wrote:
>IMO, the ultimate aim (even if this is not practical) is to deliver to 
>each recipient the same version of the Fax as they would have received IF 
>they were the only addressee while at the same time generating only the 
>minimum number of copies to allow multi-

That is a worthy and reasonable use for this capability.  And, of course 
yes, it motivated creating this option.

However that level of "perfection" is not required.  There is nothing wrong 
with a service model that is satisfied with sending the least common 
denominator.

The choice is the sender's.

d/

----------

---

From: Keith Moore <[email-masked]>
Date: Fri, 05 Jul 2002 20:00:48 -0400

> The choice is the sender's.

as long as the sender gets to specify the conditions under which the
conversion is done, that seems just fine - it is no different than
allowing the the sender to personally look at a set of potential
recipients' capabilities and decide what the minimum requirements
for reception (and therefore the LCD format) should be.

however the current proposal doesn't really give the sender a way
to specify that threshhold - unless, perhaps, the negotiation/conversion 
is performed by an MTA that is under control of the sender (in which
case the threshold could be specified by explicit configuration)
and that MTA has the abliity to speak SMTP directly with MTAs that 
know the recipients' capabilities.

in other words, enabling the sender to make the choice is not 
compatible with the vast majority of the deployed email 
infrastructure (even assuming that those MTAs are extended to 
support this extension) 

Keith

---

From: Dave Crocker <[email-masked]>
Date: Fri, 05 Jul 2002 17:04:42 -0700

>to specify that threshhold - unless, perhaps, the negotiation/conversion

The protocol is MTA-MTA.

You are discussing UA-MTA.  That's a different protocol.  Feel free to 
draft a version.

d/

----------

---

From: Keith Moore <[email-masked]>
Date: Fri, 05 Jul 2002 20:47:02 -0400

> >to specify that threshhold - unless, perhaps, the negotiation/conversion

indeed, but you said that the choice of whether to downgrade to LCD
is the sender's choice, which is kind of an odd statement when 
in the vast majority of cases the sender doesn't have any control
over the MTA's decision-making process, and you haven't suggested
a mechanism to do this.

> You are discussing UA-MTA.  That's a different protocol.  Feel free to

I'm not discussing a different protocol, I'm discussing a characteristic 
of your proposal with respect to a claim that you made about your proposal.

Ketih

---

From: Dave Crocker <[email-masked]>
Date: Fri, 05 Jul 2002 17:53:31 -0700

At 08:47 PM 7/5/2002 -0400, Keith Moore wrote:

>is the sender's choice,

sending MTA.

d/

----------

---

From: Keith Moore <[email-masked]>
Date: Fri, 05 Jul 2002 21:02:11 -0400

> >indeed, but you said that the choice of whether to downgrade to LCD

ah, that's a slightly different thing.

but I really don't think it's a good idea to have MTA making value judgements
about the suitability of converting (whether or not using an LCD strategy)
in the absence of explicit direction from either sender or recipient.

Keith

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Mon, 8 Jul 2002 19:05:04 +0200

Keith Moore <[email-masked]>
> not you cc'ed recipient B.

Negotiation can mean that A and B receive different messages, both stating
(in to/cc) that the other received the one they didn't. Sounds very fishy
to my ears.

The thing sounds as if it's 100% incompatible with S/MIME and other
signature mechansisms. Shouldn't that be mentioned in the draft's security
considerations section?

--Arnt

---

From: [email-masked]
Date: Mon, 08 Jul 2002 10:20:21 -0700 (PDT)

> (in to/cc) that the other received the one they didn't. Sounds very fishy

This is nothing new; situations arise routinely where the header recipient
lists don't tell the full story about who did or did not get a message or a
particular version of a message.

> signature mechansisms. Shouldn't that be mentioned in the draft's security

On the contrary, far from being incompatible, this facility may actually
facilitate the ability to use signatures in situations where it presently isn't
possible to use them. Specifically, if a client is aware of a recipient's needs
it can tailor a message for that recipient and then sign it after the 
tailoring is complete. Without this knowledge what tends to happen is that the
tailoring is done after the signature is applied, which of course renders the
signature invalid.

Now you may argue that such transformations are a bad idea. And on a
theoretical level I'd agree with you. But on a practical level the need for
them is there, and I'd much rather see then done by the client than by some
intermediary.

The fact that multipart/signed is designed so that signatures can be attached
to content on the fly is also useful in this context.

As for mentioning any of this in the security considerations section, I
have no problem with that.

				Ned

---

From: [email-masked]
Date: Mon, 08 Jul 2002 14:08:04 -0400

> tailoring is done after the signature is applied, which of course renders the

Unfortunately, the draft as written doesn't provide a mechanism to push that
information back as far as the sending MUA - you could only get this to work
if the MUA included its own sender-side MTA functionality (which would be at
odds with the push to use port 587 and the practice of many ISP's of blocking
outbound to port 25 except from their mail servers).
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: Keith Moore <[email-masked]>
Date: Mon, 08 Jul 2002 18:18:14 -0400

> (in to/cc) that the other received the one they didn't. Sounds very fishy

That's an interesting point.  However it's always been possible for 
recipients listed in the to/cc fields to receive different versions
of the message, for some of those listed to not receive the message
at all, or for additional recipients to receive the message.  

Keith

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 9 Jul 2002 11:47:33 +0200

[email-masked]
> lists don't tell the full story about who did or did not get a message

Agreed. It's nothing new -- just more of a bad thing.

I suppose it's okay, though. If to/cc never were truly reliable, making
them even less reliable is no big deal. Even if it's the first time a
Proposed Standard does it.

> or a

Hm? I know that some mailing lists do rather a lot of header field
rewriting and/or append a new signature.  Are there other examples?

...
> have no problem with that.

Good. That's what I want.

--Arnt

---

From: [email-masked]
Date: Tue, 09 Jul 2002 02:48:04 -0700 (PDT)

> Proposed Standard does it.

Hardly the first time. MIME itself has the notion of downgrading from 8bit to
7bit or from binary to 8bit.

> rewriting and/or append a new signature.  Are there other examples?

Sure. There's plenty of content conversion/charset
conversion/reformatting/virus removal, etc. etc. etc. going on out there
already.

				Ned

---

From: Graham Klyne <[email-masked]>
Date: Tue, 09 Jul 2002 19:49:47 +0100

>infrastructure (even assuming that those MTAs are extended to

Having the sender make the final choice of format is fundamental to the 
content negotiation work completed so far for Internet fax, and is very 
much in line with the fax service model that we are trying to emulate.

(I assume that the main goal of the SMTP extension is to optimize this 
service model in the case where direct connection is available to a relay 
that can directly discover the final recipients' capabilities.)

#g

-------------------
Graham Klyne
<[email-masked]>

---

From: Keith Moore <[email-masked]>
Date: Fri, 12 Jul 2002 23:48:38 -0400

> much in line with the fax service model that we are trying to emulate.

okay, but I don't think that's clear from reading the document.  

and like I said, it won't work for current typical SMTP deployments -
for those the MTA making the decision is almost certainly going to be 
distant from the recipient.  

and given the huge problems with viruses and spam, I don't think we're
likely to see a movement to open up direct access to SMTP servers.

actually I have to wonder - will network admins open up port 25 even
to allow incoming traffic to go directly to their internet fax machines
when they won't do so for SMTP servers on general-purpose hosts?
in any network of sufficient size it seems like it would be a major
pain to keep track of which hosts were fax machines and which were 
something else.  far easier to just require all port 25 traffic to
go through a firewall of some kind.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Sat, 13 Jul 2002 16:57:35 +0900

>okay, but I don't think that's clear from reading the document.

Keith, if you have wording changes to suggest, please feel free to offer them.

d/

----------

---

From: Keith Moore <[email-masked]>
Date: Sat, 13 Jul 2002 08:43:55 -0400

> Keith, if you have wording changes to suggest, please feel free to offer them.

Dave,

This document cannot be fixed by a few simple wording changes, and your 
continued insistence that I only provide input in the form that you prefer 
(and that makes the document look better than it is) is, well, ridiculous.

I've said on multiple occasions that the simplest fix is to stop claiming
that the document is a general-purpose content negotiation mechanism for
SMTP, and to rename the keywords etc. appropriately to reflect that.
This wouldn't fix all of the problems with the document, but it's the single 
change that would have the greatest positive effect.

Naturally a general-purpose content-negotiation mechanism would be much better.
But we should be honest with ourselves that the current SMTP infrastructure
doesn't lend itself to that.

Keith