Subject: Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard

From: Gregory Neil Shapiro <[email-masked]>
Date: Tue, 2 Jul 2002 11:50:37 -0700

Some comments regarding draft-ietf-fax-esmtp-conneg-02.txt:

- It became increasingly difficult to understand as the draft inconsistently
  refers to the server as both "server" and "target", at times in the same
  section.  It makes it look like there is a third party ("client",
  "server", and "target").  "server" should be used throughout.

- Section 4.3 states:

     If the target support the CONNEG parameter, but it can not return 
     the recepient's capability temporarily, the target MUST reject the 

"recipient's" is spelled incorrectly.

- Section 4.3 states:

     If the client specifies "CONNEG = OPTIONAL" in the RCPT-TO, the 
     target MUST process the address and message as if the requested 
     CONNEG capabilities had not been specified.

If the server MUST ignore the CONNEG capabilities, then why should the
client even both sending it?  I think this MUST should be a MAY.

- Section 4.3 states:

     Regardless of the value of the parameter, if the target does 
     support the CONNEG parameter, then it MUST issue a 250 reply, 
     followed by the capabilities of the target that is specified by 
     the RCPT-TO address.

I think this should be changed to:

     Regardless of the value of the parameter, if the target does 
     support the CONNEG parameter and the address is acceptable,
     then it MUST issue a 250 reply, followed by the capabilities of
     the target that is specified by the RCPT-TO address.

Note the addition of "and the address is acceptable".  The draft should not
specify that the command "MUST issue a 250 reply" as there may be other
reasons outside of the scope of CONNEG not to do so.

- Section 4.3 states:

     If the SMTP server supports ENHANCEDSTATUSCODES, the response 
     strings for a success are "250-2.1.5 CONNEG" and "250 2.1.5 
     CONNEG". The response strings for indicating a permanent 
     failure are "504-5.3.3 CONNEG" and "504 5.3.3 CONNEG". 
     The response strings for a temporary failure are "404-4.3.3 
     CONNEG" and "404 5.3.3 CONNEG".
                      ^^^^^

That last enhanced status code should be 4.3.3, not 5.3.3.

- Section 5 states:

     Command with "CONNEG":
      "RCPT TO:" ("<Postmaster@" domain ">" / "<Postmaster>"
                   / Forward-Path) (SP "CONNEG =" ("REQUIRED" /
                 "OPTIOANL") CRLF

* Why is postmaster specifically listed in the syntax?
* "OPTIONAL" is spelled wrong in the syntax
* Why are spaces required around the CONNEG, the =, and the REQUIRED or
  OPTIONAL?  In all other ESMTP RCPT extensions, spaces separate
  extensions.  As an implementor, I would prefer "CONNEG=REQUIRED", not
  "CONNEG = REQUIRED".

- Section 5 states:

     Reply:
      ( ("250-" CRLF) *("250-CONNEG" capability CRLF)
        ("250 CONNEG" capability CRLF) )/
      ( ("250-2.1.5" CRLF) 
          *("250-2.1.5 CONNEG" capability CRLF)
           ("250 2.1.5 CONNEG" capability CRLF) )/
      ("504" CRLF) /
      ("504 5.3.3" CRLF) /
      ("404" CRLF) /
      ("404 4.3.3" CRLF) /

The syntax shown for replies does not allow for any text after the SMTP
code.  E.g., this would be illegal according to the draft syntax:

	250-Recipient Ok
	250 CONNEG ....

As the only thing the syntax allows to follow the 250- is a CRLF.
This applies for all of the replies, not just the 250 case.

Note that all of draft examples below break the rules set out by the
syntax for replies.

- Section 6 states:

     C: RCPT TO:<[email-masked]> CONNEG = REQUIRED
     
Again, the spaces make this difficult to parse given that all other ESMTP
arguments will not have spaces between the key and value.

- Section 6 states:

     S: 250-<[email-masked]> recipient ok
     S: 250 CONNEG (&(image-file-structure=TIFF-minimal)
     S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
     S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
     S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
     S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
     S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
     S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
     S: (ua-media=stationery) )

That makes it look like the CONNEG response spans 8 lines.  If that's the
case, any SMTP client should reject it as it doesn't use proper SMTP
continuation.  This should be:

     S: 250-<[email-masked]> recipient ok
     S: 250-CONNEG (&(image-file-structure=TIFF-minimal)
     S: 250-CONNEG (MRC-mode=0)(color=Binary)(|(&(dpi=204)
     S: 250-CONNEG (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
     S: 250-CONNEG (dpi-xyratio=[200/100,1]) )(&(dpi=400)
     S: 250-CONNEG (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
     S: 250-CONNEG (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
     S: 250-CONNEG (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
     S: 250 CONNEG (ua-media=stationery) )

- Section 6 states:

     An example of succssful RCPT-TO response when CONNEG-capable 
     server supports ENHANCEDSTATUSCODES.

"successful" is spelled incorrectly.

     S: 250-2.1.5<[email-masked]> recipient ok
     S: 250 2.1.5 CONNEG (&(image-file-structure=TIFF-minimal)
     S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
     S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
     S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
     S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
     S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
     S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
     S: (ua-media=stationery) )

Again, the same comment as above.

- Section 6 states:

     An example of ESMTP sequence with parmanent failure RCPT-TO 
     response. 

"permanent" is spelled incorrectly.

     S: 220 ifax1.jp IFAX
     C: EHLO ifax1.jp
     S: 250-ifax1.jp
     S: 250-DSN
     C: MAIL FROM:<[email-masked]>
     S: 250 <[email-masked]> sender ok
     C: RCPT TO:<[email-masked]> CONNEG = REQUIRED

The comment above should probably indicate that this is a broken client.
The client shouldn't be sending the "CONNEG = REQUIRED" since CONNEG was
not given in reply to the EHLO command.

Also, this is invalid -- the reply to the EHLO command doesn't end the
continuation.  It should be:

     C: EHLO ifax1.jp
     S: 250-ifax1.jp
     S: 250 DSN

Note the DSN line is "250 DSN" instead of "250-DSN" as shown in the
example.

- Section 6 states:

     An example of an ESMTP sequence with temporary failure RCPT-TO 
     response when the value of parameter is "REQUIRED":
...
     C: RCPT TO:<[email-masked]> CONNEG = REQUIRED
     S: 404 <[email-masked]> recipient ok

Should the text for a '404' be 'recipient ok'?  Perhaps, it should be
'conneg temporary failure'.

- Section 7 states:

     On publicatiom of this document by the RFC Editor, the IANA 
     shall register the Content_Negotiation ESMTP extension defined 
     in section 3.

"publication" is spelled incorrectly.

- Section 12 states:

     Copyright (C) The Internet Society (2001).  All Rights
     Reserved.

Should this be updated to include 2002?

---

From: Alexey Melnikov <[email-masked]>
Date: Tue, 02 Jul 2002 15:33:56 -0600

More comments about the document.

The draft doesn't explain what to do if the same fax messages was sent to
multiple recipients with different rendering capabilities.
Should it downgrade for the least capable recipient? Or should it use RSET and
start multiple mail transactions to differently capable recipients? Correct me
if I am wrong, but it seams that extending VRFY command is more appropriate for
the purpose than using RCPT.

Regards,
Alexey Melnikov
__________________________________________
R & D, ACI Worldwide/MessagingDirect
Richmond, Surrey, UK
Phone: +44 20 8332 4508
Home Page: http://orthanc.ab.ca/mel

I speak for myself only, not for my employer.
__________________________________________

---

From: Dave Crocker <[email-masked]>
Date: Tue, 02 Jul 2002 17:15:52 -0700

At 07:49 PM 7/2/2002 -0400, John C Klensin wrote:
>--On Tuesday, 02 July, 2002 15:33 -0600 Alexey Melnikov
><[email-masked]> wrote:
> >  Correct me if
> > I am wrong, but it seams that extending VRFY command is more
> > appropriate for the purpose than using RCPT.
>
>Hmm.  _That_ is an interesting idea, since we have always
>permitted VRFY to do somewhat more poking around in real time
>than one might normally expect of RCPT (where "poking around"
>efforts more often result in a 250 code and an email rejection
>if needed).

If I understand both your comments and Keith's suggestion, the key is a 
separate command.

It, too, find that aesthetically preferable.  The problem is with efficiency.

A separate command means 2 commands per recipient and it means at least one 
synchronization event, between issuance of this new command and issuance of 
the first RCPT-TO.

That's quite a bit of overhead, for something that frequently will have no 
effect on the content being transferred.

d/

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

---

From: [email-masked]
Date: Tue, 02 Jul 2002 17:20:24 -0700 (PDT)

> > The draft doesn't explain what to do if the same fax messages
> > was sent to multiple recipients with different rendering
> > capabilities. Should it downgrade for the least capable
> > recipient? Or should it use RSET and start multiple mail
> > transactions to differently capable recipients? Correct me if

> if needed).  While VRFY is, I think, less and less often
> supported in a serious way, an SMTP extension that could not be
> advertised unless it was supported for this function would get
> around that problem.

I considered proposing this back when I initially reviewed the document. It is
certainly far more aesthetically appealing to me than the RCAP command Keith
proposed.

However, after looking at the actual use cases I become convinced that this
isn't the clear win it appears to be a first glance.

The advantage of having a separate command that returns recipient capabilities
withing adding that recipient to the set of active recipients is, of course,
that you don't have to RSET and start over if you uncover an incompatibility
between multiple recipients.

But there's a serious side effect: Advance capability retrieval tends to stall
the pipeline. And while it is natural to view SMTP as a series of commands and
responses, the way modern SMTP clients and servers need to work to be efficient
is by sending batches of commands and getting back batches of responses.

The naive implementation of capabilities exchange in a separate command then
becomes the worst case scenario: If you send a VRFY (or RCAP) prior to sending
each RCPT TO you end up with at least one round trip per recipient.

Of course you can work around this by sending all the VRFYs/RCAPs in a batch
prior to sending any RCPT TO. But the implementation complexity  here is
starting to be a serious issue.

Another alternative would be to allow capabilities retrieval on either RCPT TO
or VRFY. But now the client has a lot of options and approaches to choose from,
which I don't think is a good thing.

I think the determining factor needs to be how clients will actually handle
recipients with differing capabilities. If the least common denominator is
going to be the dominant approach (and I suspect that it will be) then RCPT TO
capabilities return is the simplest and most direct approach. If, on the other
hand, messages tailored for a each recipient is going to be the norm, then
VRFY is the correct approach.

				Ned

---

From: [email-masked]
Date: Tue, 02 Jul 2002 17:46:34 -0700 (PDT)

Our responses seem to have crossed...

> synchronization event, between issuance of this new command and issuance of

You can avoid the need for additional round trips by batching all the
capabilities queries with the first RCPT TO. But this results in a rather sharp
rise in implementation complexity.

> effect on the content being transferred.

Indeed. The average number of recipients for legitimate mail does tend to be
quite small, which implies we're optimizing the uncommon case here.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Tue, 02 Jul 2002 18:09:04 -0700

At 05:20 PM 7/2/2002 -0700, [email-masked] wrote:
>going to be the dominant approach (and I suspect that it will be)

no surprise that I agree.

However...

>  If, on the other

If the sender is concerned about optimizing for each recipient, they can 
get that effect by reducing to a single RCPT-TO per DATA.

So it is not as pretty as the separate command, but it permits roughly the 
same mode of operation.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 22:14:27 -0400

> get that effect by reducing to a single RCPT-TO per DATA.

if several recipients have similar capabliities, sending a separate
copy of each message hardly seems like an optimization.

> So it is not as pretty as the separate command, but it permits roughly the

overloading RCPT is simply unacceptable.  It would be a Bad Idea for
any SMTP extension, but this proposal pre-empts RCPT error reporting 
for a very specific application.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Tue, 02 Jul 2002 19:39:46 -0700

>copy of each message hardly seems like an optimization.

Keith, the one thing that is certain is that no specification can satisfy 
requirements that are constantly being changed.

You expressed concern about content being tailored to each recipient's 
capabilities.  This proposal supports that.  You would prefer to achieve 
that in a different way, with is certainly your right, but the current 
specification satisfies it.

Now you are expressing concern about content that is tailored to "several 
recipients" that have "similar" capabilities.  That is quite different from 
anything you have raised before.

I am sure that you can discover all sorts of boundary conditions that are 
unlikely and for which the current specification is not optimized.

And, of course, you continue to ignore minor matters of efficiency for what 
is expected to be the typical cases, namely sending a common set for all of 
the recipients and sending a single content for a single recipient.

>overloading RCPT is simply unacceptable.

Keith, your term "overloading" suggests that RCPT-TO parameters are being 
invented here.  They aren't.

In any event technical discussions are facilitated by have a discussion 
about technical merits.  They usually are not facilitated by having 
participants pretend that they have a veto.

d/

----------

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 22:54:28 -0400

> synchronization event, between issuance of this new command and issuance of

The bandwidth required by the extra command is negligible.  I don't think
the extra command needs to add another synchronization event; offhand it looks
like it can piggyback on the event following the MAIL command.  Since all 
downgrade decisions should be made by the client anyway, there's actually 
no need for the RCAP command to change the state of the SMTP protocol engine.  

In other words, the client would be be doing

MAIL
RCAP a
RCAP b
RCAP c
<wait for responses, then decide what conversions to do for what recipients>
* ( *RCPT DATA . )

instead of

MAIL
<wait>

even if another synchronization event were needed, I don't think that's
a big price to pay, particularly when the alternative (from a protocol
architecture perspective) is to clutter up the RCPT command and
(from a implementor perspective) to either 

a) require a separate RCPT/DATA exchange for each recipient, 
   (far more overhead than your extra synchronization event)

b) downgrade to the least common acceptable format for all recipients
  (assuming there is one - which might be true for fax but is not
   a reasonable assumption in general), or

c) send RSET following all the RCPTs if it is desirable to send
   different versions to different recipients, followed by 
   another set of RCPT/DATA exchanges - again resulting in far more
   overhead than a single additional synchronization eent. 

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 23:05:15 -0400

> rise in implementation complexity.

surely is it less complex than what is required by the current protocol - i.e.
doing a RSET followed by additional MAIL and RCPT commands.

> > effect on the content being transferred.

actually I don't see why putting the conneg response in a separate command
is "quite a bit of overhead" - the extra bandwidth required for
"RCAP <recipient>" is neglibible, there's no difference in the response
at all, and the client logic to deal with separate commands is simpler since
the client doesn't have to distinguish ordinary RCPT failures from CONNEG
failures. and the client doesn't have to do RSET followed by a separate set
of MAIL/RCPT commands for each recipient if for some reason it wants or needs
to deliver different versions of the message to different recipients.

> quite small, which implies we're optimizing the uncommon case here.

to understand the tradeoff it may be useful to compare each proposal under 
the same set of scenarios.  e.g. 

a. single recipient
b. multiple recipients, all with similar capabilities
c. multiple recipients, different capabilities
d. multiple recipients, mutually exclusive capabilities

with each case evaluated for 
a. ease/simplicity of implementation, 
b. necessary round-trip delays,
c. future extensibility

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 23:24:34 -0400

> each RCPT TO you end up with at least one round trip per recipient.

I don't understand which argument you're making - is it that the naive 
implementation is slow, or that it's too hard to do an efficient 
pipelining implementation?

as long as RCAP doesn't force any new decision points for the client,
I don't see why the efficient pipelining implementation is any more 
difficult for the RCAP case than the RCPT/RSET case.

if RCPT is overloaded then the client sends multiple RCPTs, waits for
all of the responses, then decides whether to just send one copy or
do RSET/MAIL/etc and send multiple copies.  

if there's a separate command then the client sends multiple RCAPs,
and one RCPT, waits for all of the responses, and then decides
whether the version that's right for the first recipient is also
appropriate for other recipients.   

seems like the implementation complexity is about the same in either case. 
either way the sender wants to divide up the recipients into separate 
classes and batch up the messages to each.  assuming it's trying to
optimize, that is.  of course the naive/inefficient implementation is 
quite easy (and equally slow) for both proposals - just use no more 
than one recipient per SMTP session.

but it's hard to see how the client can avoid implementing some 
kind of logic to convert differently for different recipients, since
there's clearly the potential for the recipients' conneg strings to 
have no format in common.  perhaps if the client can only do one
or two different kinds of conversion (otherwise it has to bounce)
then the decision becomes a bit easier, but this again looks more
like a fax-specific facility than a general one.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 02 Jul 2002 23:36:43 -0400

> capabilities.  This proposal supports that.  

it doesn't support that explicitly, and the only ways that you can do it 
with your proposal are more complex and/or expensive than the alternative 
I've suggested.

> You would prefer to achieve

the current specification allows it, though it doesn't explain how
it allows it, and it's incomplete.

> anything you have raised before.

there are different cases that need to be considered, and I don't claim
to have enumerated them all yet.  nor do I believe there's a requirement
to enumerate all of my concerns with your proposal in a single round-trip.  
indeed, I'm still trying to articulate some of them. 

> unlikely and for which the current specification is not optimized.

and among them, some boundary conditions which are likely and some failure
modes which aren't considered.  but of course you'll assume that everybody
will use the protocol exactly as you intended, even though you didn't bother
to even explain how you assumed the protocol would be used.

that's no way to write a standard.

> the recipients and sending a single content for a single recipient.

I didn't ignore it.  but you failed to mention these in the document.

and somehow you seem to think it's okay to insist on a minor optimization
of saving a round-trip by overloading RCPT when you have the major lossage
of having to send separate copies to each recipient when there's no common
set available.  

nor is it clear to me why downgrading to a single common format, just because 
the sender happened to specify multiple recipients on the TO, cc or bcc field,
is acceptable from the sender or recipient's point of view.  (last time I 
sent you a fax you got high quality color, but this time you got low-res 
black/white ... what gives?)

even if it's okay with the fax people I'd take serious issue with this
kind of haphazard conversion being applied to ordinary email.

> invented here.  

no, it suggests that overloading the RCPT reply code to indicate a condition
other than "this message cannot be sent to this recipient" is a bad idea.

> participants pretend that they have a veto.

nor are they facilitated by having the proposer pretend that His is the
unimpeachable Word of God and trying to deflect technical criticisms
with personal attacks.

Keith

---

From: [email-masked]
Date: Tue, 02 Jul 2002 21:21:58 -0700 (PDT)

> pipelining implementation?

Both: The naive implementation has latency problems while the efficient
pipelining implementation is more complex than it needs to be.

> difficult for the RCAP case than the RCPT/RSET case.

Then I suggest you haven't actually looked closely at how you'd actually code
this. Two responses for each recipient lead to a bunch of corner cases that
have to be dealt with.

> seems like the implementation complexity is about the same in either case.

Sigh. This is pretty much a strawman argument, but rather than get into that
let me simply repeat my earlier conclusion: I believe that if support for
sending more than one variant of the same message is likely then a separate
command makes sense. If, on the other hand, this is the very uncommon case,
then RCPT TO piggybacking is much simpler and a separate command doesn't make
sense.

Do you really wish to claim otherwise?

> like a fax-specific facility than a general one.

On the contrary, I suspect that this issue can and will be avoided. John
phrased it well: Select the least common demoninator that's above a certain
threshhold. That sounds like the right implementation strategy to me.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Wed, 03 Jul 2002 01:26:41 -0400

> pipelining implementation is more complex than it needs to be.

okay, I'll certainly grant that the naive implementation has worse latency 
if you use extra commands.  though it seems like if you're going to have
the complexity of parsing conneg strings and doing format translation,
the additional complexity of pipelining doesn't make it much worse.

as for the pipelining, to me the complexity seems about the same
either way, but with overloading of RCPT response codes then 
you're asking the server to declare failure modes that really 
need to be detected by the client, and having the server report 
the error doesn't seem to simplify the client logic at all.  

nor have I managed to find a use case for treating no conneg 
string as a fatal error, though I don't claim to have exhaustively
analyzed all possibilities yet.

> this. Two responses for each recipient lead to a bunch of corner cases that

no, I haven't tried to code it in detail yet.  but it doesn't seem like
rocket science to map a cross product of the 3 or 4 existing RCPT
responses that clients need to distinguish, and the 3 or so conditions
for RCAP (here is the string, there is no string, tempfail) onto
a small number of cases.  and the client pretty much has to do that anyway,
because there can be failures due to inability to convert even if all 
RCAPs return a conneg string.

> Do you really wish to claim otherwise?

Hmm.   If I'm trying to compare protocol and implementation complexity then 
I care about the amount of code that's required to accomodate all cases even 
if many of those cases aren't used very often.  If I'm thinking about protocol 
efficiency then I care more about the commonly occuring cases.

as for what is a commonly occuring case, I get a different answer depending
on whether I think of this as a fax-specific mechanism or a general purpose
SMTP mechanism.  if it's only fax specific then I suspect the common case
is for most messages to have only one recipient, so if the rare multi-recipient
message takes a slow path that might not be so bad.  if the mechanism is
more general purpose then I suspect multi-recipient messages (and widely-varying
capabilities) will be much more common and it will be more desirable to
avoid sending a separate copy of each message, and this in turn makes 
reasonable implementations (of either proposal) more complex.

The other concern I have is something like "protocol architecture" -
will this extension mess up the protocol in such a way that it
creates silly states or makes it more difficult to add other kinds of
extensions?

what I'm discovering is that I'm much more concerned about overloading
the RCPT reply/status code rsponses (using them to mean two different)
things than I am about piggybacking a conneg string onto the RCPT
response.   if the conneg response didn't affect the reply and status 
codes, and if other extensions could also piggyback additional data
onto the RCPT response, then I don't think that the fact that additional
information is being requested along with RCPT would bother me.  the 
thing that bothers me is that RCPT error codes are being used to signal a 
case that really has nothing to do with the recipient's ability to 
receive a message.

>t 

that assumes that there is a LCD, which is not true in general.
(though it might be true for a fax-only extension.)

and unless the set of permissable content-types is severely restricted,
how can the client know what is a reasonable threshhold?

Keith

---

From: [email-masked]
Date: Tue, 02 Jul 2002 21:57:41 -0700 (PDT)

> Is this complex?  Sure it is.  But, if one is going to end up
> with least common denominator, the limiting (and probably
> common) case is to just send that and forget about capabilities
> negotiation.  Or, if the number of useful differentiated formats
> is _very_ small, just sending both with multipart/alternative
> might turn out to be easier and cheaper than messing around
> trying to find feature intersections.  There isn't nearly enough
> information in the spec for me to be able to guess if that would
> be the case.  There shouldn't be if this is a general purpose,
> multiple-recipient, capabilities negotiation mechanism but, if
> it is, I don't see much hope other than dealing with the
> complexity (or deciding we don't need something that general).

I guess I remain to be convinced either way: That multiple versions schemes are
necessary or that a single version schemes are sufficient. I'd guess there's a
sweet spot somewhere in there. Unfortunately in this particular case the
location of that spot drives a bunch of design choices.

> > I think the determining factor needs to be how clients will
> > actually handle recipients with differing capabilities. If the
> > least common denominator is going to be the dominant approach
> > (and I suspect that it will be) then RCPT TO capabilities
> > return is the simplest and most direct approach. If, on the
> > other hand, messages tailored for a each recipient is going to
> > be the norm, then VRFY is the correct approach.

> Ignoring all of the relay issues, which I still don't think the
> document addresses,

I must say I don't see any of this as particularly useful in a relay scenario.

> and the fallback to fax while claiming it
> isn't fax-specific, etc., I'd partition the above a bit
> differently.  I'd guess the dominant approach (especially if one
> generalizes beyond fax) is going to be greatest (most capable)
> common denominator above a threshold.  I.e., if there is some
> minimal, default, capability, and some recipients can support
> only that (e.g., they don't handle option negotiation at all),
> then two messages are going to be sent: one corresponding to the
> common feature set of the more capable recipients and one for
> the minimal/default feature set.  Everyone gets downgraded to
> the minimum/default because one recipient isn't highly capable
> just doesn't feel commercially viable to me, while assuming a
> mix of high-function and low-function hosts does.

I certainly agree with the common denominator above a threshhold. But given
that a single recipient per destination is likely to be a very common case, I'm
less sure that simpler approaches won't be commercially viable. Or perhaps
capabilities will be used to reduce the number of variants sent with
multipart/alternative.

> Other
> combinations are possible, but my sense of reality says they
> would rapidly turn into message-copy-per-recipient rather than
> strange combinatorics on feature sets.

It may indeed turn out that per-recipient message copy is the right way to
finesse the feature set combination issues.

> But, in that "two message" case, one really doesn't want 5yz
> codes coming back from the server in response to a RCPT TO.  If
> the "one digit" rule is observed, there is no way to
> differentiate between "bad mailbox" and "bad parameter".  If it
> is not, and three digits are processed, the server cannot
> effectively differentiate between a bad parameter to CONNEG and
> one to some other option by the code -- the reply string has to
> be parsed, and that leads to bad trouble.   So I would think
> that a batch of capability-determining commands before the first
> RCPT one would be a better idea, complexity notwithstanding.

I'm not sure I get what you're driving at here. The client gets
to choose whether the server treats a recipient with unknown
capabilities as an error or simply by indicating as much.

> Again, the way to get most of the complexity out of this, IMO,
> is to change the problem.  For example, if the server makes the
> determination of the common capability set among all recipients
> and returns it only once (a bit tricky with the SMTP model, but
> not impossible), then the client doesn't have to figure out how
> to intersect whatever comes back, break things up into separate
> messages, etc.  But I don't know if that would do the job; it
> certainly doesn't handle a mix of high- and low-function
> recipients in a plausible way.

I considered proposing this as well, but it seemed to be that knowledge
of what the client can actually do may determine how differing
feature sets are combined.

				Ned

---

From: [email-masked]
Date: Tue, 02 Jul 2002 23:09:19 -0700 (PDT)

> analyzed all possibilities yet.

The obvious case would be one where interoperability between sender and
recipient has to be guaranteed. Whether this is a case we should care
about is another matter.

I note in passing that the original proposal required treating the lack of a
conneg string as a fatal error. I saw far more utility in allowing the case of
no conneg string to be allowed without an error.

> reasonable implementations (of either proposal) more complex.

I'm less sanguine about making this assessment a FAX-vs-other-uses thing, but
the bottom line is that I think we're pretty much in agreement here.

> case that really has nothing to do with the recipient's ability to

Well, it is certainly possible to arrange the additional conneg data so it is
identifiable as such. As for not affecting the response, I'm not sure I see the
problem: If the client wants an interop guarantee, it requires a conneg
response, and handles 4yz and 5yz codes as temporary or permanent failures
without regard to what's causing the error. If, on the other hand, it wants
conneg info as long as its available, it makes a conneg response optional, in
which case the status codes no longer have anyhing to do with conneg
information.

> how can the client know what is a reasonable threshhold?

Such thresholds are driven by the client's own capabilities, not on the
capabilities of the recipient.

				Ned

---

From: [email-masked]
Date: Tue, 02 Jul 2002 23:43:22 -0700 (PDT)

> First, let me say that I haven't read the draft, and don't intend to.
> So, if you want to ignore my comments (questions really) feel free.

Fascinating. Although your detailed understanding of how this works is fairly
far off the mark, you nevertheless have suggested a viable approach that I, at
least, hadn't considered.

> But from what I have picked up from the (smallish) number of messages
> from an even smaller set of people (no great surprise, fax is dead
> technology, who cares???) on the ietf list in the past day or so, I
> surmise that this protocol work something like the following.

At least part of the problem is that the potential use of this stuff extend far
past FAX.

> It sounds as if what is being proposed is that the SMTP client
> sends

> 	RCPT TO:<address> "I want 1200 dpi"	

It is more like "RCPT TO:<address> "tell me what address can handle:"

> Then the SMTP server says

> 	250 OK

More like: "250 address can handle 1200 dpi".

> If this is how it works, I don't understand why people are talking about
> doing RSET?   Surely the protocol isn't built in a way where the server
> says "no" to a request, and yet still includes the address in its list
> of recipients.

it can say "recipient accepted but capabilities are unknown"

> Keith's operational plan can be done by sending RCPT TO (where he would
> send RCAP) and terminating the sequence with RSET.   This way the RCPT
> TO acts just as the RCAP would act in his model (as I understand it).
> Sure, it isn't as clean by any means, and makes the server do a little
> extra work, as it doesn't know the RCPT TO's aren't real ones when it
> receives them, but as a trade off, it seems OK.

Ah. Using the first round of RCPT TOs as capabilities checks only and sending
the RSET unconditionally is really quite cute. It still has the added
complexity of having multiple commands per recipient, but now it is the same
command repeated, which eliminates a bunch of the edge conditions and makes the
silly states very silly indeed.

> Others who prefer just to send and see what arrives, just do that, and
> it all works the way they expect, with down grading, so the data only
> ever needs to be sent once, or with separation of capabilities in the
> cases (I am guessing this probably doesn't happen often, and most times
> a server will offer the same capabilities to all addresses it can handle?)

I suspect there will be cases where the capabilities will be the same and
others where they are quite different.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Wed, 03 Jul 2002 09:52:09 -0400

> without regard to what's causing the error. 

I guess I don't see how this helps the client much, because you can get
interop failures even when there is a conneg string for each recipient.

So it seems that the proposal could be usefully simplified  by removing the
OPTIONAL vs REQUIRED flag from the CONNEG parameter, and having the server
report back the CONNEG status (yes, no, or tempfail) as part of the 
response rather than in the reply/status codes.

If the client wants an interop guarantee (how does the sender specify this?)
then it can reat the absence of the CONNEG string in the response, or the
absence of CONNEG in the EHLO response, as a failure.  This should really
be a client-generated error anyway since there's really no error on the server
side.

I think this would be a significant improvement - particularly if this extension
isn't generalized to support things other than fax and it then becomes necessary
to add another extension later for voice mail or whatever.

Keith

---

From: Keith Moore <[email-masked]>
Date: Wed, 03 Jul 2002 09:55:59 -0400

> the RSET unconditionally is really quite cute. 

"cute" is rather opposite of the four-letter word I would have chosen.

> It still has the added

well, I'll admit I'm curious - how does using the same command in two different
ways eliminate a bunch of edge conditions?  (if we had only one command in SMTP
and used it over and over would that eliminate even more edge conditions?)

really, I don't have a guess as to what you're talking about here, and I am curious.

Keith

---

From: [email-masked]
Date: Wed, 03 Jul 2002 09:40:06 -0700 (PDT)

> and used it over and over would that eliminate even more edge conditions?)

The edge conditions arise when the responses from the different commands
disagree in some way. And this is likely to happen when there are two commands
and two different code paths. The usual outcome then is that things which
shouldn't make a difference do make a difference, and that then leads to the
time honored whine of "but it works when I do it over there so this stuff here
must be broken".

> really, I don't have a guess as to what you're talking about here, and I am

Apparently you managed to miss the round of SMTP clients that did a VRFY prior
to each RCPT TO. Horrible mess, just horrible.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Wed, 03 Jul 2002 13:02:24 -0400

> time honored whine of "but it works when I do it over there so this stuff here

ah.  well it was sort of obvious to me that RCAP responses should be defined to
provide no indication whatsoever as to whether the recipient is valid, so you 
couldn't get a conflict between a RCPT indication and an RCAP indication.  

on the other hand either command could return a value that would prevent delivery
of the message - RCPT could return 5xx and RCAP could return "this recipient
supports no content types you've ever heard of", and either one could cause a
temporary failure of message delivery.  I don't consider that a silly state - 
it's a natural consequence of being able to specify what content types you're 
willing to accept.

however I really don't have a problem with an option to RCPT to return
UA capabilities as long as it is an extensible mechanism (other options could
return other information and could be asserted at the same time as CONNEG) 
and as long as it doesn't affect the response and status codes.

Keith

---

From: [email-masked]
Date: Wed, 03 Jul 2002 09:31:50 -0700 (PDT)

> interop failures even when there is a conneg string for each recipient.

It helps the client because it can now distinguish between temporary and
permanent conneg failures.

> response rather than in the reply/status codes.

That would also work, but I don't think it is a useful simplification because
it creates even more instances where the command succeeded but the client
needed for it to fail. The server side of this is trivial either way.

> to add another extension later for voice mail or whatever.

I don't think you have sufficiently demontrated the problematic nature of
REQUIRED to claim this is a significant improvement. However, absent a
convincing argument that such interop guarantees would actually be used, I
personally don't have a problem with making this simplification.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Wed, 03 Jul 2002 12:56:02 -0400

> permanent conneg failures.

granted it's necessary for the client to be able to do that, but my point
is that having the server report conneg faliures in the RCPT status code
doesn't relieve the client of the necessity to detect other conditions
resulting from valid conneg strings that can cause delivery failure.
it's not as if the client can rely only on the response code to determine
how to handle the message.

> needed for it to fail. The server side of this is trivial either way.

what it accomplishes is to clearly separate different potential sources of 
failure into different indications.  a RCPT status codes indicating failure
would mean that the server has a problem with sending the message to the 
recipient, as distinct from failures caused by the client's inability
or unwillingness to deliver the message to the recipient based on the
server's conneg report for that recipient.

right now we have the situation where the server is essentially being asked 
to report an error (and an error code) which is really happening on the client.
the server doesn't know the reason that the client doesn't want to deliver 
the message to the recipient, and it shouldn't be expected to report the
error (or the enhanced status code) in the absence of such knowledge.

that, and once we've established that it's okay to use RCPT to request 
information about the recipient's UA capabliities and preferences,
it's hard to imagine that there won't be a demand to use RCPT to request
other information about the recipient's UA or preferences, such as
whether the recipient will honor MDN requests, what languages the
recipient can read, whether the recipient wants to opt-out of spam, 
whether the recipient supports vendor-proprietary extension FOO, etc.
(I didn't say that I liked all of these ideas, just that these are
obvious uses of a facility that lets you query UA capabilities.
OTOH I do like some of them.)

and if we go down that path we might really regret having allowed CONNEG
to alter the RCPT reply code.

Keith

---

From: [email-masked]
Date: Wed, 03 Jul 2002 10:12:06 -0700 (PDT)

> how to handle the message.

Sure, but it is only useful to treat such failures as permanent.

> server's conneg report for that recipient.

I remain unconvinced that this is a useful distinction. What REQUIRED says is
"I absolutely require CONNEG info and as such a failure to have it is the same
to me as any other sort of failure and should be responded to accordingly".

> error (or the enhanced status code) in the absence of such knowledge.

I guess this is a valid assessment, but somehow it doesn't offend me
the way it offends you.

> OTOH I do like some of them.)

Language preferences are, I believe, already defined as something you can get
out of feature algebra.

> to alter the RCPT reply code.

Why? If the client doesn't want CONNEG to alter the status all it has to do is
use OPTIONAL rather than REQUIRED. Again, REQUIRED implies that the client is
willing to treat different sorts of failures in the same way.

Again, I think a means of distinguishing conneg material in the response is a
good idea and I'm not opposed to eliminating REQUIRED. But really don't believe
REQUIRED causes the problems you think it does.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Wed, 03 Jul 2002 16:24:32 -0400

> Sure, but it is only useful to treat such failures as permanent.

how so?  if there is a temporary failure to get conneg information, the
client might quite reasonably decide to retry later.   or if the client
can't translate part of the message, it might reasonably decide that
delivering the remainder of the message is preferable to bouncing it.

> to me as any other sort of failure and should be responded to accordingly".

it's still poor separation of function, and it's still asking the server
to report a failure condition which the client understands better than the
server does.  E.g. the server is told to report errors as 4.3.3 or 5.3.3
when codes x.6.1 or x.6.2 or x.6.3 might be more appropriate.  The server
can't tell the difference between those conditions; the client can.

> I guess this is a valid assessment, but somehow it doesn't offend me

My offense is for the document taken as a whole, especially the lack of
attention to non-fax applications.

> Language preferences are, I believe, already defined as something you can get

perhaps, these were just off the top of my head.  and I don't think the 
feature algebra yet contains everything that a recipient might like to specify.  

> willing to treat different sorts of failures in the same way.

well, if we had such other features then the client wanting to assert such other
features would not be able to assert CONNEG=REQUIRED, it would just have to
implement the error detection by itself.  

> REQUIRED causes the problems you think it does.

The overall effect of REQUIRED is probably to increase the number of 
implementation errors and the number of mis-reported delivery failures.

It's by no means the worst thing I've ever seen in a protocol, nor is
it the worst thing in this proposal.  Getting rid of REQUIRED just 
makes the protocol slightly cleaner.  It doesn't get rid of the RSET
ugliness, it doesn't address the applicability of this extension to
non-fax scenarios, nor the lack of attention to other issues.
These other things need to be considered also.

But given the importance of SMTP and the fragility of the mail system
I think that the fewer warts we add to SMTP the better off we'll be.  

Keith

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Wed, 3 Jul 2002 16:16:36 -0400

At 18:09 -0700 on 07/02/2002, Dave Crocker wrote about Re: Last Call: 

>So it is not as pretty as the separate command, but it permits roughly the

This "one address per copy" approach insures that there will ALMOST 
ALWAYS be wasted processing and message uploading in any situation 
where knowing the capabilities prior to message transmission would 
have allowed the piggybacking of additional addresses onto a copy of 
the message (IOW: 5 message versions going to a total of 25 addresses 
instead of 25 copies each addressed to one of the 25 addresses saving 
thus saving 20 message transmissions). You need to decide what the 
trade-offs are between minimal number of separate copies (with its 
processing overhead) and the one message copy per address (with its 
wasting of bandwidth and throttling of delivery speed due to queuing 
of messages that could have been passed to the server as part of a 
prior transmission).

---

From: Graham Klyne <[email-masked]>
Date: Wed, 03 Jul 2002 21:11:33 +0100

>Language preferences are, I believe, already defined as something you can get

Yup.  RFC 2987.

#g

-------------------
Graham Klyne
<[email-masked]>

---

From: John C Klensin <[email-masked]>
Date: Wed, 03 Jul 2002 20:32:04 -0400

--On Tuesday, 02 July, 2002 21:57 -0700 [email-masked]
wrote:

>> common) case is to just send that and forget about
>> capabilities negotiation.  Or, if the number of useful
>> differentiated formats is _very_ small, just sending both
>> with multipart/alternative might turn out to be easier and
>> cheaper than messing around trying to find feature
>> intersections.  There isn't nearly enough information in the
>> spec for me to be able to guess if that would be the case.
>> There shouldn't be if this is a general purpose,
> I guess I remain to be convinced either way: That multiple
> versions schemes are necessary or that a single version
> schemes are sufficient. I'd guess there's a sweet spot
> somewhere in there. Unfortunately in this particular case the

I agree completely.

>> Ignoring all of the relay issues, which I still don't think
>> the document addresses,
> 

I don't either.  My concern is that it is fairly poorly defined
in relay scenarios -- partially, I assume, because its utility
is, at best, unclear -- but that the document says it applies to
relays.  That leaves us in what may be an "almost nothing but
edge cases" situation, which is typically a bad way to be doing
standards.   If we can't see enough utility for relay situations
to justify working out the cases --and I can't-- then why not
ban the relay cases until and unless someone demonstrates that
they are relevant is does the work.  

>> differently.  I'd guess the dominant approach (especially if
>> one generalizes beyond fax) is going to be greatest (most
>> capable) common denominator above a threshold.  I.e., if
>> there is some minimal, default, capability, and some
>> recipients can support only that (e.g., they don't handle
>> option negotiation at all), then two messages are going to be
>> sent: one corresponding to the common feature set of the more
>> capable recipients and one for the minimal/default feature
>> set.  Everyone gets downgraded to the minimum/default because
>> one recipient isn't highly capable just doesn't feel
>> commercially viable to me, while assuming a mix of
>> high-function and low-function hosts does.
> 
> I certainly agree with the common denominator above a
> threshhold. But given that a single recipient per destination
> is likely to be a very common case, I'm less sure that simpler
> approaches won't be commercially viable. Or perhaps
> capabilities will be used to reduce the number of variants
> sent with multipart/alternative.

Makes sense.  Makes much more sense if relaying is banned.
Otherwise, "one recipient per destination" slides over really
easily into "send everything to her, and let her deal with
splitting the stuff up by destinations and capabilities", and
that introduces all the complexity.

That still leaves "one virtual fax, one recipient" (and, by
definition, one destination) as a probably-very common case,
given real-world fax behavior today.  Also given that behavior,
that case is certainly commercially viable.   But this is
awfully complex for that case: we could generalize it into a
single command or capabilities statement.

> It may indeed turn out that per-recipient message copy is the
> right way to finesse the feature set combination issues.

Of course, that is another generalization of "one virtual fax,
one recipient": one message, one recipient, one set of
capabilities.  And a vastly simplified protocol.

>> codes coming back from the server in response to a RCPT TO.
>> If the "one digit" rule is observed, there is no way to
>> differentiate between "bad mailbox" and "bad parameter".  If
>> it is not, and three digits are processed, the server cannot
>> effectively differentiate between a bad parameter to CONNEG
>> and one to some other option by the code -- the reply string
>> has to be parsed, and that leads to bad trouble.   So I would
>> think that a batch of capability-determining commands before
>> the first RCPT one would be a better idea, complexity
>> notwithstanding.
> 

Suppose the client sends
     RCPT TO:<[email-masked]> CONNEG=REQUIRED;bazoption=REQUIRED
and gets back
     504 option parameter stinks - REQUIRED
or
     504 bad parameter - CONNEG REQUIRED
in the one-digit case, it can't distinguish either one from
     500 Never heard of foo
in the three-digit case, the first 504 variant doesn't give a
clue as to the difference between "don't know the client's
capabilities" and "REQUIRED is not a valid parameter for
bazoption".  The second variant is better, but requires parsing
of a string whose format is not precisely specified.   To some
degree, this is a general problem with multiple ESMTP
option-parameter sets, but it seems to be that the CONNEG case
is a little worse than average.

>> is to change the problem.  For example, if the server makes
>> the determination of the common capability set among all
>> recipients and returns it only once (a bit tricky with the
>> SMTP model, but not impossible), then the client doesn't have
>> to figure out how to intersect whatever comes back, break
>> things up into separate messages, etc.  But I don't know if
>> that would do the job; it certainly doesn't handle a mix of
>> high- and low-function recipients in a plausible way.
> 
> I considered proposing this as well, but it seemed to be that
> knowledge of what the client can actually do may determine how
> differing feature sets are combined.

Yes, but what a nightmare.   To really get it right may require
fairly complete knowledge of what the client can actually do
_and_ fairly complete knowledge of what the recipient (not just
the receiving server) can actually do.  In a better world, that
requires an MUA-MUA negotiation, but we don't have any semantics
for talking about that (especially in SMTP).  Of course, forcing
the problem into 
   one receiving MUA
   no relay
would permit getting close without giving all of us a bad
headache.  But I don't know if that is feasible.  What I'm
getting fairly sure of is that the present document is rather
seriously underspecified for the mind-boggling general case.

    john

---

From: John C Klensin <[email-masked]>
Date: Wed, 03 Jul 2002 20:52:47 -0400

--On Tuesday, 02 July, 2002 23:09 -0700 [email-masked]
wrote:

> Well, it is certainly possible to arrange the additional
> conneg data so it is identifiable as such. As for not
> affecting the response, I'm not sure I see the problem: If the
> client wants an interop guarantee, it requires a conneg
> response, and handles 4yz and 5yz codes as temporary or
> permanent failures without regard to what's causing the error.
> If, on the other hand, it wants conneg info as long as its
> available, it makes a conneg response optional, in which case
> the status codes no longer have anyhing to do with conneg
> information.

Ned, purely as a strawman, suppose we turned things around as
follows:

C: EHLO ...
S: 250-Yep we speak that here
S: 250-...
S: 250-CONNEG
C: MAIL FROM:<[email-masked]> CONNEG
S: 250 OK
C: RCPT TO:<recipient1@target1>
S: 250-OK I actually know her capabilities
S: 250-<...capability stuff>
S: 250-<more capability stuff>
C: RCPT TO:<recipient2@target2>
S: 250 OK but that is a relay site and I dont have a clue

Client now figures out that it has one recipient with known
capabilities and one with unknown ones.  Based on that, it
either  sends RSET and tries something else, or sends DATA and
the default [fax?] format.  Considerably cleaner, IMO, than
"REQUIRED" on some, "OPTIONAL" on others, etc.

This does, however, point out another problem with the general
approach, regardless of whether CONNEG appears with MAIL, RCPT,
or VRFY.  Let's assume the latter, since it doesn't get involved
with other semantics, but the other cases are pretty much the
same.

C: EHLO ...
S: 250-OK
S: 250- ...
S: 250- CONNEG
S: 250 SILLYSTATE
C: VRFY <[email-masked]> CONNEG=REQUIRED SILLYSTATE=FALSE
S: [email-masked] ok. probably.
S: 250-CONNEG <some capabilities stuff>
S: 250-<some silly state stuff>
S: 250 CONNEG <more capabilities stuff>

Looks like an interesting parsing/accumulation problem in the
best of cases.   Now, of course, this wouldn't occur with the
document as written, because it appears to prohibit the use of
CONNEG with any other options that might impact the RCPT reply.
Another case not analysed and documented?

     john

---

From: Dave Crocker <[email-masked]>
Date: Wed, 03 Jul 2002 20:10:20 -0700

At 04:16 PM 7/3/2002 -0400, Robert A. Rosenberg wrote:
>the message

If the sender already know the capabilities of the recipient, then the 
sender does not need to use ESMTP/CONNEG.  ESMTP/CONNEG is designed for use 
when the sender does NOT have that information.

There are a number of ways the sender might obtain capabilities before 
starting this SMTP session.

One example is 
<http://www.imc.org/draft-ietf-fax-content-negotiation>.  Another is, no 
doubt, via some LDAP-based directory.

>You need to decide what the
>trade-offs are

By "you" I assume you mean the aggregation of the implementer and the 
sending system's administrator.  There is no need for this specification to 
dictate that choice.

d/

----------

---

From: Dave Crocker <[email-masked]>
Date: Wed, 03 Jul 2002 20:25:21 -0700

At 08:32 PM 7/3/2002 -0400, John C Klensin wrote:
>in relay scenarios

What needs to be defined that is not defined?

This is a hop-by-hop protocol, just like all of SMTP.  If there is relaying 
with CONNEG support, then CONNEG is used at each hop.  Just like for DSN.

Do I think that this is the epitome of efficiency?  No.  But then, 
hop-by-hop mechanisms often suffer with respect to some aspects of 
efficiency.  The reason for hop-by-hop is not "efficiency" but rather for 
reliability/safety.  And the current specification is only one way to 
obtain CONNEG information.  If a better one is available and satisfactory, 
then by all means use it.

>  -- partially, I assume, because its utility is, at best, unclear

right.  it is not at all obvious that an organization might want to have a 
corporate email firewall accept the message with a CONNEG exchange and then 
relay it through the corporate network to the mailbox.  after all, such a 
scenario for regular email is not all that popular.

>   If we can't see enough utility for relay situations

see above.

and it strikes me that the idea of having an SMTP option that is prohibited 
from use in relaying is rather more massively significant than whether this 
particular option will work well in relaying circumstances.

>given real-world fax behavior today.

One might also want to consider that the concept of obtaining per-recipient 
capabilities rather naturally suggests a tendency to do per-recipient 
tailoring of content.

One does not need any history of fax to note the likelihood of the 
one-to-one case being dominant.  One might even suspect that it is inherent.

>   Also given that behavior,

Complicated?

I guess it is not clear what would be simpler and equally useful (as well 
as equally flexible.)

So please, provide an example of this alternative to the current 
specification that is sufficiently simpler so as to render the current 
specification inappropriate.

It also would be good to make a distinction between working well in the 
most likely scenario, versus preventing any scenario except the most likely.

d/

----------

---

From: [email-masked]
Date: Wed, 03 Jul 2002 17:18:54 -0700 (PDT)

> how so?

Because you said: "other conditions resulting from valid conneg strings that
can cause delivery failure." If the conneg result is something you do not like
I don't think treating this as a temporary failure in hopes the conneg string
will change in the future is either reasonable or appropriate.

> if there is a temporary failure to get conneg information, the

That wasn't the case you were talking about.

> can't tell the difference between those conditions; the client can.

Ah, I see. We've moved from assertions that there were sure to be problems to
the nonspecific "poor separation of function". But the entire point of REQUIRED
is to say "I don't want these functions separated; conneg failures are just
like any other failure to me". So this entire line of argument turns out to be
specious.

> well, if we had such other features then the client wanting to assert such
> other features would not be able to assert CONNEG=REQUIRED, it would just
> have to implement the error detection by itself.

Doesn't follow. If some other functionality is implemented that results in RCPT
TO failing in some new way it can return errors in the same way. Of course the
error report is limited to a single error condition, but the potential for
multiple failures having to be collapsed into a single return value already
exists in abundance; the existance of this extension or an arbitrary number of
future extensions changes nothing in this regard.

> implementation errors and the number of mis-reported delivery failures.

Well, I've worked out what code is necessary to do this, and IMO this just
doesn't rise to the level of complexity for this to be the case.

> I think that the fewer warts we add to SMTP the better off we'll be.

Keith, I was willing to give you the benefit of the doubt before. But
now that your concerns about this have become clear I'm finding I flat
out disagree with your assessment of the risks and concerns involved.

				Ned

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Wed, 3 Jul 2002 23:59:25 -0400

At 23:43 -0700 on 07/02/2002, [email-masked] wrote about Re: 

>silly states very silly indeed.

The RSET can be made optional by ONLY sending it if the replies to 
the RCPT TO requests return a result that requires multiple copies of 
the message.

IOW: If the LCD of all the replies is the BEST version you are 
capable of sending (such as getting back some "I support color" 
replies mixed with "B+W-only support" replies when you only have a 
B+W Fax), then just bypass the RSET and retransmission of the RCPT 
TOs and go direct to DATA (and collect your $200).

---

From: [email-masked]
Date: Wed, 03 Jul 2002 20:56:21 -0700 (PDT)

>      504 bad parameter - CONNEG REQUIRED

If the server after having offering the CONNEG and bazoption extensions
subsequent rejects them as "bad", it is broken. Simple as that. And trying to
accomodate arbitrary broken server behavior is an exercise in futility.

I suspect what you meant is something more along the lines of

   504 required information not available for this recipient

or 

   504 required CONNEG information not available for this recipient

> is a little worse than average.

But the entire point of REQUIRED is that client is stating categorically that
no such distinction needs to be made. The client is saying "CONNEG and
bazoption are vital to me, if either of them fails I'm giving up on this
recipient and you should too -- give me an error and I'll report that back to
the originator". A client that wants to proceed in the case where no CONNEG
information, bazoption information, or both are available it has no business
using REQUIRED.

> seriously underspecified for the mind-boggling general case.

I'm afraid I'm coming to a very different conclusion. I think these discussions
have conflated a bunch of different things: (1) Specification bugs that
definitely need to be fixed, (2) A single protocol change that definitely
should be made -- that of making conneg information and status identifiable as
such in the command response, (3) A bunch of concerns about protocol details
that aren't standing up to close inspection and which absent a clear consensus
for change can remain as-is, and (4) A general concern about the many possible
uses for this mechanism that has all of us engaging in futile speculation and
which cannot possibly be dealt sensibly with in a specification without actual
implementation and use experience.

Of course (1) and (2) need to be fixed before proceeding. As for (3) and (4),
well, this is why we have a standards process and not a rubber stamp.

				Ned

---

From: [email-masked]
Date: Thu, 04 Jul 2002 00:50:44 -0400

> one-to-one case being dominant.  One might even suspect that it is inherent.

It's dominant/inherent because nobody's figured out how to make a fax machine
talk to a conference call.

On the other hand, remember that the *SENDER* here is going to be in the
e-mail world - and people are *used* to putting half the known world into
the cc: list (heck, I had to trim *this* message's list by at least 3 or
4 addresses).

I'll give it 15 minutes, tops, before the first user tries to send to more
than 5 recipients at once.  Under an hour before the first time that more
than one recipient is at the same destination....
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked]
Date: Wed, 03 Jul 2002 21:35:01 -0700 (PDT)

> "REQUIRED" on some, "OPTIONAL" on others, etc.

Well, it certainly makes sense if you're eliminating REQUIRED to move CONNEG to
the MAIL FROM.

But as for eliminating REQUIRED, I can only reiterate what I've said before: I
was willing to be convinced there was a problem, but the discussion has done
exactly the opposite.

The only clearly valid argument I've seen so far for removing REQUIRED is that
it may not really be needed and things that aren't needed should not be there.
But whether or not it is needed is really a question for the WG. Perhaps those
who actually plan to implement CONNEG clients could comment on whether or not
they view REQUIRED as something they need.

> Another case not analysed and documented?

I think what you're driving at is that the CONNEG material needs to appear as a
unit and needs to be identifiable as such in the response. If so, this concern
was raised previously and I have previously stated that I think it needs to be
addressed. It is a simple fix, however.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Wed, 03 Jul 2002 23:18:52 -0700

At 01:26 AM 7/4/2002 -0400, John C Klensin wrote:
>That is a reasonable position.  But I suggest that it implies
>either that this material should go to Experimental, rather than
>Proposed, or that the mechanism should be defined as applicable
>to the cases that are understood and expanded only when the
>implications of that expansion are better understood.  Or...

As nearly as I can tell, your position is that folks can raise a spectre of 
myriad, unspecified, abstract and unlikely concerns and then use that as a 
claim that a specification should not go to Proposed...  Until this long 
list of negatives has been disproved.

That's very creative.  It also is at considerable odds with typical IETF 
criteria for advancement, John.  It is also a heck of a good way to make 
sure that no proposal ever goes to Proposed.

So, please explain what prompts such unique and extraordinary criteria for 
this simple and mundane proposal?

d/

----------

---

From: Dave Crocker <[email-masked]>
Date: Wed, 03 Jul 2002 23:28:03 -0700

>talk to a conference call.

And the Good Humour Ice Cream truck does a very poor job of serving hot dogs.

Oops.  That's not it's job.

And, by the way, it is inherent because a feature that is designed to 
obtain per-recipient information is likely to be implemented in a way the 
delivers per-recipient tailoring.

>the cc: list (heck, I had to trim *this* message's list by at least 3 or

The issue is not what the email originator does.  The issue is what the 
sending MTA does.

In other words, how it maps the recipient list, generated by the 
originator, into SMTP commands.

d/

----------

---

From: [email-masked]
Date: Wed, 03 Jul 2002 22:59:44 -0700 (PDT)

> > (4) A general concern about the many possible
> > uses for this mechanism that has all of us engaging in futile
> > speculation and which cannot possibly be dealt sensibly with
> > in a specification without actual implementation and use
> > experience.

> implications of that expansion are better understood.  Or...

Experimental status also occurred to me as a possible approach.

> > The only clearly valid argument I've seen so far for removing
> > REQUIRED is that it may not really be needed and things that
> > aren't needed should not be there. But whether or not it is
> > needed is really a question for the WG. Perhaps those who
> > actually plan to implement CONNEG clients could comment on
> > whether or not they view REQUIRED as something they need.

> I, at least, would find such comments helpful.  But I'd also
> like to hear comments from at least one person responsible for a
> general-purpose MTA (as distinct from a fax-over-email engine)
> who expects to implement this feature and how they propose to do
> so and how this feature fits in.

There are two cases here: Client and server implementations.

I definitely plan to implement the server side of this proposal. From that
perspective all of the proposed variants are easy to implement as long as you
have a source of conneg information to tap. I plan to use an LDAP attribute for
this purpose.

The client side for a general-purpose MTA is much harder, but not because of
any aspect of the protocol. The issue I have there is how to match up an
arbitrary set of site-provided conversion operations with the current group of
recipient feature sets. I don't see a good solution to this other than making
the whole thing something the site specifies, and if I do that I see little
chance of the capability actually being used by enough sites to justify the
cost of implementation.

One final comment. A general-purpose MTA using arbitrary site-provided
conversion routines doesn't have the luxury of being able to bound the
performance or other characteristics of those conversion routines. This makes
the entire convert-on-the-fly scenario problematic in ways that don't arise for
a device built around a specific, limited and well understood set of on-the-fly
converters, regardless of whether that device is a fax machine or something
else entirely. Indeed, if I were to implement based on past bad experiences
with various general-purpose conversion routines I'd be likely to break the
problem down into three steps: (1) Collection of conneg information online, (2)
Message splitting and conversion offline, (3) Sending the resulting emssages to
their destination. I doubt I'd support using REQUIRED in (1).

				Ned

---

From: [email-masked]
Date: Thu, 04 Jul 2002 02:35:40 -0400

> Proposed, or that the mechanism should be defined as applicable

Would this address the concerns of everybody?
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked]
Date: Thu, 04 Jul 2002 12:30:26 -0400

> delivers per-recipient tailoring.

When all you have is a hammer, everything looks like a nail...

> originator, into SMTP commands.

And there is currently disagreement whether to keep using nails, or if
bolts would be a better fastening device.  My point was that the original
"send to one recipient" mindset won't last very long, so whatever mapping
we end up using had better not be *too* horrendously inefficient. People
*will* end up sending to multiple recipients at a destination (unless we
insist that "destination" will be a "personal device" (like a PDA etc)
rather than a "mail server".
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: Keith Moore <[email-masked]>
Date: Thu, 04 Jul 2002 14:47:49 -0400

> will change in the future is either reasonable or appropriate.

When you said "such failures" I wasn't sure which failures you were talking about - 
the ones that are caused by server inability to supply a conneg string or those 
caused by conneg strings that indicate that no conversion is possible.  (since
the quoted paragraph mentioned both kinds of failure) 

> That wasn't the case you were talking about.

That response was based on a guess as to what you were talking about. 
I guessed wrong.

> the nonspecific "poor separation of function". 

Admittedly "poor separation of function" is a nonspecific description of the problem,
but a more specific description followed which described what I meant.

Asking the server to report reply codes and (more importantly) enhanced status 
codes that don't accurately describe the condition that caused the failure,
(and cannot, because the server lacks the information necessary to distinguish 
those conditions) doesn't seem like good protocol design to me.  The words 
"poor separation of function" seem like an accurate description of this kind of
design flaw, though I wouldn't expect someone to understand my criticism of the 
design (or accept the argument for changing it) from those four words alone.

> But the entire point of REQUIRED
> like any other failure to me". 

Admitted that this is the entire point of REQUIRED; the question is whether
this is good design or an appropriate feature to include.  Using REQUIRED
doesn't simplify the design of the client by any significant amount (if 
at all), and by using REQUIRED a client somewhat impairs its ability 
to produce meaningful error reports.  

In short: REQUIRED is not necessary because clients can detect a missing
CONNEG string by other means, it doesn't seem to be useful because it
doesn't simplify either the client or the server nor make them more
efficient, it encourages client implementors to produce less meaningful 
error reports, and even though the implementation overhead is low, it 
does add additional baggage to the protocol.

The real question is, why do we want this?

> future extensions changes nothing in this regard.

I agree that we already have the potential for RCPT to report several kinds of 
errors, though I do think adding new conditions is likely to contribute to less 
precise error reporting.  And prior to this proposal all of the errors reported
by the server have been for conditions which were better detected by the server -
this proposal is new in that it asks the server to report an error for a condition
which is better detected by the client.

> doesn't rise to the level of complexity for this to be the case.

The code to implement REQUIRED on either the client or server end is trivial.

On the other hand, the effort required for a new implementor to understand SMTP
and how this extension interacts with all of the other ones is nontrivial.

On numerous occasions we've both seen that even somewhat mature implementations 
frequently misunderstand far less subtle aspects of email protocols than
those being discussed here.   This proposal introduces new opportunities 
for misunderstanding.  Some of them are due to the way that the protocol
extension works, others are due to the description (or lack thereof) of 
how the extension is to be used.  

I believe that every new bit of complexity that we add to SMTP requires
significant justification, whether that complexity is measured in lines of 
code, in lines of text that implementors have to understand, or (especially!)
in undocumented assumptions on the part of the designer that implementors 
have to figure out.  And every bit of unnecessary complexity we can remove 
from a proposed extension is worthwhile.

We might disagree on the relative merits of a particular feature, and
some of that is to be expected.  But when we get into extended
arguments about a single aspect of the design we should not lose  
sight of the overall picture:  We need for email to work reliably.

In various ways this proposal adds unnecessary complexity, and lacks
the degree of background explanation that is necessary to allow 
implementors to understand how to use it, especially for non-fax cases.
  
If it seems like I'm attacking the proposal harshly, it's only because 
I'm painfully aware of how often SMTP is misimplemented, and because 
I think that these discussions should have been had in the broader
SMTP implementor community long before IETF-wide Last Call.

Keith

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Jul 2002 15:25:39 -0400

--On Wednesday, 03 July, 2002 23:18 -0700 Dave Crocker
<[email-masked]> wrote:

>> either that this material should go to Experimental, rather
>> than Proposed, or that the mechanism should be defined as
>> applicable to the cases that are understood and expanded only
>> when the implications of that expansion are better
>> understood.  Or...
> 
> As nearly as I can tell, your position is that folks can raise
> a spectre of myriad, unspecified, abstract and unlikely
> concerns and then use that as a claim that a specification
> should not go to Proposed...  Until this long list of
> negatives has been disproved.
> 
> That's very creative.  It also is at considerable odds with
> typical IETF criteria for advancement, John.  It is also a
> heck of a good way to make sure that no proposal ever goes to
> Proposed.
> 
> So, please explain what prompts such unique and extraordinary
> criteria for this simple and mundane proposal?

Dave,

I think this has ceased to be constructive, so I'm going to drop
out before the serious name-calling starts.  I would suggest,
however, that you consider three things:

(i) While there are clearly some differences in the two cases,
the notion of applying an option very broadly, more broadly than
the implications are understood, and then underspecifying it a
bit, is exactly one of the things that both of us are objecting
to in the IDN situation.  Now, we disagree on whether or not the
relaying cases (in particular) are poorly understood.   As I
understand your position, it is that we relay things all of the
time, so this is no different and relaying is well-understood.
Mine is that we have very few cases in which options, when
applied to relaying, more or less assume end-to-end knowledge
rather than hop-by-hop processing.  And when we do, I suggest
the precedent is that we carefully work out exactly what is
supposed to happen, with 8BITMIME being a reasonably good
example.

(ii) Under any theory I can construct of what is going on here,
the capabilities inquiry is expected to yield, in the normal
case, a statement about the capabilities of the destination user
agent.  Capabilities of the transport subsystem are not at
issue: unlike, e.g., 8BITMIME or various DSN options, these
capabilities have to do with message format and content, not
transport properties.  Asking an MTA about the capabilities and
properties of an MUA to which it will channel mail (possibly
through some delivery intermediate) is something that, as far as
I know, we have never done before.  In addition, it is arguably
a non-trivial layering violation.  Whatever those
characteristics imply about this proposal -- as I trust you
know, I will not resort to arguments about architectural purity
when something legitimate needs to be accomplished and the only
plausible way of doing it is architecturally-impure -- I think
they don't permit it to be characterized as "simple and mundane".

(iii) My understanding of IETF precedents is that we are
typically very careful when changes are proposed to
widely-deployed and critical infrastructure and applications.
We commonly set the bar higher for such things -- publishing as
Experimental, requiring implementation experience and/ or much
more case analysis for Proposed, etc. -- than we do for now
protocols that can't have any spillover impact.  Reasonable
people can disagree as to whether a new ESTMP option carries
with it sufficient interaction risk that it should get this
closer scrutiny.  But I note that this review process has
already identified (and agreed to fix) on place where CONNEG was
underspecified relative to other SMTP options.   And, of course,
RFC 1425, in laying the groundwork for these extensions, says:

	It must be emphasized that any extension to the SMTP
	service should not be considered lightly. SMTP's
	strength comes primarily from its simplicity.
	Experience with many protocols has shown that:

       protocols with few options tend towards ubiquity, whilst
       protocols with many options tend towards obscurity.

	This means that each and every extension, regardless of
	its benefits, must be carefully scrutinized with respect
	to its implementation, deployment, and interoperability
	costs. In many cases, the cost of extending the SMTP
	service will likely outweigh the benefit.

You will recall that I was not the author of that particular
text, and did not think it was necessary.  Clearly I was wrong.
And I am suggesting that this proposal be scrutinized in that
light and that, in that light, _no_ SMTP extension should be
added because it is "simple and mundane".

     john

---

From: John C Klensin <[email-masked]>
Date: Thu, 04 Jul 2002 15:36:07 -0400

--On Wednesday, 03 July, 2002 22:59 -0700 [email-masked]
wrote:

>> like to hear comments from at least one person responsible
>> for a general-purpose MTA (as distinct from a fax-over-email
>> engine) who expects to implement this feature and how they
>> propose to do so and how this feature fits in.
> 
> I definitely plan to implement the server side of this
> proposal. From that perspective all of the proposed variants
> are easy to implement as long as you have a source of conneg
> information to tap. I plan to use an LDAP attribute for this
> purpose.
>...

Interesting.  Instead of "ask the receiving client", you would
rely on a database that proported to know the client's behavior
and capabilities.  That certainly makes implementation easier
and might also deal with the relay situation by permitting the
first (and subsequent?) relays to ask the database rather than
replying "no clue about the client, I don't deliver to it" (the
latter is one of the cases I've been concerned about).  However:

(i) It would seem to me to be reasonable to write this option
down (as an option) rather than leaving implementers guessing.

(ii) Given the notorious "wks" experience, with trying to put
target host capabilities into an ancillary database, it seems to
me that this would further justify experimental status so that
we can get some sense as to whether the database would be kept
adequately up-to-date.  Certainly our experience should cause us
to be skeptical about that.

> The client side for a general-purpose MTA is much harder, but
> not because of any aspect of the protocol. The issue I have
> there is how to match up an arbitrary set of site-provided
> conversion operations with the current group of recipient
> feature sets. I don't see a good solution to this other than
> making the whole thing something the site specifies, and if I
> do that I see little chance of the capability actually being
> used by enough sites to justify the cost of implementation.
> 
> One final comment. A general-purpose MTA using arbitrary
> site-provided conversion routines doesn't have the luxury of
> being able to bound the performance or other characteristics
> of those conversion routines. This makes the entire
> convert-on-the-fly scenario problematic in ways that don't
> arise for a device built around a specific, limited and well
> understood set of on-the-fly converters, regardless of whether
> that device is a fax machine or something else entirely.
> Indeed, if I were to implement based on past bad experiences
> with various general-purpose conversion routines I'd be likely
> to break the problem down into three steps: (1) Collection of
> conneg information online, (2) Message splitting and
> conversion offline, (3) Sending the resulting emssages to

I can only agree with both of these comments and complement you
on the clarity of your explanation of them.

     john

---

From: Dave Crocker <[email-masked]>
Date: Thu, 04 Jul 2002 13:37:56 -0700

At 03:25 PM 7/4/2002 -0400, John C Klensin wrote:
>to in the IDN situation.

Well, no.  Relative to this discussion, my concern with IDN is that the 
technical specification has a gaping hole -- it is missing an essential 
piece of specification detail -- and that that hole pertains specifically 
to the most important immediate use.  So IDN is not about scope.

By contrast -- ignoring the few postings from folk who have diligently 
attended to writing details that need to be fixed -- the focus of concerns 
from you and Keith has exclusively been about scope that might be or 
misinterpretations that might me.  Notably missing, in fact, has been 
discussion of technical details, except for some forays into alternative 
approaches or alternative problem spaces.

>As I

Lest one miss the focus of my comment on this, let me repeat it:  "If there 
is relaying with CONNEG support, then CONNEG is used at each hop. Just like 
for DSN."

In other words, we are not breaking any architectural ground here, contrary 
to the claim that has been made.

>supposed to happen, with 8BITMIME being a reasonably good

CONNEG already covers your concern.  If the option is required, then the 
receiving MTA must commit to supporting it.  If it cannot make the 
commitment, then the address must be failed. If the option is optional, 
then the sender leaves handling outcome to the receiver.

If you have any other relaying concerns, please state them with enough 
detail to permit technical analysis and response.

Otherwise, yours and Keith's concerns reduce to a statement that you have 
generic fears, without any detailed basis, and that your fears should cause 
the community to slow-roll this specification.

>Asking an MTA about the capabilities and
>I know, we have never done before.

Isn't it wonderful that we sometimes do something new in the IETF?

1.  We have already done hop-by-hop options that have end-to-end effect.

2.  We have already done CONNEG syntax and semantics.

3.  We have already done email-based CONNEG exchanges, though the mysteries 
of the IETF standardization process are causing that work of more than one 
year ago to finally get approved by the IESG now.

4.  We have already done third-party information disclosure 
mechanisms.  That is, how is this conceptually different from an LDAP 
lookup?  For that matter, how is is different from TCP MSS 
"disclosure"?  It, too, causes the other participant to change what is sent.

So,

A.  Yes, we are defining a way to do something that you cannot do today,

B.  But the components to that mechanism all have prior existence,

C.  And the mechanism involved is pretty simple,

D.  And no one has yet put forward a concrete scenario that demonstrates 
that the specification is flawed.

>widely-deployed and critical infrastructure and applications.

And we are notably more lax when the change is optional and incremental 
and, therefore, need not be implemented except by those wishing to play in 
that sandbox.

>   But I note that this review process has
>underspecified relative to other SMTP options.

If you mean something other than have a "context" label in the portion of 
the response pertaining to the context, then it was not in Ned's summary.

If you believe that identification of the need for that label is somehow 
strategically meaningful, then it sounds as if you expect a degree of 
perfection for a Proposed Standard that is at variance with IETF 
norms.  For that matter, it suggests that you believe that a Last Call that 
uncovers the need for any changes automatically forces the specification to 
go back to square one or be classed as experimental.

>text, and did not think it was necessary.  Clearly I was wrong.

The nice thing about statements of philosophy is that they provide a 
guiding framework.  The not-so-nice thing about them is that they do not 
provide specific detail.  Invoke philosophy in decision making, without 
having any attendant detail, and all you are doing is being religious.

When you produce some technical details that demonstrate how this proposal 
fails -- or better still, how it damages SMTP operation -- then invoking 
that higher bit of philosophy will be warranted.

d/

----------

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Fri, 5 Jul 2002 12:57:17 -0400

At 22:59 -0700 on 07/03/2002, [email-masked] wrote about Re: 

>their destination. I doubt I'd support using REQUIRED in (1).

As I said in another reply, one POSSIBLE solution to this problem is 
to go for a methodology where the CLIENT does the transforms and 
ships one copy of the message as Multi-Part/Alternate with ALL the 
transforms to the MTA. The MTA then decides what Transforms are 
needed (based on the CONNEG info) and grabs out those that are needed 
from the M-P/A set. This removes the Real-Time transform overhead by 
off-loading it to the Client. It is, of course, based on a model 
where the number of different transforms are limited AND can be 
enumerated in the absence of info on the capabilities of the 
designated recipients.

---

From: [email-masked]
Date: Fri, 05 Jul 2002 23:01:31 -0700 (PDT)

> down (as an option) rather than leaving implementers guessing.

Writing it down would be fine with me, although to be honest this struck me as
the obvious way to do it. OTOH, there certainly have been cases in the past
when I refrained from documenting what I thought was obvious in one of my own
drafts, only to find it was nothing of the sort.

> to be skeptical about that.

I think the differences between this and wks far outweigh the similarities.
Specifically, the major problem with wks records was that the authority
over/responsibility for wks records and authority over/responsibility for the
aspects of systems wks describes tended to be disjoint. Back in the days when I
maintained a reasonably heftly DNS zone I never had enough time to keep wks
records up to date with all the changes the various sysadmins kept making, and
after a while I stopped trying. And of course the sysadmins didn't have the
means to maintain this information in the DNS themselves.

The situation with an LDAP directory used for messaging is quite different.
Such directories often already maintain a bunch of user-settable information
(e.g., filtering rules, personal address book) and typically provide the means
(e.g., a web interface) for users to maintain and modify that information
themselves.

Of course having such a capability doesn't mean users will actually take the
time to use it. I suspect that's going to be very dependent on both need and
the quality of the user interface; presenting raw media feature sets definitely
isn't the way to go ;-)

> on the clarity of your explanation of them.

Thanks!

I had a couple of other thoughts about this over the past couple of days:

(0) There are/will be additional ways to obtain conneg information: In
    DSNs or MDNs, through LDAP queries, via RESCAP, or through direct
    database queries within a single mail system. The database/directory
    mechanism can be used to advantage in all of these cases.

(1) Caching of conneg information is certainly possible and may even
    be desireable. Indeed, conneg information returned in DSNs or MDNs will
    have to be cached in order to be useful. This document doesn't discuss
    caching, and while caching isn't required in the ESMTP context some
    discussion of it nevertheless makes sense.

(2) The use of caches really opens the door to conneg in the context
    of relays in ways that nothing else does. For example, imagine a relay
    that doesn't do any content transformation of its own but does do forward
    conneg requests to populate a cache. It then offers the information it
    has in that cache when it acts as a server. This may sound kind of cool
    at first glance, but as the cache comes and goes it would tend to
    violate the least astonishment principle in fairly major ways. 
    Issues also arise if multiple relays are used, each with a separate
    cache. Care is needed here to preserve the ability to "rack and stack"
    mail relays without undue consequences.

    This isn't a typical cache since the goal isn't to minimize the number
    of forward queries that are done. The intent here is to provide the
    information from those queries in a completely different context.

    The variability problems here can be solved by going to either
    extreme: Ban caches of this sort entirely or insist that components
    that actually perform content transformations cache conneg information
    for a fairly long period. (It should be obvious that having intermediates
    cache information is risky and doesn't solve the problem.) At first I
    thought the right answer was to ban this stuff, but on reflection I think
    the use of DSNs and MDNs to return conneg information argue for a
    mandatory cache in the transformation agent.

				Ned

---

From: [email-masked]
Date: Sat, 06 Jul 2002 15:22:20 -0700 (PDT)

Keith, your long rant about protocol design philosophy contained, as far as I
can tell, no additional technical points about the document under discussion.
While continuing such a debate might be interesting on some level, I'm afraid I
have neither the time nor the interest in doing so now.
  
> SMTP implementor community long before IETF-wide Last Call.

I'm sorry, but I know of no IETF process requirement that working groups engage
in such discussions, nor for that matter do I know of a means a WG has to
insure that such discussions take place. (Both of your recent cross-postings to
ietf-822 and ietf-smtp respectively have produced, as far as I can tell, not a
single message from anyone who travels in those circles exclusively.)

You are of course free to follow the discussion on the various relevant WG
lists so you may make your comments on their work in a timely fashion. I
suspect people would be a lot more receptive to the sorts of changes you've
proposed if you adopted this approach. The number of IETF working groups
engaged in the SMTP/MIME/RFC[2]822 area isn't particularly large so this really
shouldn't be particularly onerous burden to assume.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Sun, 07 Jul 2002 10:22:02 -0400

--On Friday, 05 July, 2002 23:01 -0700 [email-masked]
wrote:

> (0) There are/will be additional ways to obtain conneg
> information: In     DSNs or MDNs, through LDAP queries, via
> RESCAP, or through direct     database queries within a single
> mail system. The database/directory     mechanism can be used
> to advantage in all of these cases.
> 
> (1) Caching of conneg information is certainly possible and
> may even     be desireable. Indeed, conneg information
> returned in DSNs or MDNs will     have to be cached in order
> to be useful. This document doesn't discuss     caching, and
> while caching isn't required in the ESMTP context some
> (2) The use of caches really opens the door to conneg in the
> context     of relays in ways that nothing else does. For
> example, imagine a relay     that doesn't do any content
> transformation of its own but does do forward     conneg
> requests to populate a cache. It then offers the information it
>     has in that cache when it acts as a server. This may sound
> kind of cool     at first glance, but as the cache comes and
> goes it would tend to     violate the least astonishment
> principle in fairly major ways.      Issues also arise if
> multiple relays are used, each with a separate     cache. Care
> is needed here to preserve the ability to "rack and stack"
>     This isn't a typical cache since the goal isn't to
> minimize the number     of forward queries that are done. The
> intent here is to provide the     information from those
> queries in a completely different context.
> 
>     The variability problems here can be solved by going to
> either     extreme: Ban caches of this sort entirely or insist
> that components     that actually perform content
> transformations cache conneg information     for a fairly long
> period. (It should be obvious that having intermediates
> cache information is risky and doesn't solve the problem.) At
> first I     thought the right answer was to ban this stuff,
> but on reflection I think     the use of DSNs and MDNs to
> return conneg information argue for a     mandatory cache in
> the transformation agent.

Ned,

Again, the more of these sorts of things get written down, even
as experimental alternatives for further explanation, the
happier I would get about the whole picture.  Beyond a fairly
short list of specific problems (which I think are being
addressed), my most significant concern about this proposal is
that, as it stands, it appears to be a piece of isolated
protocol work that gets injected into an [E]SMTP environment
that we know to be a little bit fragile.  Anything that can be
done to limit that fragility or its impact -- restrictions to
fax, retrictions of the relay case(s), explanations of plausible
ways to apply the protocol in a safe way, etc. -- would, it
seems to me, help a great deal.  

Which of those are chosen seems to me to be less important.  But
the current document, without any of it, seems to be an
invitation to low-quality implementations that conform to
creative readings of a syntax and somewhat underspecfied
semantics and thereby get the mail environment into trouble.

   john

---

From: Keith Moore <[email-masked]>
Date: Sun, 07 Jul 2002 17:38:50 -0400

Ned, 

I do not share your opinion that someone who is interested in a particular technology has an obligation to follow every WG that might try to alter that technology.  First of all, it's not obvious which groups might do this-for instance who would guess that the IPv6 folks would be working on a document that affects how SMTP works? Second, there is just too much traffic on many mailing lists to keep track of lists that are peripheral to one's interests. 

This is a scaling problem, like many others that we face.

-Keith

---

From: Dave Crocker <[email-masked]>
Date: Sun, 07 Jul 2002 14:52:36 -0700

At 05:38 PM 7/7/2002 -0400, Keith Moore wrote:
>I do not share your opinion that someone who is interested in a particular 
>technology has an obligation to follow every WG that might try to alter 
>that technology.  First of all, it's not obvious which groups might do 
>this-for instance who would guess that

an internet draft filename that contains the string "esmtp" is a pretty 
good clue.

and remember that i-d's are announced, including their abstracts.  the 
abstract for the current specification is pretty straightforward.

d/

ps.  to the extent that you perceive a larger, IETF-wide coordination 
issue, i suspect you will get agreement from many people.  the hard part is 
in proposing a solution that those people will also agree with.

----------

---

From: John Stracke <[email-masked]>
Date: Mon, 08 Jul 2002 10:55:14 -0400

Robert A. Rosenberg wrote:

> The MTA then decides what Transforms are

...and where you're not interested in signing the message.

-- 
/========================================================\
|John Stracke                    |Principal Engineer     |
[email-masked]   |Incentive Systems, Inc.|
|http://www.incentivesystems.com |My opinions are my own.|
|========================================================|
|This is the .sig that says... Ni!                       |
\========================================================/

---

From: Graham Klyne <[email-masked]>
Date: Tue, 09 Jul 2002 20:11:13 +0100

>     mandatory cache in the transformation agent.

It's not relevant to the current draft under discussion, but I'd like to 
flag that the (UA-driven) content negotiation approach (awaiting RFC 
publication) overcomes these caching concerns.  Indeed the design presumes 
caching to achieve 1-round-trip performance, but can correct itself (using 
a second R-T) if the cached information turns out to be incorrect.

#g

-------------------
Graham Klyne
<[email-masked]>