Subject: Keywords for "SMTP Service Extension for Content Negotiation"

From: Hiroshi Tamura <[email-masked]>
Date: Fri, 12 Jul 2002 19:11:42 +0900 (JST)

Folks,

There will be lots of discussion at FAX WG meeting in Yokohama.
For your convenience, I summarize the keywords at very random,
although they may not be covered.

- fax-specific or not
- multi-recipient
- LCD
- new command RCAP??
- RSET
- VRFY
- consideration of relay
- content transforamation
- only CONNEG or CONNEG=REQUIRED/OPTIONAL
- interoprability with existing environment
- how to handle with response/status code
- real-time
- bandwidth
- multipart/alternative
- security
- Experimental or Proposed Standard
- ...
- ...

Toyoda-san and Dave:
Could you please consider some of them at your presentation?

Regards,
--
Hiroshi Tamura, Co-chair of IETF-FAX WG
[email-masked]

---

From: Dave Crocker <[email-masked]>
Date: Sat, 13 Jul 2002 12:04:48 +0900

>although they may not be covered.

Tamura-san, thank you very much for listing these terms.  It is easy to 
forget that these terms are not familiar to everyone.  Also, the discussion 
has sometimes been confusing  I encourage people to list additional terms 
that should be explained.

Here are my own, personal comments and explanations.  I think some are 
simple, but others might be controversial.  Also there might be more than 

>- fax-specific or not

It is strange,  but I think this is the most difficult term.  When we talk 
about T.30, it is easy to know what it means to say "fax-specific".  When 
we talk about FFPIM, perhaps it is not so easy.

For the IETF Fax working group, I believe "fax-specific" refers to features 
and functions that emulate T.30.  That is, we are trying to make Internet 
mail provide the same service as T.30.  So, "fax-specific" refers to email 
enhancements that permit email to work like fax.

>- multi-recipient

Internet mail permits a message to be addressed to more than one 
person.  For example, see the To and CC fields of this message.  T.30 
permits addressing a single receiving device.  Some fax store-and-forward 
services permit the sender to specify more than one receiver.  This is 
multi-recipient.

>- LCD

Least Common Denominator.  So, we are borrowing a term from math, to apply 
to our discussion of ESMTP/CONNEG, it is possible to receive CONNEG 
parameters for multiple recipients.  If the recipients have different 
capabilities, what capabilities should the sender use?  One possibility is 
to end the current SMTP session and then send each recipient the best 
content they can receive.  Another possibility is to find the 
"intersection" of capabilities for all the recipients and send this "least 
common denominator" or "most common format" to all of the recipients at once.

>- new command RCAP??

The current ESMTP/CONNEG specification uses the RCPT command for finding 
each recipient's capabilities.

Moore-san prefers to have the capabilities query be in a separate 
command.  For convenience, he has given this command a name, RCAP.  So, the 
name refers to an idea that Moore-san has.  There is no written 
specification for that command.  In my opinion, the current specification 
is satisfactory.

>- VRFY

These are SMTP commands.  RSET ends the current SMTP session.  The command 
means "reset".  VRFY lets a client SMTP ask a server SMTP to "verify" the 
validity of an address, before using it.  This command is no longer very 
popular, because it provides an easy way for spam senders to test for new 
addresses.

>- consideration of relay

Facsimile is direct, point-to-point.  The sending device and the receiving 
device are connected to each other at the same time.  They talk with each 
other directly.

Internet mail is store-and-forward.  The sender and the receiver are not 
required to be connected to the Internet at the same time.  The email may 
go through intermediate email services, when moving from the sender to the 
receiver.  (For example, it may go from a department's SMTP mail system to 
the enterprise system; the enterprise system may send it to the receiving 
enterprise's system; the receiving system may then send it to the 
receiver's department email server.) This is like packet switching.  Each 
intermediate system is "relaying" the email.

The primary difference between relaying versus direct is that relaying 
might take much longer.  Therefore the sender does not know when it will 
receive a response.

On the Internet, a direct connection usually has a response time of a few 
minutes, maximum.  Normally it is much shorter, of course.  For email 
relaying, response time can be days, not minutes.  This difference in 
latency requires different design choices.

>- content transforamation

Converting content from TIF to JPEG, and JPEG to GIF are examples of 
content transformation.  Converting from UTF-8 to UTF-16 is also an example 
of content transformation.

>- only CONNEG or CONNEG=REQUIRED/OPTIONAL

I do not know what "only CONNEG" means, here.

>- interoprability with existing environment

When we add a new feature, how does it affect the old operation?  Sometimes 
adding a new features makes the old system stop working.  For MIME, the 
goal was to add a new feature that made no change to email transport and 
also would not break an old email user agent.

The goal with ESMTP options is to make old operations behave the old way, 
but permit participants who support a new option to use them with each other.

>- how to handle with response/status code

I believe this refers to a concern that ESMTP/CONNEG uses extended ESMTP 
response/status codes in an usual way.  The concern is not widely held.

>- real-time

I think that we use this term very informally in the Fax working group 
discussions.  Probably it simply means "quickly".  Perhaps we can simply 
use it to mean "within the limits of T.30 timeouts" -- 2 seconds?

For the design of human interaction, real-time usually mean 1/2 second, or 
less.  For the design or high performance system, it can refer to 
milliseconds or microseconds.

>- bandwidth

The amount of data that can be send within a period of time.  The bandwidth 
of a T1 (or E1) line is very much greater than a dial-up connection.

>- multipart/alternative

The MIME standard for Internet mail specifies a means of attaching one or 
more segments (body-parts) of content.

Multipart/mixed is used for specifying multiple attachments that are not 
related to each other.

Multipart/alternate is use to specify multiple attachments that are 
different versions of the same content.  For example, one might be ASCII; 
another might be HTML, and another might be PDF.

When the sender does not know the capabilities of the recipient, the sender 
can use multipart/alternative to send more than one version of the 
content.  The sender is guessing that the receiver can process at least one 
of the forms.

>- security

Gomen nasai.  This is a very big topic, of course.

I think the discussion of security for ESMTP/CONNEG only has the concern 
about violating privacy.   By using this option, the receiving server is 
divulging features of the recipient.  It is possible that such information 
is private, or that distribution of such information must be restricted.

>- Experimental or Proposed Standard

The IETF standards process has 3 classes of categorization:

And Experimental document is NOT on the standards track.  The document is 
published for the benefit of the community and the author is requesting 
feedback, especially comments about experiments that use the specification.

A Best Current Practises categorization refers to documents that describes 
operations and administration procedures.  They do not define protocols, 
they define the USE of protocols and procedures.

The IETF standards track has 3 levels:  Proposed, Draft and (full) 
Standard.  Proposed is the first level.  A document must be complete, for 
the task it defines and must have no known errors.  However the 
specification usually does not have to be implemented and 
tested.  Implementation and testing are required to reach Draft standard.

In the discussion for ESMTP/CONNEG, some participants are concerned that 
the option may cause bad effects on normal SMTP usage.  They believe that 
the specification should be classed as Experimental until it is tested.  My 
own comments about this are that all new protocols carry some risk.  The 
pressure to select Experimental (non-standards) status usually is reserved 
for specifications that are complicated and poorly understand, or for 
specifications that have an inherently high degree of risk.

In my opinion, ESMTP/CONNEG is a simple option with little risk.  I believe 
that we DO need to learn to use it.  It DOES add some complexity and it 
extends use of the ESMTP option feature in some way.  However I believe 
that the risks involve efficiency, not core functionality.  That is, I 
believe that implementors must learn to make the option work well, but that 
the option should be relatively safe.

d/

----------
Dave Crocker  <mailto:[email-masked]>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.850.1850

---

From: John C Klensin <[email-masked]>
Date: Sat, 13 Jul 2002 03:11:13 -0400

--On Saturday, 13 July, 2002 12:04 +0900 Dave Crocker
<[email-masked]> wrote:

>> - consideration of relay
> 
> Facsimile is direct, point-to-point.  The sending device and
> the receiving device are connected to each other at the same
> time.  They talk with each other directly.
> 
> Internet mail is store-and-forward.  The sender and the
> receiver are not required to be connected to the Internet at
> the same time.  The email may go through intermediate email
> services, when moving from the sender to the receiver.  (For
> example, it may go from a department's SMTP mail system to the
> enterprise system; the enterprise system may send it to the
> receiving enterprise's system; the receiving system may then
> send it to the receiver's department email server.) This is
> like packet switching.  Each intermediate system is "relaying"
> the email.

It is like packet switching except, of course, in the packet
case, each packet is typically a fragment or component of a
message.  The message, or data stream, is sent one packet at a
time; it is not completely assembled at each intermediate host
and then forwarded after it is complete.  With email, each relay
host obtains, assembles, and stores the entire message, only then
initiating a new transaction to forward it.  That difference is
significant for a few purposes, insignificant for most, but we
should be careful to not push analogies to packet-switching too
far.

> The primary difference between relaying versus direct is that
> relaying might take much longer.  Therefore the sender does not
> know when it will receive a response.

I would disagree.  The primary difference I see between relaying
and direct is that, in a direct arrangement (characteristic of
e.g., HTTP) the originating sending system can ask the ultimate
receiving one a question by, well, asking.  In the relay case,
such questions can only be asked of an intermediate system, which
must respond based on indirect information (which it finds in a
database, see below) or by acting as proxy for a remote system,
using information it has not obtained yet.   Some illustrations:

Q: Can mail be delivered to this address?

	Direct answer: "yes" or "no"
	
	Relay answer: "I have no idea, but I will accept
	responsibility for the mail and promise to forward it to
	someone else who will accept responsibility.  For each of
	us, we understand that "accepting responsibility" means
	that we will either deliver the mail to the final
	recipient, deliver it to someone else who agrees to
	accept responsibility, or return an error message which
	will, of necessity, come out-of-band relative to the
	original connection".
	
	Note: contrast this with TCP, where the packet-switching
	model and the nature of the protocol permits getting the
	possibly-analogous "yes, the destination host will accept
	traffic on that port" answer back before any data packets
	are sent.

Q: Does the recipient support this data format?

	Direct answer: "Yes" or "How about one of the following?"
	
	Relay answer 1: "The database says you should try..."
	(see below)
	
	Relay answer 2: "I don't have a clue, but I will accept
	the message on the assumption that it does, and pass it
	on, either to that destination or to some other site
	willing to take responsibility.  If it turns out the
	recipient cannot accept the format, I (or one of the
	systems to which responsibility has been passed) will
	(preferentially) convert the format to something that can
	be accepted or (less preferred) bounce it with an
	out-of-band message."

Those seem fairly different to me, much more different than
"slower".  In fact, I think that difference is part of what this
discussion is about: if the only difference were "might take
longer", I don't think we would be having this (part of) the
discussion at all.

     john

A footnote on the "find out from a database" issue.  The more I
think about it, the more I conclude that the database lookup
(LDAP or otherwise) is the only rational way of implementing the
desired functionality here in a case that might involve relaying.
To the extent that speedy and confirmed delivery are often cited
as important values in fax, the "maybe it will get delivered as
requested, maybe it will be downgraded, maybe the format will be
influenced by the format requirements of other recipients, and
maybe the message will be bounced in a form that is hard or
impossible for the originating (or early relay) MTA to detect"
alternatives are unreasonable.  

But, if database lookup is the only plausible implementation that
is consistent with the underlying design requirements, then we
should not be looking at capabilities negotiation in the SMTP
stream.  Instead, we should be looking at a piece of protocol by
which the sending system (probably the MUA, rather than the MTA)
can figure out the capabilities of the recipient(s) and then
prepare the message, make address grouping decisions, etc.,
cleanly.   It seems to me that would be more clear, would put the
responsibility back on the MUA or local system which has maximum
information about sender intent, would avoid layering violations
and other complications, and generally be a cleaner and more
efficient model.

---

From: Dave Crocker <[email-masked]>
Date: Sat, 13 Jul 2002 17:43:03 +0900

At 03:11 AM 7/13/2002 -0400, John C Klensin wrote:
> > Internet mail is store-and-forward.  ... This is like packet 
> switching.  Each intermediate system is "relaying" the email.
>
>It is like packet switching except, of course, in the packet case, each 
>packet is typically a fragment or component of a message.

"Typically", perhaps, but it is not true for all packet switching systems 
and it is not part of the formal definition of packet-switching.

Packet switching is about independent addressing (and usually independent 
routing) and handling of discrete chunks of data.  It is not about the 
relationships among that data.  To that extent, Internet mail is quite 
similar to lower-level packet switching.

(I say it is exactly the same, but therein lies a debate that some of us 
have enjoyed for a few years -- errr, decades -- and it is not pertinent to 
the specification under review.)

>   The message, or data stream, is sent one packet at a time; it is not 
> completely assembled at each intermediate host and then forwarded after 
> it is complete.

On the contrary, Arpanet IMP packet switching did fragmentation and 
reassembly in the net, not in the hosts.  (Any further debate on this issue 
might be interesting, but it, too, is not relevant to the current topic.)

>receiving one a question by, well, asking.

The UA-based CONNEG mechanism does query/response between two UAs that are 
connected only by the store-and-forward email service.  There is nothing 
that dictates that application-level query/response exchanges be conducted 
over connection-oriented, direct or real-time associations.

Just to underscore how fuzzy all this gets:  HTTP relies on the underlying 
store and forward of IP.  Email UAs rely on the underlying store and 
forward of SMTP.  And, of course, SMTP also relies on the store and forward 
of IP.

More importantly, these concerns are essential matters for successful 
implementation, deployment and use.  But they have nothing to do with core 
architectural principals.

With respect to direct vs. relay, the architectural difference between a 
UA-UA exchange and an HTTP exchange is remarkably small, especially given 
that HTTP usage pretends it has no context.  These differences get even 
smaller with the introduction of caching servers, and the like, because 
they introduce a very high degree of indirection between consumer and provider.

>desired functionality here in a case that might involve relaying.

Yes, it is an appealing model, except that LDAP has yet to achieve 
Internet-scale deployment and use.  Consequently it is not wise to make a 
new service dependent on it.

It is also worth noting that such a model requires real-time 
connectivity.  An email-based mechanism can operate with asynchronous access.

It makes far more sense to postulate a range of choices -- including 
directly asking the own of the information via a UA mechanism, as well as 
direct intermediaries via ESMTP/CONNEG -- and incorporate LDAP when it 
starts being a serious global service.

>requested, ... is hard or

To the extent that a higher degrees of certainty is required, then the 
operational components should be configured to provide it.  Using LDAP does 
not provide any guarantees of proper or speedy performance, any more than 
using UA- or SMTP-based content negotiations guarantees the lack.

d/

----------

---

From: Keith Moore <[email-masked]>
Date: Sat, 13 Jul 2002 09:51:03 -0400

> specification for that command.  In my opinion, the current specification 

this is a little misleading.

if a message is being sent to more than one recipient then there is a 
possibility that different recipients will need different versions of the 
content, or that some recipients will be unable to accept the content at all.

if the message being transmitted is a fax and all recipients are capable of fax, 
then perhaps it is acceptable to downgrade the message to a lower quality fax 
format which is acceptable to all.   however this is not always possible for
recipients in general - a SMTP server is allowed to support CONNEG if some, 
none, or all of its recipients support fax reception.

the use of the RCPT response to tell the client of the recipient's capabilities
is odd because it asserts that the client intends to deliver a message to
each of several recipients before the client actually knows whether it can 
deliver the same version of the message to each recipient, or whether it can
deliver any version of that message to the recipient.  

so as the proposal is written, any client which attempts to deliver a message
to multiple recipients in a single transaction must be prepared to abort the
current SMTP transaction (presumably by using RSET) after sending multiple RCPTs, 
to divide up the RCPTs according to which version of the message each is capable 
of receiving, and to start some number of new transactions for that purpose.  

if nothing else, that seems a bit confusing and it introduces the potential
for implementation errors.  

it's been claimed  that using RCPT in this way is faster that using a separate 
command, though I haven't actually seen an analysis of this.  I wonder if
such analysis is based on an assumption that it will usually be possible to 
downgrade the message to a common format that is acceptable to all recipients,
and I wonder if that will actually be true in practice.    but even if performance 
considerations justify using RCPT instead of a separate command, it would still 
be appropriate to document the need for clients to abort the current transaction 
if not all recipients can accept the same format.  the current proposal fails
to mention this at all.  this would be easy to fix.

another problem with RCPT is that it expects the server to return status codes
for error conditions that can only be distinguished on the client.

RCAP is just one of several ideas that have been suggested to do content
negotiation in a way that is simpler, more robust, and less confusing to 
implementors than the current proposal.  Dave is correct that there is no 
formal proposal for RCAP - though since the entire reason for proposing RCAP 
is to have something simpler, a detailed proposal hardly seems necessary to 
describe it.  Anyone with a good understanding of SMTP should be able to 
evaluate the relative merits of the current proposal, RCAP, and the other 
suggestions without having a detailed proposal for each one.

The following description for RCAP should be enough:

1. the command looks like this:   RCAP <recipient-address>

2. the server responds with one of the following  

2xx-recipient capabilities are as follows:
2xx-<conneg string goes here>
2xx

2yy recipient has no specified capabilities

2zz no information available about recipient capabilities

4xx recipient capabilities are temporarily unavailable

5xx error (for example, syntax error)

specific values for 2xx, 2yy, 2zz, 4xx, 5xx are to be determined.

3. if the server supports the ENHANCEDSTATUS codes extension then
the most appropriate enhanced status code should follow the 2xx,
2yy, 2zz, 4xx, or 5xx response code in the manner specified in RFC 2034.

4. the RCAP command is acceptable at any time that MAIL or RCPT 
would be acceptable.  clients MAY issue RCAP before or after MAIL,
and they MAY issue RCAP and RCPT commands in any order.
(though generally it makes more sense to issue RCAP for a 
recipient before sending RCPT for that recipient)
The RCAP response MUST be the same regardless of whether the
command is issued before or after MAIL, or before or after RCPT.

5. the RCAP command has no effect other than to query the server
about a recipient's capabilities.

6. the RCAP command does not indicate whether a recipient address
is valid or whether delivery of a message to the recipient will 
succeed.  any response - 2xx, 2yy, 2zz, 4xx, 5xx - is potentially
valid even for a nonexistent recipient or one which is not allowed
to receive mail.  for example, an SMTP server for a webmail 
environment which provided the same capabilities for all recipients
could return 2xx and the same conneg string for any recipient
address, regardless of whether that recipient were valid.

Keith

---

From: Keith Moore <[email-masked]>
Date: Sat, 13 Jul 2002 10:12:21 -0400

> desired functionality here in a case that might involve relaying.

I'm reaching the same conclusion.

> and other complications, and generally be a cleaner and more

I generally agree, or at least I think it's worth considering this
alternative.    Which makes more sense:

- explicitly configuring every general-purpose SMTP server that acts 
  as a mail exchanger (and that supports conneg) to talk to a database 
  for each of the recipient domains that it serves?

  or

- defining a standard recipient capability query mechanism
  (say, something like rescap) and having a standard way to
  advertise via DNS which servers to use to find capabilities 
  for recipients at a particular mail domain (say, something
  like SRV records)

Problem is, you get different answers depending on whether:

- you build appliances that receive fax over the internet (in 
  which case it's simpler to just implement one server process 
  and you'd rather avoid imposing the burden of extra DNS setup 
  on your customers), or

- you build or operate general-purpose SMTP servers (in which 
  case you might rather avoid having to explicitly configure
  those servers, and also avoid having your SMTP servers wait
  on more database queries)

The two approaches are not mutually exclusive, but if we wanted
to support both query methods then it would be good idea to 
understand how they interact sooner rather than later.

Keith

---

From: Keith Moore <[email-masked]>
Date: Sat, 13 Jul 2002 10:16:15 -0400

> over connection-oriented, direct or real-time associations.

perhaps not, but once a message is relayed, the conversion (and the decision
of whether to deliver the message) is no longer under control of the sender - 
actually the most likely scenario when using the conneg proposal with the 
current infrastructure is that the message will be delivered, without 
conversion, regardless of recipient capabilities.

I thought that this is what the fax people wanted to avoid?

---

From: [email-masked]
Date: Sat, 13 Jul 2002 08:55:41 -0700 (PDT)

> and other complications, and generally be a cleaner and more

I think this is a fine piece of theoretical analysis which unfortunately has no
applicability to the real world. In the real world directory/database systems
that let sites provide Internet-wide access to information about their users do
not deploy.

I'm not sure why this is the case, and indeed I'm fairly sure that there is no
good reason for it, but a vast tract of experience tells me it for-sure is the
case. (Now is not the time to get into whys in more detail.)

The IETF has tried to tackle this problem several times, and indeed continues
to do so now in the RESCAP WG. But I don't think anyone seriously believes that
even if RESCAP produces some specifications it is going to succeed at its
intended scale. Frankly, the silence is pretty deafening over there.

Lacking direct capabilities to find capabilties information necessarily leads
to various piggybacking options, that is, deploy this facility on top of or
inside other protocols. This has some of the same characteristics as the
everything-over-HTTP approach, but with several important differences: There is
a close relationship between the piggybacked data and the intended purpose of
the underlying protocols (a relationship that is often lacking with stuff
travelling over HTTP) and the operational case for doing this is far more
compelling (getting the OK to drill a hole in a firewall for a new protocol is
triviality itself compared to the difficulty in setting up even restricted
access to  directory information).

I think the availabilty of capabilities information has the potential to be as
important a step as MIME was. As such, I don't begrudge healthy debate about
how and where it is done. But let's not forget that the development of MIME was
informed by the catastrophic failure of various other schemes for doing
multimedia messaging -- some of them more elegant than MIME -- leading to
compromises in the design intended to insure deployability. I think the
discussion of where and how to do content negotiation needs to be similarly
informed by the catastrophic failure of Internet-wide directory services to
deploy.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Sat, 13 Jul 2002 14:17:58 -0400

> that let sites provide Internet-wide access to information about their users do

Widespread deployment of LDAP as a public directory service was hampered
by several factors - despite the designers' intention that it be used
for publically accessible information it was widely hyped as a generic 
database access protocol, so an organizations' LDAP servers were 
typically filled with information that they wouldn't dream of making publically 
available.  Also, there wasn't an obvious global namespace or infrastructure
that was easy for LDAP users to tie into, nor any widespread understanding of 
why it would be a good idea to do so.

None of this experience seems particularly applicable to the question at hand, 
where we are comparing two different mechanisms for exporting only a small 
amount of information - SMTP, or some other protocol (of which LDAP is only a 
token example).  If an enterprise is reluctant to export "information about 
their users" in general, then they will be about as reluctant to do so via SMTP
as via some other protocol.

I do think it's reasonable to assume that any other protocol server is likely 
to be viewed as a separate maintenance burden that needs to be justified, whereas 
a wart on the side of SMTP will probably be viewed as just some incremental pain 
that must be endured.  But if large sites end up needing a separate database server 
anyway to support CONNEG because the MX servers don't have direct knowledge of 
recipient capabilities, then that separate maintenance burden is there anyway, 
at least for those sites.

I also believe that trying to use LDAP for this purpose would be a marketing 
disaster - because there is already such a mindset about what LDAP is used for,
and what kind of data goes into it, that any proposal to expose this specific
piece of information using the LDAP protocol will immediately get attacked
by people who are concerned that other informaion often stored in LDAP will be
inappropriately leaked.  

> intended scale. Frankly, the silence is pretty deafening over there.

I suspect the lack of significant interest in rescap is because it lacks a
killer app - there are many uses for rescap but no one application for
it sees a significant enough win to drive the people who work on that
application to contribute to rescap.   That and each of those uses involves 
some deployment pain - and the first one involves significant pain, so nobody 
wants to be first.  Another way of saying this is that (at least in the apps 
area) we tend to divide our work around product lines - we tend to work on 
email, fax, voice mail, web replication, instant messaging and so on - and 
rescap cuts across all of these.

(on the other hand, any technology that is initially driven by a narrow set 
of interests tends to get warped to serve that set of interests at the 
expense of others, even if the technology was generally useful.)

I'm not sure whether this is good or bad, nor whether it is inherent or an
artifact of our organizational structure.   Mostly I view this as a given.
There are exceptions, but they're rare, at least in the apps area.

But there's nothing stopping fax people, or email people, or voice mail
people from considering a standard protocol for the lookup of recipient
capability information and (separately) from considering whether that
protocol should be accessed directly or through SMTP.  And while I suspect,
intuitively, that accessing that information via SMTP is actually a 
better way to go overall, I'd like to see some more analysis on it rather
than a quick dismissal based on experience with LDAP.

for instance:

One attractive thing about using SMTP is that SMTP servers for small 
sites can implement CONNEG without a separate server, so the cost is 
kept low for small sites.  Another one is that if there is a separate 
server then it's probably easier for the mail system administrator to
configure his/her MTAs to talk to that server (and to fix bugs in
that configuration) than for the mail system administrator to configure
DNS so that external queries go to that server.

If queries are made via SMTP then the interface to the database is not 
standardized, so MTA vendors end up supporting multiple interfaces.
Some might see this as an advantage, since it allows use of a variety
of existing back-ends.  Actually enough MTAs support mail forwarding
via LDAP (or other database) that getting recipient capabilities from 
the same source is probably a straightforward extension.

And assuming that the SMTP client-SMTP server connection has a 
significantly longer delay path than the SMTP server-database connection
(which seems reasonable) then it could take less time overall to deliver
the message via SMTP alone than via SMTP + a database query, provided
the queries can be made in the same SMTP session as when the message
is delivered.

I think there are benefits that favor of a rescap-like approach also.

One is that it allows the conversion to be done by the sender's
MUA - or at any rate as early as possible - whereas with SMTP
this is not always possible because a number of sites either 
block access to port 25 on nonlocal servers, or impose interception
proxies that pretend to be the remote server even if it's really a 
relay or firewall.  I'm very much in favor of having negotiation 
and transformation work predictably and with accountability to the 
sender rather than having some random MTA in the chain decide that 
a conversion is needed.

Another is that the mechanism could be used for other things besides
just recipient capabilities - say to distribute a recipient's 
public key, to opt out of spam, etc.  These could also be provided 
via SMTP but some of them absolutely require access by the sender's
UA to be useful.

Keith

---

From: [email-masked]
Date: Sat, 13 Jul 2002 12:08:07 -0700 (PDT)

> why it would be a good idea to do so.

LDAP is but one example, and in some ways not a good one. There are plenty of
others.

> as via some other protocol.

I'm afraid this is another piece of theoretical considerations that fly in the
face of actual experience. The obvious counterexample is HTTP.

> at least for those sites.

While I obviously cannot speak for all large sites, I can tell you that
database usage is sufficiently pervasive inside the sites I'm familiar with
that this is basically a nonissue.

> by people who are concerned that other informaion often stored in LDAP will be

Nowhere did I mention using LDAP to expose this information. So this appears to
be a complete strawman argument to me.

> than a quick dismissal based on experience with LDAP.

First, this is not a quick dismissal, it is a conclusion based on decades of
experience, experiencce in which LDAP only plays a small and relatively recent
part.

Second, I believe the notion of using a directory for this sort of thing has
been discussed in the FAX WG on several occasions. And while it is axiomatic
that nothing precludes the WG's having chosen to use a directory, nothing
requires that a WG pursue it either. I take the rather stunning lack of
interest in this overall approach as yet another sign that it is seen as
nonviable.

> the same source is probably a straightforward extension.

There's no "probably" about it, although it is also true that more generic
callouts aren't exactly rocket science to implement.

> sender rather than having some random MTA in the chain decide that

I agree that predictability is nice, but that's not something any approach can
ever guarantee. Indeed, I can argue that a directory approach will lead to less
predictability, not more, if for no other reason than directory access will
tend to work locally but not remotely. SMTP access with appropriate caching, on
the other hand, has the potential to work both locally and globally.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Sat, 13 Jul 2002 15:55:56 -0400

> face of actual experience. The obvious counterexample is HTTP.

The relevance of HTTP is even less obvious to me because HTTP and directory 
services (LDAP or otherwise) tend to be used in very different ways.  I don't 
see many sites exporting their company directories via either HTTP or LDAP.  
Nor do I see how any of this is relevant  to the problem of exporting recpient 
capability information for content negotiation.

> that this is basically a nonissue.

Well, I guess it hinges on what is meant by "large".  Are there a significant
number of sites "large" enough to not have their MX servers deliver directly
to recipient mailboxes, but not "large" enough to be already using a database
to store recipient information?  I suspect the answer is yes because there
are other reasons besides scale that a site would not have its MX servers
deliver directly to recipient mailboxes.  Inbound firewalls are fairly common,
and some of them operate as relays.

> experience, experiencce in which LDAP only plays a small and relatively recent

not to dismiss your experience at all - but your stated justification didn't
seem (to me) to be particularly relevant.

> interest in this overall approach as yet another sign that it is seen as

again, is this a fax-specific extension or is it supposed to be generally
applicable?   I'll readily admit that fax machines don't need a directory.
but should the level of interest within the fax group be used as a measure
of the merit of the idea for other uses of email? 

> the other hand, has the potential to work both locally and globally.

The last two statements seem fairly odd.  Maybe the problem is the use of the
word "directory" which isn't my favorite word for this anyway.  So let's not
call this a directory - let's call it a capability discovery protocol.  Now if
the capability discovery protocol is defined in such a way that in order for
your recipients to benefit from content negotiation you have to make it accessible 
(just like if you want to be able to accept mail then you have to make port 25 
accessible) then it seems fairly likely that sites that want to enable content 
negotation for their recipients will do so.  On the other hand, port 25 is one 
of the few ports that is commonly blocked/filtered for outgoing traffic - 
perhaps even more often (and with more drastic filtering) than port 80.

So the argument that a facility that requires direct connection over port 25
(from the origin to the MX server, not necessarily to the final MTA)
in order to work properly (i.e. in order to let the sender control the 
conversion) will function both locally and globally seems a bit dubious.

Keith

---

From: [email-masked]
Date: Sat, 13 Jul 2002 13:04:52 -0700 (PDT)

> see many sites exporting their company directories via either HTTP or LDAP.

Then you haven't looked very hard. The former is quite popular; the latter
practically nonexistant.  And the fact that the same information is exposed
either way yet one approach is preferred is directly relevant to the matter
at hand.

> capability information for content negotiation.

In a rational world it would not be relevant. We don't live in one of those.

> negotation for their recipients will do so.

My point was they may make it available internally but not externally. And
using anything that remotely smells like a directory protocols makes this
outcome far more likely.

> On the other hand, port 25 is one

Keith, my point was and is that the minute you start talking about using a
directory protocol -- and I insist on using the term because it's the directory
aspect that's important -- it makes the whole thing problematic in a way that
piggybacking the information on top of other type of protocols does not.

You can talk till you're blue in the face about how this is capabilities
discovery, how that's intrinsicly different from a general directory
protocol, and so on and so forth. It doesn't change the perception that
a protocol intended solely for looking up stuff, no matter how innocuous
that stuff is, no matter how divorced from the cporpoate directory or
whatever it can be made, is somehow more problematic than piggybacking
the exact same information on top of other protocols.

> conversion) will function both locally and globally seems a bit dubious.

I have pointed out previously that such direct connection isn't required by the
present proposal. So this is nothing but another strawman.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Sat, 13 Jul 2002 16:33:03 -0400

Ned, Keith, and others,

I'm going to try to respond to several messages in one, please
bear with me.  Also, please note that I've had more than my share
of terrible ideas and it is quite possible that the notion of a
MUA getting information out of a directory about recipient
capabilities is another one.

--On Saturday, 13 July, 2002 08:55 -0700
[email-masked] wrote:

>> (LDAP or otherwise) is the only rational way of implementing
>> the desired functionality here in a case that might involve
>> relaying. To the extent that speedy and confirmed delivery are
>>...
>> But, if database lookup is the only plausible implementation
>> that is consistent with the underlying design requirements,
>> then we should not be looking at capabilities negotiation in
>> the SMTP stream.  Instead, we should be looking at a piece of
>> protocol by which the sending system (probably the MUA, rather
>>...
> I think this is a fine piece of theoretical analysis which
> unfortunately has no applicability to the real world. In the
> real world directory/database systems that let sites provide
> Internet-wide access to information about their users do not
> deploy.

Well, it depends on how important they are.  And, while it may be
tautological, the observation that these things have not deployed
is an indication that they haven't been important enough, given
deployment and maintenance costs and security risks.  Of course,
we do have one wildly-successful counterexample and, while I hate
to even mention it as a possibility, if this capability is
important enough, and it were the right answer (I have serious
doubts about both), nothing prevents us from thinking about a
mail-capabilities RR.  It would be a bad idea if only because the
capabilities are really an MUA issue and the DNS identifies MTAs
(today), but...

>...

> The IETF has tried to tackle this problem several times, and
> indeed continues to do so now in the RESCAP WG. But I don't
> think anyone seriously believes that even if RESCAP produces
> some specifications it is going to succeed at its intended
> scale. Frankly, the silence is pretty deafening over there.

I certainly agree.

> Lacking direct capabilities to find capabilties information
> necessarily leads to various piggybacking options, that is,
> deploy this facility on top of or inside other protocols. This
> has some of the same characteristics as the
> everything-over-HTTP approach, but with several important
> differences: There is a close relationship between the
> piggybacked data and the intended purpose of the underlying
> protocols (a relationship that is often lacking with stuff
> travelling over HTTP) and the operational case for doing this
> is far more compelling (getting the OK to drill a hole in a
> firewall for a new protocol is triviality itself compared to
> the difficulty in setting up even restricted access to
> directory information).

Yes, but as Keith points out (more politely), using the general
organizational directory for this purpose would be madness.  I
hope you didn't assume I was proposing it.   But the "piggyback
everything, rather than think about new protocols" approach
(which I don't think you believe in either, at least in the
limiting case) leads to a different sort of madness, one which
often manifests itself as "give up on all other application
protocols and just use HTTP" (or, more recently, SIP).

> I think the availabilty of capabilities information has the
> potential to be as important a step as MIME was. As such, I
> don't begrudge healthy debate about how and where it is done.
> But let's not forget that the development of MIME was informed
> by the catastrophic failure of various other schemes for doing
> multimedia messaging -- some of them more elegant than MIME --
> leading to compromises in the design intended to insure
> deployability. I think the discussion of where and how to do
> content negotiation needs to be similarly informed by the
> catastrophic failure of Internet-wide directory services to
> deploy.

That should clearly inform many of the things that we do, but we
should learn the right lessons.   Just as "firewalls get in the
way of anything new, therefore we should run all new protocols
and services over HTTP" is typically a wrong answer, developing a
paranoid fear of directory-like services. Prior attempts have
been over-ambitious for what they promised to deliver relative to
what people needed.  Or they have failed for other reasons.   But
it may be that the lesson is "more focused and less ambitious and
general purpose approaches", rather than discarding the
bathwater, the baby, and anything that vaguely resembles either
because, e.g., it is wet.

So, given that I'm already generating plausibly-bad ideas, think
about a strawman.  Suppose we said "if you want to use email
capabilities, you need to be able to establish a connection to
the final recipient" (whatever that means).  This is, of course,
a variation on the "no relay" scheme.   But then suppose that,
instead of piggybacking on top of SMTP, we required that the
destination host have an A RR, or a very specific and slightly
peculiar SRV RR, or something similar, such that the client MUA
could open a connection to it, ask "what sort of fax capabilities
are you willing to provide to a message coming from foo.bar.baz
to [email-masked]", and get back a quick answer.  At the
price of a new RR type, it could even be done over UDP, on a "get
an answer or give it up and send the minimum" basis.  Or, at the
price of committing an obscenity, it could be done pretty easily
over HTTP or SIP.  The UDP approach, and probably the other two,
would almost certainly be faster than getting anywhere near an
LDAP server.   Those approaches are consistent with what I
intended to imply by "...LDAP (or something else)".  

It would require some configuration, but so does piggypbacking on
SMTP.  Is it important enough to deploy?  I don't know.  But, if
it isn't, is this capability really worth cluttering up SMTP?
Remember that quote from RFC 1425 and its successors... your name
is on that document too.

You also wrote:

> Second, I believe the notion of using a directory for this sort
> of thing has been discussed in the FAX WG on several occasions.
> And while it is axiomatic that nothing precludes the WG's
> having chosen to use a directory, nothing requires that a WG
> pursue it either. I take the rather stunning lack of interest
> in this overall approach as yet another sign that it is seen as
> nonviable.

Since I have been paying only intermittent attention to the Fax
WG, I missed those discussions.  But most of the discussions
(about other topics) I remember from that WG were focused,
appropriately, on fax.   One of the reasons we do IETF last
calls, IMO, is to be sure that applicable considerations outside
the scope of the WG are exposed and aired.  So, unless you are
convinced that the WG carefully and accurately balanced the
general (1425/1869, etc) argument against SMTP extensions on
behalf of the overall community, I'd be interested in their
reasons for rejecting alternate approaches but might not find
them compelling.

> understand how they interact sooner rather than later.

And it is a good idea to weigh the strengths and weakness of
each, in an overall Internet and email context, not just a fax
one (unless this capability is to be narrowed to fax
applications).  That is why, regardless of the outcome of this
discussion, I think it is a good use of time and appreciate your
willingness to engage in it without dismissing the issues as
spurious.

     john

---

From: John C Klensin <[email-masked]>
Date: Sat, 13 Jul 2002 16:42:21 -0400

--On Saturday, 13 July, 2002 09:51 -0400 Keith Moore
<[email-masked]> wrote:

> would be acceptable.  clients MAY issue RCAP before or after
> MAIL, and they MAY issue RCAP and RCPT commands in any order.

Keith, I don't assume this would be heavily used, but, as with
SIZE, I can imagine some sites being willing to accept more
complex formats from some senders than from others.  Of course,
such mechanisms are easily thwarted in the absence of SMTP
authentication or trusted transport (and would not be available
with some of the other proposals). But, unless you have a
substantive reason for permitting RCAP before MAIL, it would seem
desirable to force it to come after MAIL, thereby preserving this
implementation option at substantially no cost.

      john

---

From: John C Klensin <[email-masked]>
Date: Sat, 13 Jul 2002 17:11:13 -0400

Dave,

You are, I think, equating packet fragmentation and reassembly to
email messages.  IMO, that is bogus (and, as you point out, this
argument is decades old and I believe it has been bogus for at
least that long), unless you constrain those packets to reliable
delivery and permit them to be very large.  I.e., the only
reasonable comparison is to a collection of packets in a a
connection-mode (or other serialized and reliable) context.  It
was in that context that I used the term "message" -- typically a
multipacket entity in the TCP and IP layers just as an email
message consists of a series of SMTP turnarounds over TCP.  In
that context, I stand by my assertion.   And the reassembly of
packets (as distinct from data streams/ messages) in the ARPANet
is, I'm sure, fascinating to the few people on this distribution
who didn't know about it and care.

      john

--On Saturday, 13 July, 2002 17:43 +0900 Dave Crocker
<[email-masked]> wrote:

>> message.
> "Typically", perhaps, but it is not true for all packet
> switching systems and it is not part of the formal definition
> of packet-switching.
> 
> Packet switching is about independent addressing (and usually
> independent routing) and handling of discrete chunks of data.
> It is not about the relationships among that data.  To that
> extent, Internet mail is quite similar to lower-level packet
> switching.
> 
> (I say it is exactly the same, but therein lies a debate that
> some of us have enjoyed for a few years -- errr, decades -- and
> it is not pertinent to the specification under review.)
> 
> 
>>   The message, or data stream, is sent one packet at a time;
>>   it is not  completely assembled at each intermediate host
>> and then forwarded after  it is complete.
> 
> On the contrary, Arpanet IMP packet switching did fragmentation
> and reassembly in the net, not in the hosts.  (Any further
> debate on this issue might be interesting, but it, too, is not
> relevant to the current topic.)
> 
> 
>> > relaying might take much longer.  Therefore the sender does
>> > not know when it will receive a response.
>> 
>> I would disagree.  The primary difference I see between
>> relaying and direct is that, in a direct arrangement
>> (characteristic of e.g., HTTP) the originating sending system
>> can ask the ultimate receiving one a question by, well, asking.
> 
> The UA-based CONNEG mechanism does query/response between two
> UAs that are connected only by the store-and-forward email
> service.  There is nothing that dictates that application-level
> query/response exchanges be conducted over connection-oriented,
> direct or real-time associations.
> 
> Just to underscore how fuzzy all this gets:  HTTP relies on the
> underlying store and forward of IP.  Email UAs rely on the
> underlying store and forward of SMTP.  And, of course, SMTP
> also relies on the store and forward of IP.
> 
> More importantly, these concerns are essential matters for
> successful implementation, deployment and use.  But they have
> nothing to do with core architectural principals.
> 
> With respect to direct vs. relay, the architectural difference
> between a UA-UA exchange and an HTTP exchange is remarkably
> small, especially given that HTTP usage pretends it has no
> context.  These differences get even smaller with the
> introduction of caching servers, and the like, because they
> introduce a very high degree of indirection between consumer
> and provider.
> 
> 
>> relaying.
> Yes, it is an appealing model, except that LDAP has yet to
> achieve Internet-scale deployment and use.  Consequently it is
> not wise to make a new service dependent on it.
> 
> It makes far more sense to postulate a range of choices --
> including directly asking the own of the information via a UA
> mechanism, as well as direct intermediaries via ESMTP/CONNEG --
> and incorporate LDAP when it starts being a serious global
> service.
> 
> 
>> To the extent that speedy and confirmed delivery are often
>> cited as important values in fax, the "maybe it will get
>> delivered as requested, ... is hard or
> To the extent that a higher degrees of certainty is required,
> then the operational components should be configured to provide
> it.  Using LDAP does not provide any guarantees of proper or
> speedy performance, any more than using UA- or SMTP-based
> content negotiations guarantees the lack.
> 
> d/
> 
> ----------

---

From: Keith Moore <[email-masked]>
Date: Sat, 13 Jul 2002 17:11:24 -0400

> either way yet one approach is preferred is directly relevant to the matter

what can I say - most of the company directories I've seen are not publically
accessible.  I don't doubt that many companies use HTTP to provide internal 
access; whether they make them available to the general public (which is what
I meant by "export") is a different question.  

> using anything that remotely smells like a directory protocols makes this

ah.  okay, I think we're in agreement about that (and that was very similar
to my point about LDAP) - though whether this capabliity discovery protocol
smells like a directory protocol may to some degree be a matter of social 
engineering, aka marketing.

> piggybacking the information on top of other type of protocols does not.

well, if *we* insist on calling it a directory protocol then of course people 
will confuse it with other things they call directories.  On the other
hand sites seem happy to consider exporting limited information via DNS,
and extensive information via HTTP, SOAP, .NET, or whatever.  So if the
marketing lingo has to be "recipient capability web service" or some such,
so be it.  (as long as we don't actually *use* SOAP or .NET... :)

as to whether rescap would be branded a directory - the number of people who 
kept insisting that it should be LDAP was fairly significant, indicating
that many people couldn't readily distinguish the two problems.  this
even though one of the major reasons for making rescap a separate protocol was 
to distinguish it from directories and specifically to have a mechanism 
tailor-made for exporting public information.  but frankly part of that problem
was that some LDAP people were seeing rescap as a competitor/threat to LDAP.

as you say, it's not a sane world.  URNs have an even worse mis-perception 
problem, they always have, and it's not clear what to do about it.

> the exact same information on top of other protocols.

okay, I'll concede the point that there's this dark cloud over things
that (in the minds of network administrators) resemble directories.  
I don't think that should by itself stop working groups from using things 
that resemble directories if that's the right technical solution,
but it's useful to be aware of it.  it does I think illustrate that good
technical design is not always sufficient - unless we somehow convey
to users an understanding of new protocols, how they're meant to be used,
the risks associated with using them, etc.  And we haven't been very
effective at that since the 'net grew up...

> present proposal. So this is nothing but another strawman.

ah, but either the client has a direct connection to the MX or it has to
relay the message to some MTA which may or may not be under control of the
client.  so the relaying case introduces the same problem that OPES had
when it was first proposed - that of having an intermediary make transformations
to the content that aren't authorized by either party of the conversation.

Keith

---

From: Keith Moore <[email-masked]>
Date: Sat, 13 Jul 2002 17:29:58 -0400

> implementation option at substantially no cost.

my assumption was that the return address is essentially useless for 
determining the kind of content that would be acceptable to the 
recipient - partially since they are easily forged, and partially
since there are various reasons to use a different return address than
one's normal email address.  SMTP authentication credentials might 
be useful for determining the conneg string returned, but those 
would presuambly be presented before MAIL.

my reason for allowing RCAP before MAIL was to allow query-only
sessions without having to specify a return address that won't
get used anyway and just serve to slow the query down.

still I don't claim to have nailed down every detail - this is just
a "strawman" proposal so that people have something to compare
with the proposal that uses RCPT.

Keith

---

From: [email-masked]
Date: Sat, 13 Jul 2002 14:22:19 -0700 (PDT)

> I meant by "export") is a different question.

I speak as someone who used to sell and support software that did this sort of
thing, both via HTTP and LDAP. I saw plenty of examples of public export of
directory information, including but not limited to user information, through
HTTP, very few through LDAP.

I really can't explain the difference. My assessment was and is that a
carefully designed LDAP->LDAP gateway is likely to be more secure than a random
Web server lashed up to an LDAP server.

> was that some LDAP people were seeing rescap as a competitor/threat to LDAP.

As an aside, I think we actually foster this attitude given the degree and tone
of pushback that's often present in the IETF. For what may seem like the best
of reasons people often have to fight to get their work through the process.
But once that's done you're left with people trained to fight.

> effective at that since the 'net grew up...

Well, let's be fair here: We can lead the horse to water...

I'm probably a bit biased since I wrote a bunch of the text, but I think we did
a reasonable job of documenting the risks of active content back when MIME was
first specified. And yet active content in the form of email viruses is one of
the major operational problems, if not THE operational problem, we face in
email today. We did our bit and nobody listened.

It's quite a conundrum. Even when we do our job things still go wrong. Some
folks argue that things capable of abuse should not be standardized. But there
are substantive and possibly even greater risks associated with the lack of
standardization as well.

> to the content that aren't authorized by either party of the conversation.

The case I was thinking of was the one where caching of conneg information by
intermediaries eventually brings the information back to the server adjacent to
the originating client. So the transformation is still done on the originating
client or on a server very close to it.

I also discussed the situation where conneg transformation is done by a central
intermediary in a previous message and concluded that there seemed to be
no business case for it.

I should also note that the class of tranformations envisioned by OPES
is considerably broader than the class of transformations envisioned
here.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Sun, 14 Jul 2002 06:50:34 +0900

...

>format which is acceptable to all.

Ever since multi-recipient addressing was introduced, we have had 
downgrading, by virtue of users having to choose the least common 
denominator approach to sending attachments.

So the fact that this new mechanism allows that downgrading to happen later 
in the transmission sequence should not confuse anyone into thinking that 
downgrading is a new or unusual requirement.

>deliver any version of that message to the recipient.

"or whether it can"???  There has been no posting that provides a practical 
basis for such a conclusion.

Therefore, any implications of this unproved -- and frankly 
counter-intuitive and highly unlikely -- assessment has nothing to do with 
the current specification.

>current SMTP transaction (presumably by using RSET) after sending multiple 

It is NOT required to do that.  Nor is the ability to send an RSET during a 
session a big deal.  In fact, that's what the command is there for.

>command,

Faster for single-recipient use, yes.  Faster for multi-recipient use, with 
common format, yes.

In both cases there are fewer commands and fewer round-trips than when 
using a separate capabilities-query language.  It does not take deep 
analysis to know that the current proposal is faster.

Faster for the particular scenario you used, with an RCPT? No.  That has 
not been claimed. Using RSET is expensive.  A design that calls for its use 
makes sense only if that use is expected to be atypical.

As has been discussed previously use of the current specification is based 
on the expectation that use of RSET will not be a typical.

>for error conditions that can only be distinguished on the client.

There is no such expectation:

         "If the SMTP server supports ENHANCEDSTATUSCODES [RFC1893], the..."

Apparently you missed the "If".

In any event, i have no idea why the codes are somehow uninterpretable by 
any system other than the client.

At 10:16 AM 7/13/2002 -0400, Keith Moore wrote:
> > connected only by the store-and-forward email service.  ...
>

I have no idea what you mean by "under control of the sender" nor do I have 
any idea how the current proposal has any practical difference in control, 
from any other proposal or idea for content negotiation.

>conversion, regardless of recipient capabilities.

And I have no idea where this assessment is based on, nor does it make any 
obvious sense.

d/

----------

---

From: Dave Crocker <[email-masked]>
Date: Sun, 14 Jul 2002 06:14:20 +0900

Folks,

Ned's note explains some operational realities and some architectural 
implications.  His note  is the clearest, the most pragmatic, and the most 
compelling statement on this issue that I have seen.

It is easy to miss just how important his explanation is.  So I am taking 
the slightly theatrical step of re-posting it, in order to stress the 
importance of its insight:

At 08:55 AM 7/13/2002 -0700, [email-masked] wrote:

>In the real world directory/database systems

----------

---

From: Dave Crocker <[email-masked]>
Date: Sun, 14 Jul 2002 10:19:46 +0900

At 05:11 PM 7/13/2002 -0400, John C Klensin wrote:
>email messages.

No I was not.

I DID, however, note that the issue of fragmentation was a) not part of the 
definition of packet switching, and b) irrelevant to the use of the term in 
the current discussion.

d/

----------

---

From: Graham Klyne <[email-masked]>
Date: Sun, 14 Jul 2002 15:06:26 +0100

At 12:04 PM 7/13/02 +0900, Dave Crocker wrote:
>For the IETF Fax working group, I believe "fax-specific" refers to 
>features and functions that emulate T.30.  That is, we are trying to make 
>Internet mail provide the same service as T.30.  So, "fax-specific" refers 
>to email enhancements that permit email to work like fax.

I'm uneasy about (appearing to) equate fax-specific with just T.30 -- I 
think your last sentence carries the intent if not the desired precision -- 
maybe:
[[
"fax-specific" refers to email enhancements that permit email to negotiate, 
transmit and confirm the transfer of images in timely fashion, like 
traditional fax.

Traditional email, by contrast, transmits any data and does not impose any 
requirement that the data can be processed by the recipient or that the 
recipient confirm its receipt.
]]

>services permit the sender to specify more than one receiver.  This is 

If you mention fax store-and-forward, I'd suggest a comment to the effect 
that such services behave in a fashion more like ordinary email than 
fax.  (I.e. saying that fax does store and forward like email doesn't of 
itself mean that providing a fax-like service requires Internet fax should 
support all fax features for multiple recipients.)

>common denominator" or "most common format" to all of the recipients at once.

This term has often puzzled me as being incorrect in its mathematical 
allusion.  The terms "greatest common divisor" or "highest common factor" 
seem more apposite.

>latency requires different design choices.

I think there's a more fundamental architectural issue that is exploited by 
traditional fax operating in a non-relay mode:  normal operation requires 
that the sender can receive some indication from the receiver before it 
starts to send any actual data.

>When we add a new feature, how does it affect the old 
>operation?  Sometimes adding a new features makes the old system stop 
>working.  For MIME, the goal was to add a new feature that made no change 
>to email transport and also would not break an old email user agent.
>

A further consideration that very strongly influenced the user-agent-based 
negotiation mechanism is that no messages should be sent in assumption that 
the receiver is anything other than a plain old fashioned mail client, and 
that all messages received by such a client could be presented to the 
recipient user as ordinary email messages.  ("No administrative non-messages").

>milliseconds or microseconds.

My working definition of real-time:  if some data is received late, then it 
is wrong.  This pointedly doesn't say what it means to be late.

>>- bandwidth
>
>The amount of data that can be send within a period of time.  The 
>bandwidth of a T1 (or E1) line is very much greater than a dial-up connection.

A closely related concept is latency:  the amount of time that elapses from 
sending some data to the point that the receiver starts to receive it.

#g

-------------------
Graham Klyne
<[email-masked]>

---

From: Graham Klyne <[email-masked]>
Date: Sun, 14 Jul 2002 15:19:10 +0100

>informed by the catastrophic failure of Internet-wide directory services to

(Not to disagree with anything you say...)

One of the considerations informing the UA-driven negotiation design was 
the idea that information obtained from any source other than the final 
recipient was potentially tentative, and that some kind of closing of the 
loop was needed to make negotiation reliable.  The better the initial 
information that is available, the more efficient the 
negotiate-and-transfer process (one round trip).  But incorrect initial 
information doesn't cause the transfer to fail (except possibly for reasons 
of timing).

#g

-------------------
Graham Klyne
<[email-masked]>

---

From: Keith Moore <[email-masked]>
Date: Sun, 14 Jul 2002 12:40:58 -0400

> email today. We did our bit and nobody listened.

I have a slightly different take - we did our bit and the major operating
system vendor listened and irresponsibly chose to ignore us, putting their
desire to have their applications' data formats (including executable content)
transferred over email ahead of their customers' need for security.

I was explicitly told this by a former program manager for that major 
operating system vendor, though he didn't seem to think it was irresponsible.
He called it a marketing decision.

> client or on a server very close to it.

I think it's a stretch to believe that caches are going to solve this problem.
It's true that if A keeps sending messages to B then eventually A's MTA will 
be able to cache B's recipient information.   But this assumes that the 
intervening MTAs support CONNEG (and cache CONNEG information) - which itself 
seems like a stretch because, as you say, there's no business case for them
doing conversion.  It's also a stretch to believe that this is going to work well 
for arbitrary pairs of senders and recipients - it might work for those pairs who 
exchange lots of messages, not so well for those pairs that occasionally exchange 
messages.  And of course there's the problem that CONNEG information can be
expected to change over time, but there's no mechanism in the current proposal
to timeout caches.  Frankly the method of returning capabilities in a bounce
message seems more effective and more reliable because it doesn't rely on
intermedaries.

The only way I see to make the CONNEG SMTP proposal work end-to-end is for
all of the intermediate MTAs (those that don't have direct knowledge of the
recipient capabilities) to open up SMTP connections to the next hop *in real time* -
so when the sender asserts CONNEG in a RCPT command then the MTA has to stop
acting like a relayer and start acting like a proxy - finding a MTA for the
next hop, opening up a connection, sending EHLO and seeing if it supports
CONNEG, sending MAIL and all previous RCPTs, and finally sending the RCPT
that has the CONNEG option, just so that it can propagate the CONNEG response
back to the sender's UA.  Even then it will be very slow, it introduces
race conditions due to timeouts, and it still requires all of the intermediate 
MTAs to support CONNEG in proxy mode.

I haven't followed the fax discussions, but weren't the fax people looking for 
a mechanism that would work predictably and reliably - e.g. if sender and 
recipient both have (say) high-res capability then the document is always 
transferred in high-res - and a mechanism that would give the sender the option 
of requiring that the recipient have certain minimum capabilities before sending 
the message?  (e.g. don't bother sending if recipient doesn't have color?)
I just don't see a way to make that work through the existing SMTP infrastructure.

Keith

---

From: Keith Moore <[email-masked]>
Date: Sun, 14 Jul 2002 13:12:53 -0400

> downgrading is a new or unusual requirement.

Dave, it's a gross exaggeration to claim that the fact that human senders have
long found it advantageous to send messages to multiple recipients using a
common format of their choosing, that this is a good justification for 
standardizing the idea that MTAs that are responsible to neither the sender 
nor the recipient should be able to "downgrade" a message, using arbitrary
and unspecified methods, without any knowledge of the consequence of applying
conversion to the content, and without any explicit authorization.

> basis for such a conclusion.

Yes there has, but perhaps you missed it.  Regardless you should be able to
figure out that your proposal provides no assurance that downgrading is even 
possible.  just to take a trivial example - the message might contain only 
audio while one recipient accepts only fax.  Another example - the recipient 
accepts only HTML and plain text, the message is in microsoft word, and while 
conversion is possible the MTA doesn't know how to do it.  

so yes, "whether it can" is entirely appropriate wording.

> the current specification.

Dave, the most charitable thing I can say is that you must be living in a 
different universe.  You have consistently responded to valid, well-explained,
and defensible technical criticisms by flat-out stating that they don't 
exist or that they don't make any sense, without any defense.  You seem 
to think that everyone who uses this extension you're propsing will have
the same assumptions that you do, and adhered to a set of constraints
that exists only in your head.  And if you don't understand your own 
proposal well enough to make intelligent responses to technical criticism
then frankly you need to withdraw it and stop wasting our time.

Keith

---

From: John C Klensin <[email-masked]>
Date: Sun, 14 Jul 2002 14:22:49 -0400

--On Sunday, 14 July, 2002 06:50 +0900 Dave Crocker
<[email-masked]> wrote:

>...
> Ever since multi-recipient addressing was introduced, we have
> had downgrading, by virtue of users having to choose the least
> common denominator approach to sending attachments.
> 
> So the fact that this new mechanism allows that downgrading to
> happen later in the transmission sequence should not confuse
> anyone into thinking that downgrading is a new or unusual
> requirement.
>...

Dave, I want to be sure I understand what you are saying here,
because, if I am reading it correctly, I find it astonishing.

First of all, I don't find, anywhere in the document, a statement
that the originator, in attempting to use CONNEG, is authorizing
any intermediate MTA that might touch the message to alter its
content.  Nor do I find in the "security considerations" an
explanation of that authorization and the associated risks.  If
such text is actually there, then some of what follows may be
irrelevant (although it wouldn't convince me it is a good idea).

Certainly, originating MUAs downgrade and change formats based on
what they, on behalf of the user, believe the recipient will be
able to process and have done that for a long time.  But they do
so before the message enters a transmission stream.  We have
typically tried to keep that transmission stream as opaque to
message content as possible.   In recent years, we have made some
formal allowances for fixups by originating MTA for
low-capability MUAs, but we have rather carefully delimited them
(see sections 3.7, 6.1, and 6.3 of RFC 2821 (the references to
section 2.4.1 are incorrect and should refer to 6.3)).

We also permit downgrading through ESTMP options like 8BITMIME,
which is quite explicit that an initiator who asks for this
feature is agreeing to a (very specific, known-result) downgrade
by relaying systems.

But, unless I read the above incorrectly, you are suggesting that

	* Since the originating MUA clearly has the ability and
	authority to alter content to reflect its assumptions
	about receiver capability (before the content is handed
	off to an SMTP server and placed into the transport
	stream)
	
	* Other alterations to content ("downgrades") are just
	moving the process "later in the transmission sequence"
	
	* Hence intermediate/relay MTAs have the right/authority
	to make content changes based on their inferences about
	what the receiver wants.

I hope that isn't what you meant, because it is pretty scary.  It
also creates a contradiction between this draft and RFC 2821 (as
well as the way most of us have read 821 for years), raises a
collection of security issues (e.g., in-transit downgrading would
seem to me to be quite hard on signed content), and so on.

Could you explain what you did mean?

   john

---

From: [email-masked]
Date: Sun, 14 Jul 2002 14:54:11 -0400

> case. (Now is not the time to get into whys in more detail.)

I'll just add 2 words to the "why" and leave it at that:

"datamining" "spammers".
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked]
Date: Sun, 14 Jul 2002 15:28:15 -0400

> downgrading is a new or unusual requirement.

My MUA is authorized to make decisions regarding downgrading and choice of
C-T-Es for attachments.  It listens to my choices regarding base64 versus QP,
For the most part, said conversions are lossless (and a few, such as
PGP-signing are even information-added), but I have authorized it to do lossy
conversions under a a few sets of circumstances, and I *know* about my MUA's
actions *before* I hit 'send'.  This is not news, and hasn't been for at least
a decade.

On the other hand, once my MUA hands it to my MTA, *I DO NOT EXPECT THE MTA TO
PERFORM LOSSY CONVERSIONS*. Period. End Of Discussion.

To quote the 'Security Considerations' of the original draft:

     A man-in-the-middle attack might change the capabilities reported 
     for a given recipient. For example: Suppose the sender knows the 
     recipient has the ability to view color documents so they mark 
     some things in red in what is otherwise a black and white 
     document. But someone interferes with the returned capabilities, 
     indicating that the recipient only supports black and white. The 
     document is duly downgraded, with the result that the recipient 
     doesn't see what the sender marked. 

Hey hey... What happens if this happens by *ACCIDENT*?  Currently, if I
think the recipient has color, and I do this, the recipient hopefully
gets at least a *warning* "trying to open a color image, converting to
gray scale", and can notify me that there was a problem.  However, the spec
as written doesn't even require that the MTA add a 'X-Downgraded-by:' header,
and the recipient has *no clue* that what he received wasn't what I sent.

And that, Dave, *IS* a "new and unusual" state of affairs.
-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked]
Date: Sun, 14 Jul 2002 12:48:22 -0700 (PDT)

Private note since this is now entirely off topic...

> "datamining" "spammers".

This is the opposite of an explanation, I'm afraid. Techniques for protecting
against datamining are well known and widely implemented. In fact it is almost
certainly easier to implement them in LDAP than in, say, a Web interface. Yet
the former is rarely done, the latter pretty common.

				Ned

---

From: [email-masked]
Date: Sun, 14 Jul 2002 13:20:41 -0700 (PDT)

My apologies, I intended to edit the cc list but neglected to do so.

				Ned

---

From: Dave Crocker <[email-masked]>
Date: Mon, 15 Jul 2002 09:56:00 +0900

At 02:22 PM 7/14/2002 -0400, John C Klensin wrote:
>content.

For a specification that is designed to permit a sending agent to know how 
to tailor the form of a content, it is difficult to imagine what benefit 
there can be in having that specification  contain a statement saying that 
an implementation is "authorized" to tailor the form of that content.

>   Nor do I find in the "security considerations" an
>explanation of that authorization and the associated risks.

You clearly have a better sense of this issue than I do, since I do not see 
that there is a problem.

So, feel free to offer some text.

>   We have
>message content as possible.

Yup.  It sure is scary to try to do new things.

Except when there is a well-established basis for doing them.

I am not inclined to dismiss that established basis solely because it came 
from a non-IETF effort, particularly given the scale of success of that 
prior work.

>         to make content changes based on their inferences about

Let's try to stay on the current topic.  Arguing about generic principles 
and then trying to dismiss current work based on some subtle variation in 
interpretations of that abstract discussion is not going to be productive.

You might want to review prior work on interprocess communication, which 
explored trade-offs between a "sender makes right" model and a "receiver 
makes right system".

The significant difference between esmtp/connect and multipart/alternative 
is exactly this distinction.

(Hint:  sender makes right has nearly always been the preferred IETF 
choice.  esmtp/conneg is sender makes right.)

>Could you explain what you did mean?

What I mean is very simple:

         The  behavior that is specified in esmtp/conneg is a careful 
increment in functionality and is based on extensive prior experience and 
is known to be useful.

----------
Dave Crocker <mailto:[email-masked]>
TribalWise, Inc. <http://www.tribalwise.com>
tel +1.408.246.8253; fax +1.408.850.1850

---

From: Keith Moore <[email-masked]>
Date: Sun, 14 Jul 2002 22:15:06 -0400

> an implementation is "authorized" to tailor the form of that content.

Dave, that may be what your specification is *designed* to do.  It is
not what your specification *does*, except in the rare case where the
sender's UA is able to communicate directly with the recipient's MX.
And the sender's UA is the only "agent" that is typically under control of
the sender.

> So, feel free to offer some text.

How about this:

This extension MUST NOT be used by any SMTP client which is not authorized
to perform conversion by either the sender or the recipient of the message.

> from a non-IETF effort, particularly given the scale of success of that 

Nor, apparently, are you inclined to cite that established basis.
Perhaps it is because it is not applicable?

> Let's try to stay on the current topic.  

Yes, lets.  Why don't you actually address these technical criticisms
of your proposal rather than disingenenously pretending that they don't 
apply?

> increment in functionality and is based on extensive prior experience and 

Dave, unless you can actually cite relevant experience in the field of
multi-hop, store-and-forward multimedia messaging, I see no reason why we 
should not conclude that you have no basis whatsoever for claiming that your 
current proposal is worth any more consideration.  

Keith

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Mon, 15 Jul 2002 00:06:57 -0400

At 06:50 +0900 on 07/14/2002, Dave Crocker wrote about Re: Keywords 

>downgrading is a new or unusual requirement.

?

There are ways of delaying the downgrading until the recipient has 
actually received the content (and thus gets the message they would 
have received if they were the sole addressee). One that is obvious 
is the use of Multi-Part/Alternative. The sender creates the 
"downgraded" versions that are acceptable/required and hands the full 
set off to EACH OF THE RECIPIENTS. Each one then selects the version 
THEY WANT from the supplied versions without any heavy-handed LCD 
interference by an intermediary just due to one of the addressees 
being unwilling/unable to accept one of the "Better" versions.

I differentiate this from the type of on-the-fly downgrading that 
occurs with 8-Bit->QP transforms during relay since the original 
message CONTENT gets delivered even after the transform (so long as 
the receiving MUA understands QP) unlike the LCD transform where the 
original content is replaced with a lower quality LCD version.

---

From: [email-masked]
Date: Mon, 15 Jul 2002 01:39:06 -0700 (PDT)

> doing conversion.

What's a stretch is your assumption that caching implies conversion
capabilities. They are completely different activities. Conversion by an
intermediary is a nasty mess, whereas caching conneg information is trivial.

> It's also a stretch to believe that this is going to work well
> messages.

That's certainly a possible outcome. But you can game this in all sorts of ways
and reach various different conclusions.

> And of course there's the problem that CONNEG information can be
> to timeout caches.

I've already indicated that I believe conneg information caches are something
that needs to be discussed.

> Frankly the method of returning capabilities in a bounce

The problem I see with depending on notifications is another one of those
"looks good on paper" things: For whatever reason, support for sophisticated
handling of notifications has been slow to materialize in user agents.

> MTAs to support CONNEG in proxy mode.

This scheme was actually discussed and I believe rejected in the FAX WG
some time ago. Regardless, I disagree with your assessment that this is
a requirement for the present proposal to be effective.

> I just don't see a way to make that work through the existing SMTP infrastructure.

I'm sure there are plenty of places in the present infrastructure where this
scheme won't work well. I'm also sure there are plenty of places where it will.
My assessment of the notification scheme is roughly the same. And experience
should inform us that our ability to predict what portions of the
infrastructure will be upgraded is pretty limited.

The bottom line is that we're dealing with partial solutions in every case
here. This makes it easy to criticize any given scheme. IMO this also makes it
very much the WG's call as to how to proceed.

				Ned

---

From: Keith Moore <[email-masked]>
Date: Mon, 15 Jul 2002 08:45:37 -0400

Regarding caching: I see no problem with MTAs that obtain conneg strings 
from a database caching those lookups.  OTOH I think caching is extremely 
dubious as a mechanism for propagating conneg strings in the direction
from the recipient's MTA to the sender's UA, at the rate of one hop per 
email message transmitted.

> very much the WG's call as to how to proceed.

I think this is quite a reasonable statement if we are talking about a
fax-specific extension; less reasonable if we are talking about a generic
SMTP extension.

And regardless, I think more discussion is needed in the document about 
which parties have the responsibility for conversion (presumably the
sender's UA), and which parties have responsibility for providing
recipient information (presumably *all* of the listed MXes for that domain) -
this in addition to addressing several other issues that have been raised.

Keith

---

From: Dave Crocker <[email-masked]>
Date: Tue, 16 Jul 2002 16:15:12 +0900

At 03:06 PM 7/14/2002 +0100, Graham Klyne wrote:
>think your last sentence carries the intent if not the desired precision 
>-- maybe:
>[[
>"fax-specific" refers to email enhancements that permit email to 
>negotiate, transmit and confirm the transfer of images in timely fashion, 
>like traditional fax.

My use of the T.30 term was intentional.  I believe it is precise, complete 
and correct.  It is THE technical definition of the world's use of the term 
"facsimile".

Or at least, that is my understanding.  If I am wrong, we should get a 
precise, complete, technical definition that IS correct from the folks who 
know such things.

In other words, my intent was to use "fax" as a technical term, rather than 
a more general term describing a concept or service.  It's not that the 
latter is irrelevant, but rather that it invites fuzziness and debate.

>that such services behave in a fashion more like ordinary email than fax.

Well, no.  They negotiate content, during the T.30 phase of communication. 
And that phase is, of course, strictly one addressee per copy of the content.

>>>- LCD
>>
>>Least Common Denominator.  So, we are borrowing a term from math, to 
>>apply to our discussion of ESMTP/CONNEG, it is possible to receive CONNEG 
>>capabilities, what capabilities should the sender use?  One possibility 
>>is to end the current SMTP session and then send each recipient the best 
>>"intersection" of capabilities for all the recipients and send this 
>>"least common denominator" or "most common format" to all of the 
>>recipients at once.
>

I agree but dislike those terms.  They sound yucky.

(Or perhaps it will be more constructive for me to note that they are not 
in common use and hence will have no intuitive appeal.)

Can someone suggest a term that is more palatable?

>I think there's a more fundamental architectural issue that is exploited 
>by traditional fax operating in a non-relay mode:  normal operation 
>requires that the sender can receive some indication from the receiver 
>before it starts to send any actual data.

You are saying that this is different from getting a 250 in response to an 
RCPT?

How is it different?

d/

----------

---

From: "MAEDA Toru" <[email-masked]>
Date: Wed, 17 Jul 2002 11:05:36 +0200

Hi Dave-san

> and correct.  It is THE technical definition of the world's use of the term

ITU-T T.30 defines only the protocol for Group 3 facsimile.
In general, there are many Group 3 facsimie and some Group 4.

It is very hard to implement traditional Group 3 facsimle protocol
over E-mail environment, realizing capability negotiation and 
confirmation same level as fax.

Regards.

Toru Maeda

-- 
***********************************
MAEDA TORU
Canon Research Centre France S.A.S.
***********************************

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Wed, 17 Jul 2002 22:46:16 -0400

At 16:15 +0900 on 07/16/2002, Dave Crocker wrote about Re: Keywords 

>And that phase is, of course, strictly one addressee per copy of the content.

To evaluate this comparison you have to look at the equivalent 
situations. The Emailing of a multi-recipient Fax Message is most 
closely equivalent to using Fax Machine that has an Address book that 
can assign a delivery list to an entry (IOW: Select this entry and it 
will dial each phone number in order and deliver a copy of the fax) 
and will read and cache/buffer the full fax into memory so it can 
send from the memory instead of reading/feeding the pages in real 
time [as it is transmitting]. Yes - it will negotiate the content 
(Speed/Resolution) separately for each connection but then that is 
what is supported to be happening with a multi-recipient Fax Message.

---

From: [email-masked]
Date: Wed, 17 Jul 2002 20:15:06 -0700 (PDT)

> At 01:39 -0700 on 07/15/2002, [email-masked] wrote about Re:
> Keywords for "SMTP Service Extension for Content Ne:

> Agreed. OTOH, since the client is going to have to handle CONNEG in
> any case, adding a notification parser to the mix as part of the
> CONNEG implementation is a different issue since it is not adding the
> notification parser to existent support but adding new support that
> INCLUDES a notification parser as part of its implementation
> requirements. Note that I am not advocating or rejecting a
> notification parser solution but only pointing out what I perceive as
> a fallacy in your analysis of the issue.

I'm not entirely clear on what you're driving at here, but regardless: My point
was and is that resistance against doing certain things in certain ways can be
a lot higher than you'd ever expect it to be, to the point where things just
don't get implemented or deployed.

There are all sorts of good reasons to parse DSNs. But despite the multiple
advantages this capability provides it hasn't materialized in a lot of places
where you'd expect it to. Indeed, what has happened in some of those cases is
that some quite extraordinary kludges have been adopted instead.

I'm guilty of this myself to some extent, I guess: MIXER-compliant X.400
gateways require DSN parsing, but back when I coded such things this capability
was the absolute last thing I added.

I still cannot explain the reluctance. This just isn't rocket science.

				Ned

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Wed, 17 Jul 2002 22:35:21 -0400