Subject: [ietf-smtp] Adding a new value to the WITH protocol type subregistry

From: Alexey Melnikov <[email-masked]>
Date: Thu, 12 Jul 2018 16:58:22 +0100

Hi,

I would like to add a new value to the "WITH protocol types" subregistry
(see
<https://www.iana.org/assignments/mail-parameters/mail-parameters.xhtml#mail-parameters-5>)
which is not going to be specified in an IETF Stream RFC. The current
registration policy is defined in RFC 5321 as "Standards Action or
IESG-Approved Experimental RFC". Is the registration policy too strict
and should it be relaxed to allow something like "Specification
required" choice, so that registrations can be added by, say,
Independent Stream RFCs?

Best Regards,
Alexey

---

From: [email-masked]
Date: Thu, 12 Jul 2018 12:38:18 -0400

> I would like to add a new value to the "WITH protocol types" subregistry

What would this value correspond to?  In other words, what's the protocol
you're trying to add, and how is it different from the already extant values?

> required" choice, so that registrations can be added by, say,

I think this is a non-starter.  You're going to need a standards-track RFC
similar to what RFC4356 does for Internet<->MMS gatewaying to get
any real traction.

Plus, having the draft go through the standards process has a high likelyhood
of people pointing out problems and potential improvements in your protocol :)

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 12 Jul 2018 17:48:17 +0100

Hi,

> you're trying to add, and how is it different from the already extant values?

It is an alternative to SMTP for some HF environments. It can be
gatewayed to and from SMTP though, so it would be good to be able to
have a token in the Received line to signal that a particular message
was not transported over SMTP at some point. See
draft-melnikov-email-over-pmul-05.txt

> of people pointing out problems and potential improvements in your protocol :)

For this there must be enough interest in discussing it. The protocol is
rather specialized and I don't think there would be enough interest. So
it is a perfectly reasonable to publish it as an Independent Stream RFC.

Best Regards,
Alexey

---

From: "John Levine" <[email-masked]>
Date: 12 Jul 2018 13:32:52 -0400

>required" choice, so that registrations can be added by, say,

Given that keywords are cheap and the namespace is effectively
unlimited I agee that specification required makes more sense.

Since we are not the Internet Police, people will invent stuff anyway
so we should make the bar as low as possible to avoid keyword
collisions.

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Thu, 12 Jul 2018 10:44:44 -0700

> unlimited I agee that specification required makes more sense.

I'm inclined to agree.

I don't recall why the original requirement was made so stringent.  Does 
anyone else?

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: [email-masked]
Date: Thu, 12 Jul 2018 14:34:27 -0400

> I don't recall why the original requirement was made so stringent.  Does

If I remember correctly, it was due to fears of people tossing a poorly
written Informational RFC, saying "register my entry kthxbai " and
evaporating...

---

From: John C Klensin <[email-masked]>
Date: Thu, 12 Jul 2018 14:41:03 -0400

--On Thursday, July 12, 2018 16:58 +0100 Alexey Melnikov
<[email-masked]> wrote:

> Hi,
> 
> I would like to add a new value to the "WITH protocol types"
> subregistry (see
> <https://www.iana.org/assignments/mail-parameters/mail-paramet
> ers.xhtml#mail-parameters-5>) which is not going to be
> specified in an IETF Stream RFC. The current registration
> policy is defined in RFC 5321 as "Standards Action or
> IESG-Approved Experimental RFC". Is the registration policy
> too strict and should it be relaxed to allow something like
> "Specification required" choice, so that registrations can be
> added by, say, Independent Stream RFCs?

Alexey,

Valdis's concerns aside (personally, I would not want to see a
"with" value in the registry unless it could point to a
well-documented and publicly available specification of the
gateway, not just the "SMTP over..." specification), I see two
nearly-procedural issues with your question.

First, we cannot just agree to make this type of change to the
registry requirements.  It would have to be done via a
standards-track update to RFC 5321.  Just my opinion (although
based on experience), but I believe that embarking on such an
update would create some pressure to review the errata file and
update 5321 to fix other issues that have been identified along
the way (including addressing the ""@domain issue in 5321 and
5322 and possibly some other areas of overlap) and would also
bring people out of the woodwork who would say "if you are
updating 5321, I have the final ultimate solution (FUS) to XYZ
that should be included in the update".   

It is not clear to me that there is sufficient energy to open
those cans of worms.   Speaking personally and as editor of
5321, I could find the energy for the first, although I would
probably push for a complete 5321 update/replacement at the same
time (actually less work for me given the way I've kept notes on
errata) but don't have much energy for dealing carefully and
respectfully with trolls, purveyors of FUS ideas, and people
with bright ideas who don't understand Internet mail well enough
to put those ideas in context.

Second, your question is, through little fault of your own,
badly-timed.  The IESG has an appeal in front of it because the
IESG concluded that a document, processed as an Independent
Submission, should not be published because it interacted with
IETF work, despite the IANA Considerations sections of the
relevant documents not calling for IETF or IESG review of any
sort.  My understanding at this point (and I am thoroughly
confused about the IESG's reasoning, so this may not be correct)
is that the reasoning was that the extension implied by the
specification in that document could change the character of the
underlying standard (or non-IETF work) sufficiently that its
publication without IETF review and action was inappropriate.
Against that backdrop, an attempt to shift anything that the
community concluded, in a standards-track document that is
rather clear about it, required IETF review and action --
presumably at least for clarity, completeness, and
appropriateness -- into the Independent Stream seems wildly
unwise, at least until the IESG responds to the appeal by
clarifying where the boundary of what is acceptable for
Independent Stream publication lies and that the boundary isn't
just about excuses to try to suppress documents some or all IESG
members don't like (the latter would almost certainly lead to
more appeals or serious unpleasantness).

So my suggestion is that you put this question aside until at
least after the situation that caused the appeal is clarified
and, if the outcome of the RFC++ BOF is, as some of the
suggestions on that mailing list have suggested, that the
Independent Stream be discontinued, exported, or changed in
other significant ways, where that discussion is headed.

In that context, I wonder if the Independent Stream would look
equally attractive for this purpose if its documents were not in
the RFC Series.  I also whether the IESG is prepared to launch
work to update every document whose IANA Considerations says
"Informational or Experimental RFC", "including Independent
Stream publications", or the like to allow for new labels and
new terminology?   

best,
   john

---

From: "John R Levine" <[email-masked]>
Date: 12 Jul 2018 14:49:34 -0400

> written Informational RFC, saying "register my entry kthxbai " and

I believe it, but so what?  WITH KTHXBAI is now syntactically valid.  The 
world will not end.

R's,
John

PS: I guess I should register WITH BLOCKCHAIN

---

From: S Moonesamy <[email-masked]>
Date: Thu, 12 Jul 2018 13:11:29 -0700

Hi Alexey,
>required" choice, so that registrations can be added by, say,

A change to the sub-registry policy would require an "Updates" to RFC 
5321.  The registration policy could be considered as too strict if 
there are people unable to go through the process because of IETF 
"barriers".  As a comment about a specification in the Independent 
Submissions Stream, the registration request will still require IESG approval.

Regards,
S. Moonesamy

---

From: John C Klensin <[email-masked]>
Date: Thu, 12 Jul 2018 16:20:52 -0400

--On Thursday, July 12, 2018 14:34 -0400 [email-masked]
wrote:

>> I don't recall why the original requirement was made so
>> stringent.  Does anyone else?
> 
> If I remember correctly, it was due to fears of people tossing
> a poorly written Informational RFC, saying "register my entry
> kthxbai " and evaporating...

My memory is quite vague, but I don't think that is it.  A
completely unused string, or one that is unused except by the
systems of a defunct vendor, is pretty harmless and that has
been more or less the argument for very permissive registration
procedures across the board (and consistent with John L's and
Dave's comments).

There were, IIR, two major concerns.  One was to get at least a
clarity and minimum technical review on gateway specs because
real problems could be caused, on both sides of the gateway by
sloppy work, security-, content-type-, or identity-related
assertions that didn't hold up, etc.  Those issues would
obviously not be a problem for the keyword itself, but, because
we've published very few proprietary (or even public, but
non-SMPT) mail transport specs in the RFC Series, the gateways
seemed to be a way to get a handle on the problem, references to
the transport specs, etc.  Would some people ignore the rules
and simply use their own?  Sure, but that would give anyone who
cared about "with" a possible clue about trust, which would not
be a bad thing.

The other, perhaps less important, is that we didn't want to see
MTA brand names and the like in "with", lest it become
completely useless and we were pretty sure that, if someone
wrote a proposal for FubarMail, which claimed to be an
SMTP-confirming MTA in its documentation and literature to use
"with FubarMail", the IETF would make short work of them.

For the latter, the Independent Submission process, if it worked
no worse than it does today, would almost certainly be adequate.
For the former, it depends on who you trust, which goes back to
parts of my previous long note.

And, of course, if we propose to modify 5321 to allow lighter
weight review, someone is likely to ask for a careful review of
what "with" is actually used for now (as distinct from whatever
was assumed in 2008 or even in 1982 (it might even be me).   If
it has fallen into disuse and anyone wants to move SMTP to
Internet Standard at some point, the rules would probably argue
more strongly for moving the keyword to some sort of "obsolete"
list (paralleling 5322) than for fine-tuning the registry (and
maybe for making the registry requirement "specification
required" or "FCFS").

    john

---

From: Dave Crocker <[email-masked]>
Date: Thu, 12 Jul 2018 13:29:36 -0700

> cared about "with" a possible clue about trust, which would not

So, let's wander down this path a bit.  It sounds plausible and, more 
importantly, I think it summarizes a style of thinking about 
registrations that has been common in the IETF (including mine, fwiw.)

Over time, my own views have changed.  Where I currently sit is that we 
need to avoid conflating 'registration' with 'quality assessment'.

It makes sense to merge them when there is serious danger in registering 
use of a bad spec, but I believe those cases are extremely rare.

My view for registration is that it typically needs to accomplish two 
things:

    1. Reserve a name, to avoid collisions

    2. Identify an accountable entity for use of the name, typically 
through an accessible document controlled by that entity.

For something like the WITH parameter, I think the issue is being able 
to track down the defined meaning of the parameter, rather than trying 
to make sure that conduct a quality assurance process on the 
specification of that meaning.  This latter task is what standardization 
is for, not registration.

d/

---

From: "John R Levine" <[email-masked]>
Date: 12 Jul 2018 16:44:05 -0400

On Thu, 12 Jul 2018, John C Klensin wrote:
> it has fallen into disuse ...

No problem there, it's quite useful.  MTAs reliably put ESMTP or ESMTPS or 
ESMTPSA in the WITH clause so you can tell whether a hop used TLS or was 
authenticated.  When explaining to people that yes, they really did send 
the spam I complained about, WITH ESMTPSA is a smoking gun to tell them 
that they have a compromised user account.

I see a trickle of WITH UTF8SMTPS tellimg me about EAI flagged mail.

Gmail puts WITH HTTP on messages coming from webmail which is wrong but 
fairly benign.

Microsoft of course gets it wrong, viz. these headers in a recent message

Received: from FLLSPAC.flls.org (10.1.1.11) by exchange.flls.org (10.1.1.28)
  with Microsoft SMTP Server id 14.3.399.0; Wed, 11 Jul 2018 13:01:47 -0400
Received: from FLLSPRO ([10.1.1.10]) by FLLSPAC.flls.org with Microsoft
  SMTPSVC(8.5.9600.16384);        Wed, 11 Jul 2018 13:01:47 -0400

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Brandon Long <[email-masked]>
Date: Thu, 12 Jul 2018 15:07:03 -0700

> Gmail puts WITH HTTP on messages coming from webmail which is wrong but

Yeah, I think that was copied from somewhere like Y!Mail back in the day.

I remember getting into an argument with someone who complained about it.
In general,
we lie and say SMTP or SMTPS when in reality all Received/X-Received
headers between
internal servers have used our own rpc mechanism.

Which goes to the other thing, if you make this that strict, people will
just drop it or lie.  You're not
going to review every internal "gateway" protocol.

And using that value for whether or not the message itself is 6532 or 5322
seems particularly bad, but
I know we disagreed about that one before.

Brandon

---

From: John C Klensin <[email-masked]>
Date: Thu, 12 Jul 2018 18:19:51 -0400

Good to know.
thanks
   john

--On Thursday, July 12, 2018 16:44 -0400 John R Levine
<[email-masked]> wrote:

>> weight review, someone is likely to ask for a careful review
>> of what "with" is actually used for now (as distinct from
>> whatever was assumed in 2008 or even in 1982 (it might even
>> be me).   If it has fallen into disuse ...
> 
> No problem there, it's quite useful.  MTAs reliably put ESMTP
> or ESMTPS or ESMTPSA in the WITH clause so you can tell
> whether a hop used TLS or was authenticated.  When explaining
> to people that yes, they really did send the spam I complained
> about, WITH ESMTPSA is a smoking gun to tell them that they
> have a compromised user account.
> 
> Gmail puts WITH HTTP on messages coming from webmail which is
> wrong but fairly benign.
>

---

From: John C Klensin <[email-masked]>
Date: Thu, 12 Jul 2018 18:40:01 -0400

--On Thursday, July 12, 2018 13:29 -0700 Dave Crocker
<[email-masked]> wrote:

>> There were, IIR, two major concerns.  One was to get at least
>> a clarity and minimum technical review on gateway specs
>> because real problems could be caused, on both sides of the
>> gateway by sloppy work, security-, content-type-, or
>> identity-related assertions that didn't hold up, etc.  Those
>> issues would obviously not be a problem for the keyword
>> itself, but, because we've published very few proprietary (or
>> even public, but non-SMPT) mail transport specs in the RFC
>> Series, the gateways seemed to be a way to get a handle on
>> the problem, references to the transport specs, etc.  Would
>> some people ignore the rules and simply use their own?  Sure,
>> but that would give anyone who cared about "with" a possible
>> clue about trust, which would not be a bad thing.
> 
> 
> So, let's wander down this path a bit.  It sounds plausible
> and, more importantly, I think it summarizes a style of
> thinking about registrations that has been common in the IETF
> (including mine, fwiw.)

Indeed.

> Over time, my own views have changed.  Where I currently sit
> is that we need to avoid conflating 'registration' with
> 'quality assessment'.

Yes, as have the views of several of us.  IIR, the first
document that changed a registry from strong IETF approval
requirement was the one for media types in March 1994 (RFC
1590), which gives partial credit to me, so I've been singing
that song for a rather long time.

While I think most registries that do not involve a small range
of possible values should work that way, I believe that "most"
and "small" may involve multiple issues that should be discussed
on a case by case basis.   Saying that "capture the value and a
contact and move on" or "give an opportunity for community
review but don't prevent registration if the applicant declines
that or does not agree with the community comments" should be
the default for those cases seems fine too, but I still believe
the registration procedures and issues should be looked at one
protocol at a time. 
 
> It makes sense to merge them when there is serious danger in
> registering use of a bad spec, but I believe those cases are
> extremely rare.
> 
> My view for registration is that it typically needs to
> accomplish two things:
> 
>     2. Identify an accountable entity for use of the name,
> typically through an accessible document controlled by that
> entity.

I note that, unless I am misinterpreted the trend, the IESG has
started to prefer "stable document" over "accountable entity",
especially if that entity is an individual.   Again, I think
that is a fine default but that, when we create registries or
revise the rules for them, we should be prepared to hear
arguments that more restrictive rules are appropriate and to
consider them carefully.

> For something like the WITH parameter, I think the issue is
> being able to track down the defined meaning of the parameter,
> rather than trying to make sure that conduct a quality
> assurance process on the specification of that meaning.  This
> latter task is what standardization is for, not registration.

An entirely reasonable position, but one that I hope can either
be supported or challenged and discussed when and if there is a
specific proposal to revise RFC 5321 to change the requirement.

best,
    john

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 12 Jul 2018 17:12:51 -0700

> On Jul 12, 2018, at 3:07 PM, Brandon Long <blong=[email-masked]> wrote:
> 
> I remember getting into an argument with someone who complained about it.  In general,
> we lie and say SMTP or SMTPS when in reality all Received/X-Received headers between 
> Which goes to the other thing, if you make this that strict, people will just drop it or lie.  You're not 

This is timely, in that I just dealt with a customer complaint at work (Hushmail), where the recipient parsed the Received headers and noticed no indication of TLS on the delivery step to our border incoming MTAs.

That's because we have an L7 proxy in the way; it accepts the (TLS) connection, does some preliminary validation (greylisting, RBL checks, etc) before passing the connection through to our internal MTAs.  It doesn't insert its own Received header, and the internal proxy->MTA hop isn't encrypted.  The same thing happens on the Submission path, where our internal encryption proxies don't identify themselves via Received (for privacy reasons).

Given the proliferation of proxies, content filters, and other muddle-boxes (sic -- I will let that typo stand!), I wonder how useful the various annotations in Received actually are these days.  Most of the time, the only useful information is the client's network address, and the timestamp.  Anything else I need comes from our internal logs.  Client address + timestamp + internal opaque identifier seems to be all that's really needed these days for Received to provide useful auditing.

--lyndon

---

From: Michael Peddemors <[email-masked]>
Date: Thu, 12 Jul 2018 17:21:02 -0700

I would argue that the information is a lot more important than you 
think.. standards help.. (often the methods of authentication, or the 
way the email was processing through 'muddle-boxes' helps in trust 
factors and reputation...

But I for one agree, that allowing a more 'open' set of identifiers and 
mappings of what those are, aid in transparency, and should not be 
restricted to 'approved' identifiers (eg IETF approved) , but 
'published' identifiers..

On 18-07-12 05:12 PM, Lyndon Nerenberg wrote:
> _______________________________________________
> ietf-smtp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-smtp
> 

-- 
"Catch the Magic of Linux..."
------------------------------------------------------------------------
Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.
------------------------------------------------------------------------
604-682-0300 Beautiful British Columbia, Canada

This email and any electronic data contained are confidential and intended
solely for the use of the individual or entity to which they are addressed.
Please note that any views or opinions presented in this email are solely
those of the author and are not intended to represent those of the company.

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 12 Jul 2018 17:21:46 -0700

> On Jul 12, 2018, at 5:12 PM, Lyndon Nerenberg <[email-masked]> wrote:
> 

Sorry, I left out some context ...

Our internal MTAs report no TLS because the inside link is not encrypted.  This means incoming TLS-encrypted messages don't show that in Received.

If we were to encrypt the internal hop from the proxy to the MTA, *all* the messages would show as TLS-encrypted, even when they might not be.

The alternative is modifying the proxy code to insert its own Received header.  That would add a lot of overhead to what is otherwise a very lightweight front end defence on our SMTP servers.  The value of having this information in Received is negligible, versus having it in the logs, which is where we go to anyway whenever anyone askes how the transaction actually played out.

--lyndon

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 12 Jul 2018 17:29:20 -0700

> On Jul 12, 2018, at 5:21 PM, Michael Peddemors <[email-masked]> wrote:
> 
> I would argue that the information is a lot more important than you think.. standards help.. (often the methods of authentication, or the way the email was processing through 'muddle-boxes' helps in trust factors and reputation...

There are subtleties.  E.g., for our submission service, we should be including the IP address of the client in Received.  Yet many of our customers don't want that information exposed.  And in fact we advertise that we don't expose that data.

That helps protect our customer's privacy.  But we still have the audit information in our internal logs, should the need arise.  That could be for customer support reasons (why didn't my mail get delivered?), or for legal reasons (we always honour orders enforceable in a BC court).

--lyndon

---

From: "John R Levine" <[email-masked]>
Date: 12 Jul 2018 22:29:16 -0400

> Given the proliferation of proxies, content filters, and other muddle-boxes (sic -- I will let that typo stand!), I wonder how useful the various annotations in Received actually are these days.  M…

It's useful on hops where the two ends are under different control. e.g, 
user MUA to system MTA, or one system to the MX of another.  I agree that 
hops with a system all under the same control aren't very interesting.

Regards,

---

From: Ned Freed <[email-masked]>
Date: Fri, 13 Jul 2018 09:48:23 -0700 (PDT)

> Our internal MTAs report no TLS because the inside link is not encrypted. 
> This means incoming TLS-encrypted messages don't show that in Received.

> If we were to encrypt the internal hop from the proxy to the MTA, *all* the
> messages would show as TLS-encrypted, even when they might not be.

> The alternative is modifying the proxy code to insert its own Received
> header.  That would add a lot of overhead to what is otherwise a very
> lightweight front end defence on our SMTP servers.  The value of having this
> information in Received is negligible, versus having it in the logs, which is
> where we go to anyway whenever anyone askes how the transaction actually played
> out.

Or better still, use XCLIENT. This is it's main use-case.

    http://www.postfix.org/XCLIENT_README.html

				Ned

---

From: "John Levine" <[email-masked]>
Date: 13 Jul 2018 14:11:14 -0400

>> lightweight front end defence on our SMTP servers.

I don't get it.  The proxy gathers the info that it is putting into
the logs anyway and stuffs a Received: line to the next stage ahead of
the body of the message.  This doesn't seem like a lot of work.  It
should be able to do that before reading the body of the message from
the incoming connection so it doesn't need any extra buffering.

What am I missing?  Or are you saying the proxy passes a pointer to
the socket rather than the contents?

>    http://www.postfix.org/XCLIENT_README.html

Seems reasonable.  Are there descriptions of it beyond what's in that
web page?

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Fri, 13 Jul 2018 16:15:04 -0700 (PDT)

> the incoming connection so it doesn't need any extra buffering.

Given it's apparent simplicity it's surprising how often it's miscoded.

And that's just DATA. BDAT adds some additional complexity.

> the socket rather than the contents?

Quite a few proxies turn themselves into tunnels at some point in the protocol.

> Seems reasonable.  Are there descriptions of it beyond what's in that

Not that I'm aware of. Writing an I-D describing it is on my to-do list, but
I simply haven't had the time.

				Ned

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 14 Jul 2018 13:55:43 +0200

On Friday 13 July 2018 20:11:14 CEST, John Levine wrote:
> the body of the message.  This doesn't seem like a lot of work. 

It might not be desirable work, even if simple in a particular 
implementation. Your server serves you. It should be loyal to you. If you 
don't want to tell your correspondents where you are when you send mail, 
it's far from clear that your server should reveal anything about that, 
even just an IP address.

Arnt