Subject: Re: bogus SPF deployment survey
From: [email-masked]
To: Ned Freed <[email-masked]>
Cc: Alessandro Vesely <[email-masked]>, [email-masked], [email-masked]
Date: Mon, 30 Apr 2012 14:55:31 -0700 (PDT)
List-Id: <ietf-smtp.imc.org>

> Similarly, I am constantly receiving and assessing information from customers
> about how well things work as well as what additional capabilities they want.
> This is how I decide what features to add, remove, or enhance.

Sorry, that should have been "receiving and assessing information from
customers about problems they are having as well as what additional
capabilities they want."

				Ned

From: [email-masked]
Cc: [email-masked], [email-masked]
Date: Mon, 30 Apr 2012 13:45:52 -0700 (PDT)
Subject: Re: bogus SPF deployment survey
To: Alessandro Vesely <[email-masked]>
Sender: [email-masked]

> On Mon 30/Apr/2012 18:24:25 +0200 ned+ietf-smtp wrote:
> >
> > FWIW, I looked at the survey before I saw John's note and decided on my own not
> > to take it. AFAICT it's intended to assess what SPF policy, if any, has been
> > implemented by the responder on their own host(s).

> Correct.

> > I cannot for the life of me understand why anyone would care what SPF policy I
> > use at home (which is the only system I personally control and can speak about
> > with any authority), especially since for an email implementor that policy is
> > more likely to be the result of the need to test things rather than any actual
> > personal need.

> That's deployment experience.  Even if it's a test system or a home
> system, the way you deploy that protocol reveals your personal
> preferences and attitudes toward it.  Since SPF is (or used to be)
> somewhat controversial, I think it would be correct to take note of
> how incompatible expectations are distributed.

I'm sorry, but that's just not true. Like most of the other people on this
list, I'm a developer, and as such I deploy things for reasons having nothing
whatsoever to do with my personal preferences for how an email server should be
run. Additionally, my experience is routinely based on feedback I get from
customems *when they are having problems*. It is a rare customer indeed that
calls up and says, "We deployed such and such and it is working fine".

> IMHO, omitting to take the survey, you --in general, non responders--
> just express lack of interest in SPF.  Am I wrong?

Yes, you are completely wrong.

> > If you want to ask questions of an audience of implementors, I suggest that you
> > ask them what kind of support for SPF do they provide in their products and
> > perhaps what they know (or don't know) about actual customer usage of those
> > features.

> Agreed, if I had been more clever I'd have done that.  But then, those
> who would have taken the burden to answer such more lengthy questions
> are probably participating in SPFBIS already.  My questions are easier
> to ask as well as to answer.

And that's 0 for 3. I am comprehensively well versed in what features our
product has, including the parts of the product I didn't write, because I am
constantly answering complicated questions about how to accomplish various
things using our product's capabilities.

Similarly, I am constantly receiving and assessing information from customers
about how well things work as well as what additional capabilities they want.
This is how I decide what features to add, remove, or enhance.

Since I think about this stuff all the time, it is trivial for me to
regurgitate some subset of it when asked.

But when it comes to direct experience with the machines I use for testing and
personal stuff, I make changes so often that I often don't recall the specifics
of their configuration. (Seriously: How many of the readers of this message
have had the experience where you pulled up some config file, looked, and said,
"What the hell is that doing in there?") And I most certainly do not remember
when I did something. And I rarely bother to assess how well some very
specific functionality is working unless I'm directly concerned with that
functionality at the moment or I've determined it is causing problems.

I can get all this stuff of course, but it's far more effect that talking
about product-related matters.

				Ned

Date: Mon, 30 Apr 2012 19:14:21 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: bogus SPF deployment survey
Sender: [email-masked]

On Mon 30/Apr/2012 18:24:25 +0200 ned+ietf-smtp wrote:
> 
> FWIW, I looked at the survey before I saw John's note and decided on my own not
> to take it. AFAICT it's intended to assess what SPF policy, if any, has been
> implemented by the responder on their own host(s).

Correct.

> I cannot for the life of me understand why anyone would care what SPF policy I
> use at home (which is the only system I personally control and can speak about
> with any authority), especially since for an email implementor that policy is
> more likely to be the result of the need to test things rather than any actual
> personal need.

That's deployment experience.  Even if it's a test system or a home
system, the way you deploy that protocol reveals your personal
preferences and attitudes toward it.  Since SPF is (or used to be)
somewhat controversial, I think it would be correct to take note of
how incompatible expectations are distributed.

IMHO, omitting to take the survey, you --in general, non responders--
just express lack of interest in SPF.  Am I wrong?

> If you want to ask questions of an audience of implementors, I suggest that you
> ask them what kind of support for SPF do they provide in their products and
> perhaps what they know (or don't know) about actual customer usage of those
> features.

Agreed, if I had been more clever I'd have done that.  But then, those
who would have taken the burden to answer such more lengthy questions
are probably participating in SPFBIS already.  My questions are easier
to ask as well as to answer.

Date: Mon, 30 Apr 2012 06:28:17 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: bogus SPF deployment survey
Sender: [email-masked]

[email-masked] wrote:
> 
>> On Fri 27/Apr/2012 20:18:08 +0200 John Levine wrote:
>>> Please save your time and everyone else's by *not* responding to his
>>> message
> 
>> You did :-)
> 
>> How much time are we talking about here?  Thinking on an evolutionary
>> scale, it doesn't seem to be comparable to the time the experiment took.
> 
> ...
>
> If you want to ask questions of an audience of implementors, I suggest that you
> ask them what kind of support for SPF do they provide in their products and
> perhaps what they know (or don't know) about actual customer usage of those
> features.

+1,  It is what I suggested, especially along the lines of 
known/tracked issues, perhaps security related, on the WG table.  I am 
not sure if it means anything, but if the right question were asked, 
it may just help "open the minds" of others who are currently locked 
in certain views.

Consider that supporting SPF comes in many flavors which can as simply 
of adding a Domain Policy records but don't support any technical SPF 
logic in the server or client.  There are many brand domains that 
don't use their domain for email and a simple:

     v=spf1 -all

defines that "NO EMAIL WITH THIS DOMAIN" policy.

altavista.com makes that very clear with their TXT records:

    "v=spf1 +exists:CL.%{i}.FR.%{s}.HE.%{h}.null.spf.altavista.com -all"
    "This domain sends no email"
    "Null SPF is for tracking purposes only"
    "All mail claiming to be from altavista.com is forged"

Hard fail -ALL policies are not a fallacy and a quick scan of capture 
DNS query in my logs should roughly 17%, but it will be different for 
every site what sort of domains it extracts or caters to.

IMV, the survey might include questions related to the RFC4408 
REJECT-ON-FAIL vs MARK-ON-FAIL local SPF deployment option and 
specific to MARK-ON-FAIL, a lack of a technical specification left 
undefined or perhaps left open-ended by design:

    Please describe how MARK-ON-FAIL is implemented in your SPF server:

    [_] Received-SPF: fail is recorded
    [_] Authentication-Result: spf=fail is recorded
    [_] SPF fail mail is quarantined into a "junk email" or similar 
user folder.
    [_] SPF fail mail is bundled with user's MUA POP3 protocol mail 
pickup.

and related to what may be "FAILSAFE" consideration:

    If you publish -ALL hard fail SPF policies, what do you expect
    the SMTP SPF receiver local policy FAIL result actionable
    deployment option to be:

    (_) for fail results, we always expect REJECT-ON-FAIL
    (_) for fail results, we prefer REJECT-ON-FAIL
    (_) for fail results, we always expect MARK-ON-FAIL
    (_) for fail results, we prefer MARK-ON-FAIL
    [_] MARK or REJECT, we expect "negative" stored separation mail 
pass to user.

and strongly related :

    What kind of MUA portal access is allowed on your server?

    [_] Online MUA, i.e. Web Mail, Mobile device, i.e THIN DEVICE
    [_] Offline MUA with POP3 access,
    [_] Mixed Online/Offline MUA with IMAP access
    [_] Other

And so on.

-- 
Sincerely

Hector Santos
http://www.santronics.com
jabber: [email-masked]

Date: Mon, 30 Apr 2012 07:43:17 +0100
From: Paul Smith <[email-masked]>
To: [email-masked]
CC: Alessandro Vesely <[email-masked]>, [email-masked]
Subject: Re: bogus SPF deployment survey
Sender: [email-masked]

On 30/04/2012 07:10, [email-masked] wrote:
> If you want to ask questions of an audience of implementors, I suggest 
> that you ask them what kind of support for SPF do they provide in 
> their products and perhaps what they know (or don't know) about actual 
> customer usage of those features. Ned 

Agreed

For instance, our software checks SPF, and we have SPF records on our 
domain, but a lot of our customers don't have SPF records because 
they're too complex for them to set up unaided (and they don't want to 
spend a few pounds more to get our help). Because we're email 
implementers, we have far more understanding of the email infrastructure 
than our customers, so, obviously, we'll do far more 'cutting edge' things.

Date: Sun, 29 Apr 2012 23:50:26 -0700
From: "Carl S. Gutekunst" <[email-masked]>
To: Alessandro Vesely <[email-masked]>
CC: [email-masked]
Subject: Re: bogus SPF deployment survey
Sender: [email-masked]

[email-masked] wrote:
> FWIW, I looked at the survey before I saw John's note and decided on my own not
> to take it. AFAICT it's intended to assess what SPF policy, if any, has been
> implemented by the responder on their own host(s).
> ...
>
> If you want to ask questions of an audience of implementors, I suggest that you
> ask them what kind of support for SPF do they provide in their products and
> perhaps what they know (or don't know) about actual customer usage of those
> features.

I reached the same conclusion. I have five different mail receivers I 
use on a daily basis. Each handles SPF quite differently, ranging from 
no support to a textbook implementation to a datapoint in a spam scoring 
engine. Two use commercial products I either designed or maintain. I 
would guess Ned and anyone who implements mail handling technology for a 
living has a similar range.

<csg>

From: [email-masked]
Cc: [email-masked]
Date: Sun, 29 Apr 2012 23:10:07 -0700 (PDT)
Subject: Re: bogus SPF deployment survey
To: Alessandro Vesely <[email-masked]>
Sender: [email-masked]

> On Fri 27/Apr/2012 20:18:08 +0200 John Levine wrote:
> >
> > Please save your time and everyone else's by *not* responding to his
> > message

> You did :-)

> How much time are we talking about here?  Thinking on an evolutionary
> scale, it doesn't seem to be comparable to the time the experiment took.

FWIW, I looked at the survey before I saw John's note and decided on my own not
to take it. AFAICT it's intended to assess what SPF policy, if any, has been
implemented by the responder on their own host(s).

I cannot for the life of me understand why anyone would care what SPF policy I
use at home (which is the only system I personally control and can speak about
with any authority), especially since for an email implementor that policy is
more likely to be the result of the need to test things rather than any actual
personal need.

If you want to ask questions of an audience of implementors, I suggest that you
ask them what kind of support for SPF do they provide in their products and
perhaps what they know (or don't know) about actual customer usage of those
features.

				Ned

Date: Fri, 27 Apr 2012 19:10:22 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: IETF SMTP <[email-masked]>
Subject: I-D Action: draft-santos-smtpgrey-02.txt
Sender: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts 
directories.

	Title           : SMTP Service Extension for Greylisting Operations
	Author(s)       : Hector Santos
                           Evan Harris
	Filename        : draft-santos-smtpgrey-02.txt
	Pages           : 20
	Date            : 2012-04-27

    GREYLIST is a SMTP extension to formalize the widely supported
    Greylisting mail filtering method and to help support SMTP rejected
    transports by following a new formal structured 4yz server temporary
    rejection response by including a "retry=time-delay" tag string which
    SMTP clients can use to optimize the rescheduling of the mail
    delivery attempts.  With adoption, network overhead reduction in
    wasteful mail delivery attempts will be realized.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-santos-smtpgrey-02.txt
http://tools.ietf.org/html/draft-santos-smtpgrey-02

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

This Internet-Draft can be retrieved at:
ftp://ftp.ietf.org/internet-drafts/draft-santos-smtpgrey-02.txt

The IETF datatracker page for this Internet-Draft is:
https://datatracker.ietf.org/doc/draft-santos-smtpgrey/

-- 
Sincerely

Hector Santos
http://www.santronics.com
jabber: [email-masked]

Date: Fri, 27 Apr 2012 17:21:22 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: draft-santos-smtpgrey-01.txt
Sender: [email-masked]

Hi,

I am going to be updating this I-D tonight to rev-02.  I am wondering 
if there any implementations of the I-D.  Its been six months since we 
implemented and its been 100% successful in minimizing mail send 
attempts to 2 and the minimizing the 2nd attempt delay to the 
greylisting server recommended wait time value.

Thanks

-- 
Sincerely

Hector Santos
http://www.santronics.com
jabber: [email-masked]

Date: Fri, 27 Apr 2012 20:42:29 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: bogus SPF deployment survey
Sender: [email-masked]

On Fri 27/Apr/2012 20:18:08 +0200 John Levine wrote:
> 
> Please save your time and everyone else's by *not* responding to his
> message

You did :-)

How much time are we talking about here?  Thinking on an evolutionary
scale, it doesn't seem to be comparable to the time the experiment took.

VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org
Date: 27 Apr 2012 12:43:13 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Subject: Re: bogus SPF deployment survey
Organization: 
Sender: [email-masked]

>as you may know, an IETF working group is reviewing the SPF spec.
>The evaluation of current deployment is nearly completed, but thus far
>only analyzes published SPF records.  Please contribute to a better
>understanding by completing the survey:

When Alessandro suggested this survey a few days ago, the spfbis group
roundly rejected it for a variety of reasons.  See the spfbis archives
if you're interested in details.

Please save your time and everyone else's by *not* responding to his
message, and certainly don't send it anywhere else.

R's,
John

Date: Fri, 27 Apr 2012 08:07:33 +0200
From: Alessandro Vesely <[email-masked]>
To: SMTP Interest Group <[email-masked]>
Subject: SPF deployment survey
Sender: [email-masked]

All,
as you may know, an IETF working group is reviewing the SPF spec.
The evaluation of current deployment is nearly completed, but thus far
only analyzes published SPF records.  Please contribute to a better
understanding by completing the survey:

  https://www.surveymonkey.com/s/SPF-deployment-survey

It will only take three minutes (ten questions on a single page).

Please help reaching all mail admins by forwarding this prompt to
specialized mailing lists and mail admins you are in touch with,
worldwide.

Thank you in advance!

From: "Murray S. Kucherawy" <[email-masked]>
To: "[email-masked]" <[email-masked]>, "[email-masked]" <[email-masked]>
Subject: FW: [apps-discuss] Call for Adoption: draft-kucherawy-received-state-02.txt
Date: Tue, 24 Apr 2012 17:44:43 +0000
Accept-Language: en-US
Sender: [email-masked]

Please comment on the apps-discuss list if you have any input.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Alexey Melnikov
Sent: Tuesday, April 24, 2012 10:25 AM
To: [email-masked]
Subject: [apps-discuss] Call for Adoption: draft-kucherawy-received-state-02.txt

Murray presented this draft in at least one of the face to face IETF meetings and my recollection is that general feedback on this document was positive. So I would like to initiate acceptance call for processing this document in the APPSAWG.

Please indicate your support or objection, and opinions thereto before the close of business on May 1st.

Alexey, as an APPSAWG co-chair

_______________________________________________
apps-discuss mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/apps-discuss

Date: Wed, 18 Apr 2012 11:01:40 +0200
From: "Rolf E. Sonneveld" <[email-masked]>
To: Tony Hansen <[email-masked]>
Cc: [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

To get back to the original question:

On 2/28/12 7:17 PM, Tony Hansen wrote:
>
> I actually started writing such a doc about a year ago, but never 
> finished it.
>
> Is it worth dusting off?
>
>     Tony Hansen

which was a reply to:

>
> On 2/28/2012 12:24 PM, Dave Crocker wrote:
>>
>>
>>
>> On 2/28/2012 9:11 AM, Paul Smith wrote:
>>>>
>>> OK, I was wrong - but I've never seen a BCC handled in that way 
>>> before... I've
>>> sometimes received messages with a 'BCC' field containing addresses 
>>> who were NOT
>>> me (ie I shouldn't have known they existed), and most MUAs I've seen 
>>> (in fact,
>>> all those I've investigated fully) just list those recipients in the 
>>> envelope
>>> for a single copy of the message. So, I've never seen a BCC field in 
>>> a message
>>> header, used 'correctly'.
>>
>> We have never really standardized BCC as an end-to-end construct, at 
>> the MUA-MUA level.

and given the long thread of messages from at least ten people that 
followed, I certainly think it would be good if Tony wiped of the dust 
from his document and send a first draft to this list.

/rolf

Date: Tue, 17 Apr 2012 17:59:39 -0400
From: John C Klensin <[email-masked]>
To: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

--On Tuesday, April 17, 2012 15:35 -0400 Hector Santos
<[email-masked]> wrote:

> For example, with the many discussions regarding what an
> offline MUA may display in RFC5322 headers, it was not an
> issue with the Online MUA where the UI display rendering is
> 100% controlled by the backend. We may even defined with a new
> acronym MAS - "Mail Access Server."
> 
> The real distinction is based on how much intelligence the MUA
> had with the capability and advancement of how much can be
> offloaded to the user's device. And FYI, the concept of
> offloading data was patented by IBM long ago, long expired of
> course, which allowed for rapid growth of offline/offloading
> software to be developed.

Hector,

I'm not completely certain I agree.  In the beginning, MUAs were
entirely on host machines and we communicated with them via
terminals or equivalent.  IIR, tools like the original U**x mail
command, its predecessors, and better tools like MH were
developed in/for that kind of environment.  Certainly they were
online, but the concept was a little different from the way we
use the term today.  You, Dave, or others might have different
data, but I first remember the term "split-UA" used for
arrangements in which the work of getting mail to the user was
divided between something happening on the mail store machine
(usually the delivery MTA, but not always) using protocols like
POP, PCMAIL, IMAP, and some arrangements that have passed into
obscurity.  Each of those three originally supported a different
model: "download everything to the client side of the split and
work offline" for POP, "synchronize to the client side of the
split but mostly work offline" for PCMAIL, and "client is online
to the server, often in what we call kiosk mode today" for IMAP.
The issue is less about "how much can be offloaded" than about
styles of working with different situations involving different
mixes of advantages and disadvantages.  (IMAP4 moved beyond that
early split to support all three models -- how well differs by
implementation, but the "split-UA" part of the concept remains.

On the submission side, we had both MTAs that expected to get at
least nearly-adequate SMTP and, among other options,
intermediate MUAs and MTAs that effectively accepted an 822-ish
message body along with an envelope via some API as well as MTAs
that accepted a non-envelope 822-ish message body and tried to
construct an envelope from it.  The latter, IMO, led to lots of
woes, but it was certainly common.     I remember hearing and
talking about those (both originating MUAs/MTAs and gateways)
more in "fix the message before it is injected into the Internet
environment" than in "split-UA" terms although the latter is
certainly a reasonable model for the situation.  Remember that
most "smart" mailing list exploders perform some MUA functions
too.

>From that perspective, our explicitly describing submission
servers and giving them equally explicit permission to do
MUA-ish things simply symmetrically recreates the split-UA
situation we have on the receiving side with POP and IMAP: some
MUA-ish work gets done on the client machine, some gets done on
the server machine, and the server machine also has interfaces
to the network and/or mail store.  In both sets of cases, if the
server side of the UA function starts doing things that the
client side doesn't anticipate, seriously bad things can happen.

    john

Date: Tue, 17 Apr 2012 15:35:41 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

Dave Crocker wrote:
> 
> On 4/17/2012 8:35 AM, John Levine wrote:
>> In case it wasn't clear, the MSA making the envelope from the headers 
>> is reasonable, since you can write down a fairly simple rule for how 
>> it works. 
> 
> I think we have a nomenclature challenge here.  It is natural to refer 
> to this as something the "MSA" is doing.  But it isn't an MSA function.  
> It's an MUA function, as part of preparation for formal posting into the 
> MHS.  The best I can suggest is something more implementation oriented, 
> such as "server".
> 
> The meta-model this implies is to have "agent" mean networking 
> architecture, per the email architecture spec.  And "client" or "server" 
> as references to physical implementations which might have interaction 
> boundaries that are different from the networking architecture boundaries.
> 
> If we don't develop a discipline in how we talk about this kind of 
> distinction, we will never develop a community understanding of the 
> difference.
> 
> The term that was developed a long time ago, for having a server do 
> 'user' functions is "split UA".

+1.

Dave, I always felt that the idea of Online vs Offline MUA was often 
not considered when in came to an discussed expected behavior which 
mostly applied to RFC-based offline MUAs but didn't to Online MUAs.

For example, with the many discussions regarding what an offline MUA 
may display in RFC5322 headers, it was not an issue with the Online 
MUA where the UI display rendering is 100% controlled by the backend. 
We may even defined with a new acronym MAS - "Mail Access Server."

The real distinction is based on how much intelligence the MUA had 
with the capability and advancement of how much can be offloaded to 
the user's device. And FYI, the concept of offloading data was 
patented by IBM long ago, long expired of course, which allowed for 
rapid growth of offline/offloading software to be developed.

-- 
Sincerely

Hector Santos
http://www.santronics.com
jabber: [email-masked]

Date: Tue, 17 Apr 2012 14:11:11 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
CC: [email-masked]
Subject: Re: "proper" handling of BCC
To: [email-masked]
Sender: [email-masked]

Dave Crocker wrote:

> For the scenario being discussed, the "MSA" is actually part MSA and 
> part MUA.

Isn't the MSA also part MDA which the key different is that later does 
not require authentication?

But I guess I also read some people believe the MUA can also be a MDA. 
  But for me, an MUA is already on online agent or offline RFC based 
agent.

To me, the MSA presumed an authenticated transaction and one that also 
authorized the server to do more with this USER's submitted data than 
it can do with a MDA receiving unsolicited passthru mail.

My view.

-- 
Sincerely

Hector Santos
http://www.santronics.com
jabber: [email-masked]

Date: Tue, 17 Apr 2012 10:51:03 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: John Levine <[email-masked]>
CC: [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

On 4/17/2012 8:35 AM, John Levine wrote:
> In case it wasn't clear, the MSA making the envelope from the headers 
> is reasonable, since you can write down a fairly simple rule for how 
> it works. 

I think we have a nomenclature challenge here.  It is natural to refer 
to this as something the "MSA" is doing.  But it isn't an MSA function.  
It's an MUA function, as part of preparation for formal posting into the 
MHS.  The best I can suggest is something more implementation oriented, 
such as "server".

The meta-model this implies is to have "agent" mean networking 
architecture, per the email architecture spec.  And "client" or "server" 
as references to physical implementations which might have interaction 
boundaries that are different from the networking architecture boundaries.

If we don't develop a discipline in how we talk about this kind of 
distinction, we will never develop a community understanding of the 
difference.

The term that was developed a long time ago, for having a server do 
'user' functions is "split UA".

d/

-- 

Dave Crocker
bbiw.net

-- 

d/

--

    Dave Crocker
    bbiw.net

Date: Tue, 17 Apr 2012 12:55:26 -0400
From: John C Klensin <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

FWIW and in case the clarification is needed, I am in complete
agreement with Dave's analysis and comments.

Interestingly, before we formalized the role of MSAs with RFC
2476, I was using almost exactly the same terminology and
distinction about MTAs performing MUA functions that appear in
Dave's note (I believe he has been using it at least equally
long).  Since we started talking about MSAs, I've gotten a bit
more sloppy because both the SMTP and MSA docs assume that they
can perform MUA functions.  This discussion probably illustrates
that original distinction is worth retaining. 

   john

--On Tuesday, April 17, 2012 07:40 -0700 Dave Crocker
<[email-masked]> wrote:

> 
> 
> On 4/16/2012 8:38 AM, John Levine wrote:
>> 
>>> Of course, if one had a submission server that was smart
>>> about these things, the MUA could hand the message,
>>> including the Bcc field, to it and let it sort out envelopes
>>> (which it needs to do anyway) and the warning message.
>> 
>> That strikes me as a really unfortunate idea.  Submission
>> servers know all about envelopes and message headers.  In
>> most cases, they don't know anything about the body of the
>> message beyond what they can intuit from the MIME headers.
> 
> As noted, this is a scenario that has been supported by some
> posting MSAs, like
> sendmail and mmdf, going back a very long time.  My own view
> is that this is,
> nonetheless, an MUA function.  That is, for this feature,
> sendmail and mmdf
> were/are acting as MUAs to prepare the message for formal
> posting, and then they
> turn into MSAs.
> 
> Note that the function is really the same as most/all MUA's
> do, when processing
> a message themselves for regular SMTP/Submission postings:
> parse a number of
> header-fields as sources for addresses to be used in the
> RCPT-TO list.
> 
> This is a nice demonstration of the difference between network
> architecture and
> software architecture, in which different software
> implementation choices can
> lead to placement of network architecture functions in
> different modules.
> 
> For the scenario being discussed, the "MSA" is actually part
> MSA and part MUA.
> 
> 
>> If your submission server adds text to a body saying
>> something like "you are a blind copy recipient", in what
>> language is that message expressed?
> 
> That's a nice example of the basic reason to keep network
> architectures as far
> from user interface design as we can.  (chorus: ) Presentation
> and interaction
> design are specialties that are not well represented in
> protocol development
> groups...
> 
> 
> d/

VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org
Date: 17 Apr 2012 15:35:21 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: "proper" handling of BCC
Organization: 
Sender: [email-masked]

>> That strikes me as a really unfortunate idea.  Submission servers know all
>> about envelopes and message headers.  In most cases, they don't know anything
>> about the body of the message beyond what they can intuit from the MIME
>> headers.
>
>As noted, this is a scenario that has been supported by some posting MSAs, like
>sendmail and mmdf, going back a very long time.  My own view is that this is,
>nonetheless, an MUA function.  That is, for this feature, sendmail and mmdf
>were/are acting as MUAs to prepare the message for formal posting, and then they
>turn into MSAs.

In case it wasn't clear, the MSA making the envelope from the headers
is reasonable, since you can write down a fairly simple rule for how
it works. The MUA is still a better place to do it since it's more
likely to know what the user wanted, and the simple rule becomes much
more complex in corner cases like multiple From: addresses and
Resent-x headers.

Having the MSA add comments into the body is much less so.

R's,
John

Date: Tue, 17 Apr 2012 07:40:44 -0700
From: Dave Crocker <[email-masked]>
Reply-To: [email-masked]
Organization: Brandenburg InternetWorking
To: [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

On 4/16/2012 8:38 AM, John Levine wrote:
>
>> Of course, if one had a submission server that was smart about these
>> things, the MUA could hand the message, including the Bcc field, to it and
>> let it sort out envelopes (which it needs to do anyway) and the warning
>> message.
>
> That strikes me as a really unfortunate idea.  Submission servers know all
> about envelopes and message headers.  In most cases, they don't know anything
> about the body of the message beyond what they can intuit from the MIME
> headers.

As noted, this is a scenario that has been supported by some posting MSAs, like
sendmail and mmdf, going back a very long time.  My own view is that this is,
nonetheless, an MUA function.  That is, for this feature, sendmail and mmdf
were/are acting as MUAs to prepare the message for formal posting, and then they
turn into MSAs.

Note that the function is really the same as most/all MUA's do, when processing
a message themselves for regular SMTP/Submission postings:  parse a number of
header-fields as sources for addresses to be used in the RCPT-TO list.

This is a nice demonstration of the difference between network architecture and
software architecture, in which different software implementation choices can
lead to placement of network architecture functions in different modules.

For the scenario being discussed, the "MSA" is actually part MSA and part MUA.

> If your submission server adds text to a body saying something like "you are
> a blind copy recipient", in what language is that message expressed?

That's a nice example of the basic reason to keep network architectures as far
from user interface design as we can.  (chorus: ) Presentation and interaction
design are specialties that are not well represented in protocol development
groups...

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

Date: Tue, 17 Apr 2012 16:34:48 +0200
From: Alessandro Vesely <[email-masked]>
To: [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

On Tue 17/Apr/2012 16:26:49 +0200 John Levine wrote:
> 
> I'd rather move toward MUAs creating correct messages, and having
> submission server fixups as a backup.

+1, and if a MUA is so smart as to add such functionality, why does it have
to hijack a "Bcc" field which is already used for different purposes?

VBR-Info: md=iecc.com; mc=all; mv=dwl.spamhaus.org
Date: 16 Apr 2012 15:38:13 -0000
From: "John Levine" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: "proper" handling of BCC
Organization: 
Sender: [email-masked]

> Of course, if one had a submission server that was smart about these
> things, the MUA could hand the message, including the Bcc field, to
> it and let it sort out envelopes (which it needs to do anyway) and
> the warning message.

That strikes me as a really unfortunate idea.  Submission servers know
all about envelopes and message headers.  In most cases, they don't
know anything about the body of the message beyond what they can
intuit from the MIME headers.

If your submission server adds text to a body saying something like
"you are a blind copy recipient", in what language is that message
expressed?  If you'd rather it use another language, how can you tell
it to do so?  This all belongs in the MUA.

While there is plenty of historical precedent for submission servers
fixing up headers, adding missing ones that can be constructed
mechanically, and creating an envelope based on recipient addresses in
the headers, I can't think of fixups they do to message bodies other
than perhaps wrapping the whole unmodified body in a crypto signature.
I'd rather move toward MUAs creating correct messages, and having
submission server fixups as a backup.

R's,
John

Date: Mon, 16 Apr 2012 03:01:29 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: John C Klensin <[email-masked]>
CC: Steve Atkins <[email-masked]>, [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

John C Klensin wrote:
> 
>> Handling Bcc by copying the Bcc header to the envelope,
>> then deleting the header is fine, though I wouldn't trust any
>> MUA that relied on that functionality.
> 
> Yes.  The difficulty with this --and what caused the "add the
> comment/warning" behavior-- is that the recipients cannot easily
> tell "received a blind copy" from "received from a mailing
> list".  If they then reply, copying the explicit distribution,
> the result is a privacy compromise relative to the intentions of
> the original sender.  If the original sender were really worried
> about that, the only safe action I know of is to avoid use of
> bcc entirely, instead, for example, forwarding a copy of the
> message with the explicit recipient listed to the bcc recipient
> only.  That message could well be structured with the warning as
> a first body part followed by the original as message/rfc822 or
> message/global.  But it is a little hard to automate.

When I don't use our MUA which have the special BCC feature to split 
the streams and add the special top posted notification, and instead 
I'm using an MUA like Thunderbird, whenever I need to do this, I will 
either CC or BCC myself, and with my copy, just forward it the the 
special private address with a top posted FYI "Keeping you in the 
loop" note.

I am also very careful of not BCCing someone that is know to 
reply/response and will just BCC a  person will never get involve - 
they just wants to be keep in the loop.  I do this almost daily with 
technical support communications to keep the sales team in the loop of 
whats going on.

So IMV, we should also take into the account that people who generally 
need to use a BCC, most likely don't do so blindly themselves.

The way I look at it, due to what is today, the MUA is responsible and 
if the MUA author is going to offer a BCC feature, it needs to do so 
responsibly knowing full well how the backend works which never came 
with an backend expectation to do BCC processing, stream splitting, 
privacy cleanup.  If it exist, its a proprietary MUA/MSA concept.

IMV, if we are looking to move this MUA duty to the backend with a 
standard methodology, then we need a IETF I-D/RFC proposal and SMTP 
extension that defines rules for the router to do this BCC splitting. 
  But I wonder if its worth it because on the original mail creation 
side with the OFFLINE MUA, they are all already doing it.  The MSA 
router does not need to currently worry about it.  So it seems the 
issue is only for the receiver MUA to consider and off hand only 
because of the modern expectation of MIME based mail. If the BCC is 
passthru, then it should be more aware from the display rendering 
standpoint the special meaning it has to convey to the blind recipient.

-- 
Sincerely

Hector Santos
http://www.santronics.com
jabber: [email-masked]

Date: Mon, 16 Apr 2012 01:29:14 -0400
From: Hector Santos <[email-masked]>
Organization: Santronics Software, Inc.
To: [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

Robert A. Rosenberg wrote:
> 
> At 10:04 -0500 on 03/01/2012, Hector Santos wrote about Re: "proper" 
> handling of BCC:
> 
>> There is also the consideration regarding displaying.  The MUA may 
>> want to inform the BCC recipient to the privacy nature of the message:
>>
>>     NOTE: THIS IS A BLIND COPY TO YOU. THE OTHER RECIPIENTS LISTED IN THE
>>     TO/CC DISTRIBUTION ARE NOT AWARE OF YOUR RECEPTION OF THIS MESSAGE.
> 
> Query - For issuing this message, how do you determine that the 
> recipient is getting a copy of the message without being listed in a To 
> or Cc header due to being BCC'ed or being subscribed to a mailing list 
> (or do you treat a mailing list received copy as a BCC'ed copy)?

Hi Robert,

As John K. alluded to, this would be done at the source - author's MUA 
where the message is first created.  Perhaps the primary point is that 
the sender's distribution list and receiver's reception list is not 
based on the RFC5322 headers but rather the SMTP envelope RCPT TO 
addresses and these do not need to match what the RFC5322 headers say.

Is the question about how the MUA will do this based on it using 
RFC5322 as UI meta when the user is creating the message?

If so, the MUA does not to use RFC5322 for storage data, it could, but 
when the SEND button it clicked, it needs to begin to split the blind 
recipient streams from the others because its not expected for the MSA 
router to do this.  It might be possible if we had an SMTP BCC service 
extension.

Just winging it, using the following MUA headers:

   From:<author-address>
   To: <user1-address>
   To: <user2-address>
   Bcc: <user3-address>

The MUA session might be:

   C: EHLO client.domain
   S: 250-server.example, Hello client.domain
   S: 250-.......
   S: 250-BCC
   S: 250 HELP
   C: MAIL FROM:<author-address>
   S: 250 ok
   C: RCPT TO:<user1-address>
   S: 250 ok
   C: RCPT TO:<user2-address>
   S: 250 ok
   C: RCPT TO:<user3-address> BLIND=NOTE|BCC
   S: 250 ok
   ....

and this basically tells the MSA router to SPLIT this distribution of 
the payload into different MTA mail queues.

The BLIND[=options]

   NOTE - MSA router MAY add the "This is blind copy" body text
   BCC  - MSA router MAY add the 5322.BCC for the blind recipient only.

   none - MSA router decides.
           - For MIME mail, assume BLIND=BCC
           - For text/plain simple mail, assume BLIND=NOTE

So if the question is about rendering, displaying, then the BLIND=NOTE 
or BLIND=BCC can help. Having 5322.BCC in the blind copy can allow for 
supported advanced MUAs to highlight this special private blind copy 
to the blind recipient. This may help alleviate concerns that top 
posted warning text would interfere with the more complex MIME messages.

-- 
Sincerely

Hector Santos
http://www.santronics.com
jabber: [email-masked]

Date: Sun, 15 Apr 2012 19:52:06 -0400
From: John C Klensin <[email-masked]>
To: Steve Atkins <[email-masked]>, [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

--On Sunday, April 15, 2012 16:05 -0700 Steve Atkins
<[email-masked]> wrote:

>>> 
>> 
>> If I correctly understand the question, I simply put a "bcc"
>> field in the message as I submit it to the MUA.  It removes
>> that field. constructs the "blind copy" warning text, and
>> constructs the envelope(s) appropriately (typically one
>> envelope for the set of To and Cc recipients and one for each
>> Bcc one) for handoff to the Submission server.   Of course,
>> if one had a submission server that was smart about these
>> things, the MUA could hand the message, including the Bcc
>> field, to it and let it sort out envelopes (which it needs to
>> do anyway) and the warning message.
> 
> A submission server will have some difficulty adding a
> warning message that'll be visible to a typical recipient 
> (multipart/mime, non plain-text, DKIM, PGP, S/MIME…).

Having the MUA add such comments assumes that there is a
convenient body part (probably the first one) that is text/plain
or, worst case, that it has to add the text to both/all pieces
of a multipart/alternative.  Such plain-text or nearly
plain-text messages remain the typical case today.  In the
absence of signatures or other integrity checks added by the MUA
or earlier, a submission server should have no more trouble
doing that than the MUA itself (although I confess to not having
a submission server do that for years).    DKIM is even easier
since, at least in the cases I've seen the typical behavior is
for that signature to be added by the Submission server.

Now, clearly, if one had any of a large family of less-typical
messages (complex multipart structures; first (or maybe only)
body part not text/plain, text/html, or
text/something-else-fairly-simple-and-well-known, signatures
applied to the body part before the message can be added, etc.,
then the trick of adding a warning doesn't work.  MUAs and
submission servers typically cope with those situations the way
alert MUAs that don't try to add a warning do, which is, as you
suggest below, to delete the Bcc header field (or its contents
-- that used to be extensively debated) and then construct and
appropriate envelope.

> Handling Bcc by copying the Bcc header to the envelope,
> then deleting the header is fine, though I wouldn't trust any
> MUA that relied on that functionality.

Yes.  The difficulty with this --and what caused the "add the
comment/warning" behavior-- is that the recipients cannot easily
tell "received a blind copy" from "received from a mailing
list".  If they then reply, copying the explicit distribution,
the result is a privacy compromise relative to the intentions of
the original sender.  If the original sender were really worried
about that, the only safe action I know of is to avoid use of
bcc entirely, instead, for example, forwarding a copy of the
message with the explicit recipient listed to the bcc recipient
only.  That message could well be structured with the warning as
a first body part followed by the original as message/rfc822 or
message/global.  But it is a little hard to automate.

    john

From: Steve Atkins <[email-masked]>
Subject: Re: "proper" handling of BCC
Date: Sun, 15 Apr 2012 16:05:06 -0700
To: [email-masked]
Sender: [email-masked]

--Apple-Mail=_97FB74D3-0D24-49C2-82C7-47CE24FC49E0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252

On Apr 15, 2012, at 3:56 PM, John C Klensin wrote:
>>=20
>=20
> If I correctly understand the question, I simply put a "bcc"
> field in the message as I submit it to the MUA.  It removes that
> field. constructs the "blind copy" warning text, and constructs
> the envelope(s) appropriately (typically one envelope for the
> set of To and Cc recipients and one for each Bcc one) for
> handoff to the Submission server.   Of course, if one had a
> submission server that was smart about these things, the MUA
> could hand the message, including the Bcc field, to it and let
> it sort out envelopes (which it needs to do anyway) and the
> warning message.

A submission server will have some difficulty adding a
warning message that'll be visible to a typical recipient=20
(multipart/mime, non plain-text, DKIM, PGP, S/MIME=85).

Handling Bcc by copying the Bcc header to the envelope,
then deleting the header is fine, though I wouldn't trust any
MUA that relied on that functionality.

Cheers,
  Steve

--Apple-Mail=_97FB74D3-0D24-49C2-82C7-47CE24FC49E0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=windows-1252

<html><head></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><br><div><div>On Apr 15, 2012, at 3:56 PM, John C Klensin =
wrote:</div><blockquote type=3D"cite"><div><blockquote type=3D"cite"><font=
 class=3D"Apple-style-span" =
color=3D"#000000"><br></font></blockquote><br>If I correctly understand =
the question, I simply put a "bcc"<br>field in the message as I submit =
it to the MUA. &nbsp;It removes that<br>field. constructs the "blind =
copy" warning text, and constructs<br>the envelope(s) appropriately =
(typically one envelope for the<br>set of To and Cc recipients and one =
for each Bcc one) for<br>handoff to the Submission server. =
&nbsp;&nbsp;Of course, if one had a<br>submission server that was smart =
about these things, the MUA<br>could hand the message, including the Bcc =
field, to it and let<br>it sort out envelopes (which it needs to do =
anyway) and the<br>warning =
message.<br></div></blockquote></div><br><div>A submission server will =
have some difficulty adding a</div><div>warning message that'll be =
visible to a typical recipient&nbsp;</div><div>(multipart/mime, non =
plain-text, DKIM,&nbsp;PGP, S/MIME=85).</div><div><br></div><div>Handling =
Bcc by copying the Bcc header to the envelope,</div><div>then deleting =
the header is fine, though I wouldn't trust any</div><div>MUA that =
relied on that =
functionality.</div><div><br></div><div>Cheers,</div><div>&nbsp; =
Steve</div><div><br></div></body></html>=

--Apple-Mail=_97FB74D3-0D24-49C2-82C7-47CE24FC49E0--

Date: Sun, 15 Apr 2012 18:56:49 -0400
From: John C Klensin <[email-masked]>
To: "Robert A. Rosenberg" <[email-masked]>, [email-masked]
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

--On Sunday, April 15, 2012 16:14 -0400 "Robert A. Rosenberg"
<[email-masked]> wrote:

> 
> At 10:04 -0500 on 03/01/2012, Hector Santos wrote about Re:
> "proper" handling of BCC:
> 
>> There is also the consideration regarding displaying.  The
>> MUA may  want to inform the BCC recipient to the privacy
>> nature of the  message:
>> 
>>     NOTE: THIS IS A BLIND COPY TO YOU. THE OTHER RECIPIENTS
>>     LISTED IN THE TO/CC DISTRIBUTION ARE NOT AWARE OF YOUR
>>     RECEPTION OF THIS MESSAGE.
> 
> Query - For issuing this message, how do you determine that
> the recipient is getting a copy of the message without being
> listed in a To or Cc header due to being BCC'ed or being
> subscribed to a mailing list (or do you treat a mailing list
> received copy as a BCC'ed copy)?
> 
> I ignore the case of being listed in a To or Cc as a
> suppressed address (ie: Group-Name:add1, add2, etc. ;) since
> the existence of a group-name:; comment implies a hidden BCC
> list.

If I correctly understand the question, I simply put a "bcc"
field in the message as I submit it to the MUA.  It removes that
field. constructs the "blind copy" warning text, and constructs
the envelope(s) appropriately (typically one envelope for the
set of To and Cc recipients and one for each Bcc one) for
handoff to the Submission server.   Of course, if one had a
submission server that was smart about these things, the MUA
could hand the message, including the Bcc field, to it and let
it sort out envelopes (which it needs to do anyway) and the
warning message.

In case it wasn't clear from Hector's message, this is not
theory.  There are several MUAs, including the one I'm using,
that are able to behave exactly as outlined above.

    john

Date: Sun, 15 Apr 2012 16:14:38 -0400
To: [email-masked]
From: "Robert A. Rosenberg" <[email-masked]>
Subject: Re: "proper" handling of BCC
Sender: [email-masked]

At 10:04 -0500 on 03/01/2012, Hector Santos wrote about Re: "proper" 
handling of BCC:

>There is also the consideration regarding displaying.  The MUA may 
>want to inform the BCC recipient to the privacy nature of the 
>message:
>
>     NOTE: THIS IS A BLIND COPY TO YOU. THE OTHER RECIPIENTS LISTED IN THE
>     TO/CC DISTRIBUTION ARE NOT AWARE OF YOUR RECEPTION OF THIS MESSAGE.

Query - For issuing this message, how do you determine that the 
recipient is getting a copy of the message without being listed in a 
To or Cc header due to being BCC'ed or being subscribed to a mailing 
list (or do you treat a mailing list received copy as a BCC'ed copy)?

I ignore the case of being listed in a To or Cc as a suppressed 
address (ie: Group-Name:add1, add2, etc. ;) since the existence of a 
group-name:; comment implies a hidden BCC list.