Subject: Re: [Shutup] [ietf-smtp] Proposed Charter for the "SMTP Headers Unhealthy To User Privacy" WG (fwd)

From: Dave Crocker <[email-masked]>
Date: Sun, 29 Nov 2015 09:25:47 -0800

On 11/29/2015 9:12 AM, Chris Newman wrote:

> ====
> This WG will investigate mechanisms to conceal the information exposed
> by the submission client's IP address in the mandatory received header
> generated by the submission server. The output of this WG will provide a
> mechanism as effective at tracing abuse and fraud as current use of the
> submission client's IP address. Changing other rules related to received
> headers in SMTP is out of scope for this WG.
> ====

Chris's approach is reasonable, but I fear it is premature.

The foundational issue here is a trade-off between information hiding
and information disclosure.  Privacy vs. ops support.

I've seen essentially no public discussions, here or anywhere else,
about the technical aspects of this policy tradeoff.

Absent some community-based sense of the underlying technical issues
here, targeting a specification is, in my view, not ready for prime time.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Alexey Melnikov <[email-masked]>
Date: Sun, 29 Nov 2015 18:34:46 +0000

Hi Chris,

> On 29 Nov 2015, at 17:12, Chris Newman <[email-masked]> wrote:
> 
> I oppose the current shutup charter text and draft-josefsson-email-received-privacy as both promote the elimination of mechanisms that protect users from fraud and abuse.
> 
> As I do care about user privacy, here's a strawman charter that I would support:
> 
> ====
> This WG will investigate mechanisms to conceal the information exposed by the submission client's IP address in the mandatory received header generated by the submission server. The output of this WG will provide a mechanism as effective at tracing abuse and fraud as current use of the submission client's IP address. Changing other rules related to received headers in SMTP is out of scope for this WG.
> ====

I think this is a very reasonable counter proposal.

To address concerns from other people it might be worth tweaking it, so that if after investigation no reasonable proposal can be produced, the group should produce a document describing why the problem is not tractable or not worth solving considering tradeoffs.

> 
> I believe RFC 2442 combined with PGP or S/MIME adequately protects email headers. I worked on an implementation of that in the 1990s. I'm doubtful the memory-hole proposal is sufficiently better or sufficiently likely to deploy to be worth IETF effort.

There might be more will to implement something this time around. Whether the memory-hole proposal is it, I don't know.

Best Regards,
Alexey

---

From: "John Levine" <[email-masked]>
Date: 29 Nov 2015 18:13:46 -0000

>and information disclosure.  Privacy vs. ops support.

It's not even privacy vs. ops support, it's privacy issues via some
hints of sender's location vs. privacy issues via the recipient
getting spammed, phished, and malware'd.

You are completely correct that there's been no discussion at all of
the tradeoffs.  After all, the vast majority of mail these days is
malicious, and the largest mail provider in the world already redacts
most senders' location info.

R's,
John

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 29 Nov 2015 21:58:44 +0000

(Re-tx - there was an address typo in the earlier message)

On 29/11/15 18:13, John Levine wrote:
>  the largest mail provider in the world already redacts

As I understood it, one of the possible outcomes of this
effort might be to document the kinds of redactions that
have been done so that others could do similarly.

I've no clue as to whether that'd result in a menu of "things
one can do," or if there are parts of that that may end up
as standards, but a good first step might be to document
what's been done already, esp if it's different in different
mail providers and if those variations have different impacts
elsewhere in the overall mail infrastructure.

Seems like the above would be a useful start anyway. (If
it's not been done already - there's no reason such stuff
absolutely has to be in an RFC.)

S.

---

From: Jim Fenton <[email-masked]>
Date: Sun, 29 Nov 2015 14:53:40 -0800

On 11/29/2015 09:12 AM, Chris Newman wrote:
> I oppose the current shutup charter text
> and draft-josefsson-email-received-privacy as both promote the
> elimination of mechanisms that protect users from fraud and abuse.

Agreed, and to be more specific:

The proposed charter speaks of Received header fields leaking address
information that can expose user location. Yes, they can. But, in
general, that information is essential to identifying spoofed header
fields: it's by tracing the chain of "from" addresses in Received header
fields that one can determine that someone is attempting to do something
fraudulent. Further, I don't have a lot of sympathy for organizations
that rely on the secrecy of their network topologies as an essential
security component. We're trying to increase the trust in email, not
reduce it.

There are users for whom their privacy is critically important, such as
press informants in totalitarian societies. There are many other ways to
determine their location (network monitoring coupled with a STARTTLS
downgrade attack, for one), and it would be harmful (potentially
life-threatening) if anyone thought that this would truly protect them.
They should be using something like SecureDrop and not using email at all.

draft-josefsson-email-received-privacy mentions the issue of senders'
locations appearing on mailing lists and in mailing list archives. I
have long felt that we are conflicted on whether the output of a mailing
list is a new message or the same as the one sent to the mailing list.
It usually has a different MAIL FROM address, and often has text added
to the message body, which I would think is enough of a change to make
it a new message. Yet the Message-ID and Received header fields are
preserved. I would think that an entire new message should be created,a
new Message-ID assigned, and DKIM signed by the mailing list's domain
(of course!). Only selected header fields would be transferred to the
new message. The original incoming header fields should be available
only to the list administrators, who deal with abuse issues.

What would the motivation be for anyone to implement any header privacy
improvements? There is far too much deployed infrastructure to get a
change to be made, absent a very strong business case. We could spend a
lot of time on this and not have it matter a bit.

I would support some work on the mailing list issue resulting in a set
of recommended practices, and possibly guidance on obscuring the source
IP address when an authenticated submission is made to an MSA. But
that's it.

-Jim

---

From: "Christian Huitema" <[email-masked]>
Date: Sun, 29 Nov 2015 15:02:57 -1000

On Sunday, November 29, 2015 12:54 PM, Jim Fenton wrote:
>
> There are users for whom their privacy is critically important, such 
> as press informants in totalitarian societies. There are many other 
> ways to determine their location (network monitoring coupled with 
> a STARTTLS downgrade attack, for one), and it would be harmful 
> (potentially life-threatening) if anyone thought that this would truly 
> protect them. They should be using something like SecureDrop and 
> not using email at all.

Uh, No. This is the classic "the other side of the boat is leaking too"
argument, coupled with a dollop of "no security is better than imperfect
security." Yes, there are many ways for metadata to leak. But that does not
mean that we should not plugs the leaks that we do know about.

The discussion so far shows that one hand many people believe that we are
disclosing too much metadata in mail headers, while many more believe that
the metadata disclosure is actually useful to fight various forms of abuse,
some of which may well compromise users' privacy. 

We also heard that some of the big providers have already unilaterally
decided to suppress some of the metadata, like the first hop address. So we
have at least one data point showing that not all metadata needs to be
preserved.

The "submission" hop may be a special case, but as Jim points out, mailing
lists may well another special case, for which some guidance would be
useful.

The concern about topology disclosure may or may not justify pruning some of
the metadata.

In short, it appears that there is enough concern and enough uncertainty to
justify working at least on an analysis document, and depending on the
outcome on a best practice document. Let's have this debate, and let's make
some progress on email privacy.

-- Christian Huitema

---

From: Ted Lemon <[email-masked]>
Date: Mon, 30 Nov 2015 02:18:03 +0000

> getting spammed, phished, and malware'd.

The only Received: header fields that you can trust are the ones that were added by servers in your administrative domain.   Anything else, you have to trace back through the logs hop by hop to actually know that the mail transited those systems; otherwise, an attacker can fake up whatever Received: header fields they want in order to cast blame on whomever they want to harm.   That being the case, all the Received: header field needs to contain is a token that can be used to backtrace the message to its origin using the logs: anything else is superfluous.   So postcarding all kinds of private information about the end user is not only not actually useful for the reason you suggest, it is actively harmful to the end user.

> most senders' location info.

The conclusion I would draw here is that we ought to advice all the other providers to do this as well.   IOW, this argument supports the proposed work, rather than showing that it is unnecessary or harmful.

--
Sent from Whiteout Mail - https://whiteout.io

My PGP key: https://keys.whiteout.io/[email-masked]

---

From: Ted Lemon <[email-masked]>
Date: Mon, 30 Nov 2015 02:25:40 +0000

Sunday, Nov 29, 2015 5:53 PM Jim Fenton wrote:
> But, in
> fraudulent.

Can you cite a real-world example of a case where you did something like this recently, and explain how you were able to do what you claim, above, is possible using just the header fields in the message?

---

From: "John Levine" <[email-masked]>
Date: 30 Nov 2015 03:11:50 -0000

>> But, in general, that information is essential to identifying spoofed header
>Can you cite a real-world example of a case where you did something like this recently, and explain how you were able to do what you
>claim, above, is possible using just the header fields in the message?

Spam filters have been doing Received chain analysis for about 20
years.  The principle is straightforward, the source in each header
should match the recipient in the header below it, and timestamps
should be in the right order.  There's also heuristics based on
knowing what real headers from popular mail systems should look like.

The scripts I use to send off spam complaints do header analysis to
figure out who to complain to, and not to complain to addresses in
fake headers, so I'd say I do this about 100 times a day, every day,
in addition to spamassassin doing it on every incoming message that
it filters.

If you want to look at some code, spamassassin is at
http://spamassassin.apache.org/.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 30 Nov 2015 03:23:46 -0000

>The only Received: header fields that you can trust are the ones that were added by servers in your administrative domain.

Not at all.  If you get mail from a provider you know to be
trustworthy, you can trust the headers they add.  Given that mail is
dominated by large providers, and any individual mail system tends to
exchange mail with the same people over and over, you can cover a
great deal of your mail with a modest number of sort of special cases
for providers you know.  For example, I live near Ithaca NY so I
exchange a lot of mail with Cornell, which is full of students who
never saw a malware-infested game they wouldn't install.  So it's
useful to have scripts that can look at their headers so I can tell
them who needs to be thwacked today.

>the Received: header field needs to contain is a token that can be used to backtrace the message to its origin using the logs:

Um, we have some scaling issues here.  Large providers send and
receive billions of messages a day so looking up stuff in the logs is
non-trivial.  Also, when you're trying to shut down a phish campaign,
you want to do it ideally in minutes, at worst hours, not whenever
someone can get around to pulling some log info from a distant server.

There are a lot of people on ietf-smtp familiar with large mail
systems, so perhaps it would be more productive to ask questions about
how stuff actually works.

R's,
John

---

From: Ted Lemon <[email-masked]>
Date: Mon, 30 Nov 2015 03:25:01 +0000

Sunday, Nov 29, 2015 10:11 PM John Levine wrote:
> years.

Yes, I know.   A friend of mine founded a company that worked using this principal.   Unfortunately, it got less and less effective as spammers got better and better at faking things.   The reason I asked for recent experience is that I'm curious if anyone is _still_ getting real benefit from this.

Since the only header-field you can actually trust is the first one that your own MTA adds, SPF works just as well (actually, a _lot_ better) as a validation mechanism.   If SPF isn't in use, the sender can just claim to be a legitimate agent for the source domain, and you have no way to check that claim.

---

From: "John Levine" <[email-masked]>
Date: 30 Nov 2015 04:28:19 -0000

>> Spam filters have been doing Received chain analysis for about 20 years.
>
>Yes, I know.   A friend of mine founded a company that worked using this principal.   Unfortunately, it got less and less effective as
>spammers got better and better at faking things.   The reason I asked for recent experience is that I'm curious if anyone is _still_
>getting real benefit from this.

Yes.  See the message you just replied to.

>Since the only header-field you can actually trust is the first one that your own MTA adds, ...

No, that's not correct.  See recent message.

SPF works just as well (actually, a _lot_ better) as a validation mechanism.

What?  Header chains say "this message came through this IP".  SPF
says "I assert that these IPs are allowed to send mail for this
domain."  They're completely unrelated.

We don't use header chains for validation, we use it to figure out who
to blame, who to alert, and who to block.

R's,
John

---

From: Ted Lemon <[email-masked]>
Date: Mon, 30 Nov 2015 04:46:14 +0000

> Yes.  See the message you just replied to.

The experience you related sounds like hobbyist activity, no offense.   I can't see how what you described could possibly scale to anything a large email provider would ever do, and indeed it's not something I would ever take the trouble to do, even though my domain has exactly two users.   If I get spam, I delete it.   I do not complain about it.  I do not attempt to understand where it originated.

> No, that's not correct.  See recent message.

I saw the recent message, and again, what you described is something that doesn't scale.   If that is your model for how header-field messages are used for validation, I think what I said is actually more generally accurate.   Do you seriously think that Google has special-case header parsing to deal with spam from Cornell students' infected computers?   No, they just use machine learning.

> domain."  They're completely unrelated.

SPF allows me to discard all messages that claim to be from domain X but come from IP addresses not listed for domain X, which means that I never have to write a Received: header for that message.   If there is no SPF for the domain that sent the message, I would like to just discard it as spam, but that's not safe to do because so many small sites don't implement SPF or get it wrong.   But in any case where there is no SPF record, the site is definitely not trustworthy: I cannot rely on the Received: header fields it included in the message.   And if the site _is_ trustworthy, then modulo a few small exceptions like Cornell, it's not originating anything that can be reasonably identified as spam, because if it could have been reasonably identified as spam, it would never have been forwarded.   So I can't see how the experiences you are describing would motivate the inclusion of something like an unobfuscated Received: header field across organizational boundaries if SMTP were being designed today.

> to blame, who to alert, and who to block.

I don't know who "we" is here.   Is this really how Google, et al., operate?   It sounds like something people like you and me used to do ten years ago, that some of us old-timers are still doing (I only stopped using spamassassin recently, for example).   I would be okay with being proven wrong, but the fact that you are still doing this isn't persuasive to me.

---

From: Ned Freed <[email-masked]>
Date: Sun, 29 Nov 2015 20:53:17 -0800 (PST)

> The only Received: header fields that you can trust are the ones that were
> added by servers in your administrative domain.

Nonsense. The most obvious counterexample is when you receive mail from a
trusted server in some other trusted domain.

In fact this is now the most common case: Mail is submitted to a large service
provider which if it exits the administrative domain at all goes directly to
your administrative domain. The Received: header fields from, say, aol.com is
entirely trustable, especially since you can check the DKIM signature.

A certain amount of heurisitics are involved, but these are well understood.

> Anything else, you have to trace back through the logs hop by hop to actually
> know that the mail transited those systems; otherwise, an attacker can fake up
> whatever Received: header fields they want in order to cast blame on whomever
> they want to harm.

No, they really can't. See above.

> That being the case, all the Received: header field needs to contain is a
> token that can be used to backtrace the message to its origin using the
> logs: anything else is superfluous.

Nope. For one thing, the use of a token would require some means
of accessing logs across administrative domains. That's not going to happen
for a whole bunch of reasons.

And the "from" clause d is far from the only thing of use in a Received: field.
In fact it's far from the only thing that has privacy implications. Just
as one example, "with" clauses can be used to indicate whether or 
not SSL/TLS was used to secure the connection. An unbroken Received: chain
can provide assurance (or repudiation) of end-to-end transport protection.

Unfortunately, so far the "analysis" that's been done of the privacy
considerations of email trace information has been so shallow that stuff  like
this has been missed. 

> So postcarding all kinds of private
> information about the end user is not only not actually useful for the
> reason you suggest, it is actively harmful to the end user.

Nonsense again. In a world where almost email is spam or worse, any mechanism
that provides benefits in the fight against spam is beneficial to end users.

				Ned

---

From: Ted Lemon <[email-masked]>
Date: Mon, 30 Nov 2015 05:27:15 +0000

> entirely trustable, especially since you can check the DKIM signature.

Why would you want to check the Recieved: header fields in this case?   Under what circumstances would checking the header fields in a message known to have been received from aol.com, with a valid DKIM signature, add value over the validation you're already stipulating has happened?

> No, they really can't. See above.

I think we may be talking about different things.   If you are trying to identify spam, you wouldn't need to do this for mail received through aol.com.   I'm talking about the case where someone is committing some kind of illegal behavior that triggers search warrants or other requests for records.  In this case, lawful requests can be satisfied by searching logs, and could not be satisfied by looking at the text of a message.   The data in such a message could not be considered trustworthy.

> can provide assurance (or repudiation) of end-to-end transport protection.

Why would that be useful?

> considerations of email trace information has been so shallow that stuff  like

I don't think it's so much been missed as found to be of no obvious value.

> that provides benefits in the fight against spam is beneficial to end users.

Yes, let us think of the children!

---

From: "John Levine" <[email-masked]>
Date: 30 Nov 2015 05:52:20 -0000

>  I can't see how what you described could
>possibly scale to anything a large email provider would ever do, ...

This argument is going into the weeds.  The question is whether the
information in Received headers is useful for mail system management
(which is not a synonym for spam filtering) and your apparent
assertion that it only matters what giant mail systems do is pretty
creepy.

> Do you seriously think that Google has special-case header parsing to deal with spam from Cornell students' infected computers? No, they just use machine learning.

Unfortunately, due to NDA's I can't talk about Google's spam
filtering.

>SPF allows me to discard all messages that claim to be from domain X but come from IP addresses not listed for domain X,

Yeah, we know what SPF does, and the many and wonderful ways that it
doesn't quite do what it's supposed to.  But since nobody has ever
said that we use Received headers for sender authorization, there's
still no point here.

> And if the site _is_ trustworthy, then modulo a few small exceptions
> like Cornell, it's not originating anything that can be reasonably
> identified as spam, because if it could have been reasonably
> identified as spam, it would never have been forwarded.

Aw, come on.  I get plenty of spam from Gmail and Yahoo, all of which
is 100% SPF, DKIM, and DMARC compliant and has 100% real Received
headers.  Unless you are extremely unusual, you do too.  Crooks sign
up for public mail systems to send spam, and on mail systems of all
sizes they steal or guess AUTH credentials to spam through compromised
accounts, or compromise web servers and spam through buggy old
drupal and wordpress setups.

>I don't know who "we" is here.   Is this really how Google, et al., operate?

I can't talk about Google, but I can tell you from direct experience
working around their DMARC damage that some other large mail systems
use Received headers as part of their spam filtering process, which is
not just checking sender authorization.

Some of us go to conferences like MAAWG where we spend a lot of time
talking to people who run all sorts of medium and large mail systems,
and other conferences where we talk about security problems and
responses to them with various combinations of cops and nerds.  It's
OK that you're not as familiar with all this stuff, just as I am not
as familiar with a lot of the DNS and IPv6 stuff you do.  But you
might consider the possibility that we actually do know something
about the areas in which we work.

R's,
John

---

From: "Rolf E. Sonneveld" <[email-masked]>
Date: Mon, 30 Nov 2015 10:34:51 +0100

On 30-11-15 02:02, Christian Huitema wrote:
> decided to suppress some of the metadata, like the first hop address.

Can anyone share some information about which providers made which decision?

> So we

I fail to see the causality with the first sentence here: do you mean: 
the worlds e-mail ecosystem did not collapse, so this proves that not 
all metadata needs to be preserved?

> some progress on email privacy.

There has been done some work on this, which might be useful input to 
the discussion: see the report "Investigating the leakage of sensitive 
personal and organisational information in email headers" [1].

/rolf

[1] https://www.cs.ox.ac.uk/publications/publication9347-abstract.html.

---

From: Chris Newman <[email-masked]>
Date: Sun, 29 Nov 2015 09:12:20 -0800

====

		- Chris

---

From: Steve Atkins <[email-masked]>
Date: Sun, 29 Nov 2015 11:10:30 -0800

> On Nov 29, 2015, at 9:25 AM, Dave Crocker <[email-masked]> wrote:
> 

There are already providers who remove or falsify Received headers in
order to protect their users, so there is also the opportunity to look at
what is currently being done and the effects of it.

As one example, Gmail is one of those providers. (They're also consistently
the biggest source of B2B spam in my inbox.)

Cheers,
  Steve

---

From: Richard Clayton <[email-masked]>
Date: Mon, 30 Nov 2015 12:04:36 +0000

In message <[email-masked]>, Ted
Lemon <[email-masked]> writes

>Do 
>you seriously think that Google has special-case header parsing to deal with 
>spam from Cornell students' infected computers?   No, they just use machine 
>learning.

... and one of the things that the ML will be processing will be the
(tokenised contents of the) header fields... so having a pattern (of any
kind) within the header fields has the potential to be extremely helpful
in distinguishing good from bad

>SPF allows me to discard all messages that claim to be from domain X but come 
>from IP addresses not listed for domain X, which means that I never have to 
>write a Received: header for that message.

It rather escapes me how one of your users will be able to determine
whether you received the email from a domain which had SPF at the time
at which you received it unless you record that information along with
the email (or do you think that DNS results are constant for all time?)

If you're relaying the email on to somewhere else then you're assuming
that there's a mechanism by which your policy regarding SPF becomes
known to those other people.

I'm unaware of such a mechanism existing at the moment -- and (this
might be relevant to charters, albeit I really don't think that there's
any real mileage in this topic at all) that although we have a scheme
for originators of email to publish policy recommendations about the
handling of email from their domains we don't have a similar policy for
relaying machines (and absent a Received header field it's unclear to me
what indicator we'd use to look up what that policy was).

>   If there is no SPF for the domain 
>that sent the message, I would like to just discard it as spam, but that's not 
>safe to do because so many small sites don't implement SPF or get it wrong.   
>But in any case where there is no SPF record, the site is definitely not 
>trustworthy:

that's a shame, I consider myself very trustworthy and I've never
bothered with SPF :-(

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: Ned Freed <[email-masked]>
Date: Mon, 30 Nov 2015 07:19:09 -0800 (PST)

> I saw the recent message, and again, what you described is something that
> doesn't scale.

I'm afraid again you're demonstrating that you really have no idea how this
stuff works at scale.

It is of course the case that Received: field analysis that depends on
knowledge of the domains involved doesn't scale to handle huge numbers of
domains. But that's irrelevant: Nobody cares about smalldomain.com's Received:
fields in this context, for any value of smalldomain.com.

The present day reality is that a significant fraction of mail originates
from a small number of domains. Those are the ones you care about. They're
the ones you care about, and since you can verify that mail actually came
from one of them through other means, you now have Received: fields you
can trust for a significant fraction of the mail you receive.

> If that is your model for how header-field messages are used for validation, I
> think what I said is actually more generally accurate.

Where on earth did you get the idea that Received: fields are used for
validation?

> Do you seriously think that Google has special-case header parsing to deal with

I have no idea what Google does, but I can assure you Received: field analysis
does play a role in spam filtering elsewhere. And even if it didn't, there's
more to mail operations than spam filtering.

				Ned

---

From: Ned Freed <[email-masked]>
Date: Mon, 30 Nov 2015 07:38:21 -0800 (PST)

> Can anyone share some information about which providers made which decision?

I posted about this earlier on the perpass list in respose to the initial
discussion of the draft-josefsson-email-received-privacy draft. Here's an
updated version of that information:

Gmail:   Webmail does not disclose originating client IP, apparently using
         invalid Received: field to avoid doing so.
         Submit discloses originating IP.
Yahoo:   Neither webmail nor submit disclose originating IP, some Received:
         fields are invalid but this looks like an unrelated issue.
Outlook: Neither webmail nor submit disclose originating IP, valid Received:
         fields.
AOL:     Both webmail and submit disclose originating client IP in both
         Received: fields and X-Originating-IP: (webmail) and X-AOL-IP:
         (submit) fields.
GMX:	 Both webmail and submit disclose originating client IP.

Thanks John Levine for the AOL submit data.

				Ned

---

From: "John Levine" <[email-masked]>
Date: 30 Nov 2015 16:21:24 -0000

>I'm unaware of such a mechanism existing at the moment --

That's more or less what Authentication-Results and the deprecated
Received-SPF headers do, recording a snapshot of what the SPF results
were at the time a message was relayed. The ARC proposal in
draft-andersen-arc-00 adds a signed chain of A-R trace headers.

But again, SPF and A-R and ARC don't do the same thing as Received, which
is why we have different headers for them.

R's,
John

---

From: "John Levine" <[email-masked]>
Date: 30 Nov 2015 16:26:03 -0000

>Can anyone share some information about which providers made which decision?

Please look at the list's archives, now available through a dandy new
search engine:

https://mailarchive.ietf.org/arch/search/?qdr=a&start_date=&end_date=&email_list=shutup&q=text%3A%28gmail%29&as=1

R's,
John

---

From: Dave Crocker <[email-masked]>
Date: Mon, 30 Nov 2015 08:46:42 -0800

> Received-SPF headers do, recording a snapshot of what the SPF results
...

Probably relevant, current work:

     Authenticated Received Chain (ARC)
     https://tools.ietf.org/html/draft-andersen-arc-00

d/

---

From: Al Iverson <[email-masked]>
Date: Mon, 30 Nov 2015 10:11:17 -0600

> more to mail operations than spam filtering.

I work for an email service provider dealing with rather large volumes
of mail, yet Received header analysis still plays an important role
for us in troubleshooting operational issues. It would be a tremendous
loss to lose received headers or the IP address information in those
headers. You raise Google as an example; Gmail occasionally seems to
delay message delivery. The internal received hops in a Gmail header
tell you that it's Gmail, not the sender, delaying that delivery of
the message, because the message is likely queued on some internal
server for some period of time, after sender's initial handoff.

Some of us do actually process this data at some level of scale to
identify issues.

Not only do I seriously believe it, but I have also personally
observed AOL to use special-case header parsing to deal with certain
types of mail, to more directly answer your question.

Regards,
Al Iverson

--
Al Iverson - Minneapolis - (312) 275-0130
Simple DNS Tools since 2008: xnnd.com
www.spamresource.com & aliverson.com

---

From: Steve Atkins <[email-masked]>
Date: Mon, 30 Nov 2015 08:17:39 -0800

> On Nov 30, 2015, at 8:11 AM, Al Iverson <[email-masked]> wrote:
> 

I've heard AOL people discuss "second received header parsing", and
effective filtering of infected dorm computers behind University smarthosts
was one of the use cases. This isn't something speculative, it's something
that's been in use for years.

Cheers,
  Steve

---

From: Randall Gellens <[email-masked]>
Date: Mon, 30 Nov 2015 09:29:10 -0800

At 2:53 PM -0800 11/29/15, Jim Fenton wrote:

>>  I oppose the current shutup charter text and 
>> draft-josefsson-email-received-privacy as both promote the 
>  Agreed,

Also agree.

>   and to be more specific:
>
>  The proposed charter speaks of Received header fields leaking 
> address information that can expose user location. Yes, they can. 
> But, in general, that information is essential to identifying 
> spoofed header fields: it's by tracing the chain of "from" 
> addresses in Received header fields that one can determine that 
> someone is attempting to do something fraudulent.

Very true.  It seems to me that every few years, proposals are made 
to effectively destroy 'Received' header fields without understanding 
how and why they are so useful.

>   Further, I don't have a lot of sympathy for organizations that 
> rely on the secrecy of their network topologies as an essential 
> security component. We're trying to increase the trust in email, 
> not reduce it.

Agree.  This is security through obscurity, and generally promoted by 
people who can't explain how it really helps.
>
>  draft-josefsson-email-received-privacy mentions the issue of 
> senders' locations appearing on mailing lists and in mailing list 
> archives. I have long felt that we are conflicted on whether the 
> output of a mailing list is a new message or the same as the one 
> sent to the mailing list. It usually has a different MAIL FROM 
> address, and often has text added to the message body, which I 
> would think is enough of a change to make it a new message. Yet the 
> Message-ID and Received header fields are preserved. I would think 
> that an entire new message should be created, a new Message-ID 
> assigned, and DKIM signed by the mailing list's domain (of 
> course!). Only selected header fields would be transferred to the 
> new message. The original incoming header fields should be 
> available only to the list administrators, who deal with abuse 
> issues.

This assumes that each mailing list has the best possible spam 
filtering and does the best possible job detecting forged header 
fields.  Since that isn't the case, we need to permit each mailing 
list subscriber to see how a message was sent to the list.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
When I was younger, I could remember anything, whether it had
happened or not.                                --Mark Twain

---

From: Martijn Grooten <[email-masked]>
Date: Mon, 30 Nov 2015 19:28:41 +0000

> getting spammed, phished, and malware'd.

We're actually pretty good at fighting various kinds of malicious email
and while I know headers like Received do play a role there, I'm not yet
convinced that their role is so fundamental that removing them, or
removing essential parts of them, would see these (proactive and
reactive) defenses collapse.

And I do actually think there is a privacy issue here and that we
shouldn't dismiss that right away.

However, I agree with others that the proposed charter seems a little
premature and that, to quote Dave Crocker, we ought to have "public
discussion about the technical aspects of this policy tradeoff" first.

Martijn.

---

From: Ned Freed <[email-masked]>
Date: Mon, 30 Nov 2015 09:50:58 -0800 (PST)

> Why would you want to check the Recieved: header fields in this case?   Under
> what circumstances would checking the header fields in a message known to have
> been received from aol.com, with a valid DKIM signature, add value over the
> validation you're already stipulating has happened?

Because vast amounts of spam comes through MSPs like aol.com, all nicely DKIM
signed. And of course you can't block aol.com completely - although I admit
it's really tempting sometimes.

> I think we may be talking about different things.   If you are trying to
> identify spam, you wouldn't need to do this for mail received through aol.com.  

Except that sometimes you do. See above.

> I'm talking about the case where someone is committing some kind of illegal
> behavior that triggers search warrants or other requests for records.  In this
> case, lawful requests can be satisfied by searching logs, and could not be
> satisfied by looking at the text of a message.   The data in such a message
> could not be considered trustworthy.

When did the needs of law enforcement become part of this?

Anyway, it is of course the case that logs are the best evidence. But logs
aren't always available, and if they are available they may not be accessible,
in which case you make do with what you have.

> Why would that be useful?

Oh, I don't know, perhaps to find out where gaps exists in privacy coverage
and patch them up? Service providers have proven to be fairly responsive
to customer requests for SSL/TLS protection.

> Yes, let us think of the children!

And with this I'm done. It is now quite apparent that not only do you have zero
clue about the operational realities of modern email systems, you aren't
interested in correcting your many misconceptions. Given that, there is
no point in continuing this discussion.

				Ned

---

From: Ted Lemon <[email-masked]>
Date: Mon, 30 Nov 2015 20:07:35 +0000

> it's really tempting sometimes.

I think you misunderstood my question.   What I mean is, if the spam gets to you through aol with a dkim signature, what good does it do you to check the Received: headers?   I'm not disputing that spam comes through AOL!

> When did the needs of law enforcement become part of this?

Dave Crocker brought it up.   It's a concern: people do commit crimes (e.g., harassment) using email, and being able to get close to the source of the email can be helpful in those cases.

> in which case you make do with what you have.

Sure, absolutely.   But whether we like it or not, absent the logs, we may simply not have the information because large mail providers are making the decisions without waiting for us.

> to customer requests for SSL/TLS protection.

I think this is certainly interesting from a research PoV, but has pretty much zero applicability to everyday users of email.   That said, I don't think anybody has proposed to remove indications of whether SSL is used from Received header fields, so perhaps this is a non-issue.

> no point in continuing this discussion.

What I am not interested in is being told that I shouldn't be allowed to have an opinion about the privacy implications of Received: headers because I am more of a mail user than a mail operator.   I'm sorry that you find that position problematic, but I think that your response here is unfortunate.

---

From: Ted Lemon <[email-masked]>
Date: Mon, 30 Nov 2015 22:51:28 +0000

> in distinguishing good from bad

Very true.   And it's likely that there is some percentage, P, of messages that are prevented from being delivered because that information is in the Received: header field.   If P is 99, obviously we'd better not take that information out of the Received: header field.   If it's .0001, then this isn't a good argument for retaining the information.

> the email (or do you think that DNS results are constant for all time?)

Anything that involves a volitional act on the part of some person is an exception.   The interesting question is, does SPF provide me any greater effectiveness in my system's automatic behaviors.   The answer is yes.

> known to those other people.

Why would I be relaying mail on?   Only for a mailing list.   For the mailing list, what I want SPF to validate is that the mail came from the mailing list.   I'm relying on the mailing list provider to filter spam in this case; any secondary spam filtering that I do on the tail end is going to be compromised by the loss of information when the mail goes through the list relay.   Still might be worth it, but what is the value of P in this case, for a general solution, as opposed to a one-off heuristic?

> that's a shame, I consider myself very trustworthy and I've never

Right, and because you don't use SPF, I can't whitelist you, because anybody can send me mail claiming to be from you, and I have no prima facie basis for rejecting it (unless you use DKIM, I suppose).  The problem isn't that _you_ aren't trustworthy: it's that your system for delivering mail to me isn't.  I could maybe study your headers over time and come up with some heuristic or machine-learning system that would distinguish between you and an imposter, but you're making me do a hell of a lot of work when setting up SPF is really quite easy.

---

From: Dave Crocker <[email-masked]>
Date: Mon, 30 Nov 2015 15:02:37 -0800

On 11/30/2015 2:51 PM, Ted Lemon wrote:
> Why would I be relaying mail on?   Only for a mailing list.   For the mailing list, what I want SPF to validate is that the mail came from the mailing list. 

That's a common misconception.  It leaves out a variety of other, valid
scenarios.

The most obvious is mailbox aliasing, such as for vanity addresses such
as university alumni associations provide.

Email is store and forward, and this can and does mean transit across
/multiple/ independent administrative domains.  The fact that the vast
majority of mail goes directly (one hop) from origin AD to the
recipient's AD does not mean it is reasonable for anyone to make
systemic design decisions that constrain that fundamental flexibility.

IMO an essential design benefit in many/most aspects of Internet
technologies is avoiding making any more global assumptions (or
requirements) than essential.  "Deferring to the end systems" is a very
broad-based design requirement and it includes minimizing assumptions
about the transit infrastructure.

d/

---

From: Ted Lemon <[email-masked]>
Date: Mon, 30 Nov 2015 23:06:17 +0000

> as university alumni associations provide.

That's a mailing list with one subscriber.   Same scenario.

> systemic design decisions that constrain that fundamental flexibility.

Sure, but in order for this argument to apply to the present discussion, we would have to be able to show that what is being proposed would constrain that fundamental flexibility, and I don't think it does.

> about the transit infrastructure.

Sure, but in this case wouldn't deferring to the end systems argue in favor of allowing end systems to make the decision as to whether their private information should be exposed?

---

From: "John Levine" <[email-masked]>
Date: 1 Dec 2015 02:03:55 -0000

>Sure, but in this case wouldn't deferring to the end systems argue in favor of allowing end systems to
>make the decision as to whether their private information should be exposed?

You can expose whatever you want, but if you expect anyone to accept
your mail, you will follow the rules set by the recipient systems.
This has been the way that mail has worked for many decades, and it's
not going to change now.

For a lot of the changes proposed here, the practical effect would be
that anyone foolish enough to implement them would find that the set
of systems accepting their mail shrinks dramatically.  For anyone
planning self-righteous complaints that you HAVE to accept my mail
because I'm following some standard or other, don't bother.  See, for
example

https://jl.ly/Email/compliant.html

R's,
John