Subject: [Asrg] antiphishing idea

From: Christian Grunfeld <[email-masked]>
Date: Thu, 17 Nov 2011 15:30:07 -0300

Hi,

I dont know if this is exactly the right place for discussing my idea
but I want to do a little bit of brainstorming with experts. I come
with a "crazy" thing but the logic is not so bad ! Please dont tell me
"you are crazy" ! .....at least at first time! :p

Users believe in what they read in the header from of the mail ! Users
don't also know about the existence and differences between envelope
and header From. And they also don't know that those addresses can be
forged. They blindly believe !

My idea is to invert the logic of DNSBLs. That is, instead of asking
third parties about spam/phishing why not asking the domain involved
in envelope and header itself about non-spam ?

Domains should have to publish in their DNSs the message-id (among any
other thing) through a TXT or A record of any legit mail sent by them.
The TTLs of those records can be adjusted to compensate for queued
mails, etc.

When you receive a mail from A and "aparently" from B you can query A
and B DNSs looking for the message-id the mail has. If you have a
nxdomain or whatever error from them you can score the mail as
phishing! ..on the other hand if you have a hit from at least one of
them you can be confident that this is the real domain that sends that
mail or it sends it on behalf the real address!

The check against both is to account for multi identities in what one
mailer sends in behalf of another (like gmail). You can also check
against any internal hop in the Received: chain in order to avoid
breaks in the trust chain by means of plain forwarding (as happens in
SPF without SRS).

I know this requires all to implement it in order to work (also like
SPF and DKIM).

Time to think !

Cheers !

---

From: Martijn Grooten <[email-masked]>
Date: Thu, 17 Nov 2011 19:58:19 +0000

> The TTLs of those records can be adjusted to compensate for queued

I take it you mean the message-id is somehow converted into a domain name and a DNS lookup is performed against that DNS and if the A or TXT record for that domain is some kind of affirmative value, this means that this is a 'valid message-id'?

I've got many doubts about this: do people who fall for such scams care about the value of From? Doesn't this have the same issues as SPF -all and ADSP? Does this scale? But my most important concern is: wouldn't it be trivial to perform a relay attack using this method?

> mail or it sends it on behalf the real address!

I don't get this. Are you saying that if I send an email with MAIL FROM: example.com and Header From: paypal.com and the message-id on example.com that I can then get this message-id validated simply because I happen to control example.com?

Martijn.

Virus Bulletin Ltd, The Pentagon, Abingdon, OX14 3YP, England.
Company Reg No: 2388295. VAT Reg No: GB 532 5598 33.

---

From: Chris Lewis <[email-masked]>
Date: Thu, 17 Nov 2011 16:38:37 -0500

> I take it you mean the message-id is somehow converted into a domain name and a DNS lookup is performed against that DNS and if the A or TXT record for that domain is some kind of affirmative value…

I got the impression from the original that he was not doing that, and
looking up the _whole_ Message-ID.

Which implies rather more tightly coupled MTA/DNS servers than is
probably feasible.  High volume mail servers would have a horrendous
problem with DNS bloat and at the same time, require very low TTLs.  Ouch.

Even if you only used a domain name same problem.

There's no easy transition either - you'd never know if whom you were
doing these queries actually implemented this stuff, and NXDOMAIN could
mean either "BAD! BAD! discard!" or "I don't implement that feature".

> I've got many doubts about this: do people who fall for such scams care about the value of From? Doesn't this have the same issues as SPF -all and ADSP? Does this scale? But my most important conce…

It's trying to make "SPF -all" survive forwarding, but not as well as
DKIM would (absent message mangulation).

> I don't get this. Are you saying that if I send an email with MAIL FROM: example.com and Header From: paypal.com and the message-id on example.com that I can then get this message-id validated simp…

He's suggesting asking _both_ A and B.  So both paypal.com and
example.com would have to say "Yup, I know that Message-ID".

So, you'd have to control example.com and have one (presumably recent)
sample Message-ID from paypal.com, and you could send as many as you
want with any contents you want.

[Few (if any) MTAs enforce unique Message-IDs on inbound email.]

Neither SPF or DKIM are perfect, but it behooves us to not try to
bandaid their defects with something like this.  Patching a hole in a
submarine with window screening comes to mind as a suitable analogy ;-)

---

From: Christian Grunfeld <[email-masked]>
Date: Thu, 17 Nov 2011 19:04:40 -0300

> I take it you mean the message-id is somehow converted into a domain name and a DNS lookup is performed against that DNS and if the A or TXT record for that domain is some kind of affirmative value…

message-id is already close in that form. It has
something_random_unique_time_process@domain

> I've got many doubts about this: do people who fall for such scams care about the value of From? Doesn't this have the same issues as SPF -all and ADSP? Does this scale? But my most important conce…

SPF checks against envelope MAIL FROM: ...users dont know even that
this thing exists ! they blindly trust on mail header From:
How would be a relay attack? evils dont know every random message-id
of a mail server legit mail. Are they going to query "every" random
string? ....now you can query what you want on a DNS server...what is
the difference?

> I don't get this. Are you saying that if I send an email with MAIL FROM: example.com and Header From: paypal.com and the message-id on example.com that I can then get this message-id validated simp…

sorry, I meant both of them ! you are obviously going to get the
validation from the evil domail but you also need one from phished
domain. The last one is only true if the mail was sent by paypal.com

---

From: "John Levine" <[email-masked]>
Date: 17 Nov 2011 22:13:46 -0000

>The TTLs of those records can be adjusted to compensate for queued

Maybe I'm missing something, but why would you want to do this rather
than a DKIM signature?  DKIM validates against the DNS, and protects
the whole message.

R's,
John

---

From: Christian Grunfeld <[email-masked]>
Date: Thu, 17 Nov 2011 19:19:12 -0300

> than a DKIM signature?  DKIM validates against the DNS, and protects

because I can have a proper configured domain, I can properly sign my
mails but I can send you an email with From: paypal.com header !
DKIM does not protect you against this ! DKIM says that evil.com signs
correctly the email and no alarms will trigger. The average user only
see and "trust" the From: paypal.com mail header !

---

From: Christian Grunfeld <[email-masked]>
Date: Thu, 17 Nov 2011 19:25:54 -0300

> looking up the _whole_ Message-ID.

yes

> mean either "BAD! BAD! discard!" or "I don't implement that feature".

same as SPF and DKIM. They are more or less good if all implement them !

> DKIM would (absent message mangulation).

DKIM is good for forwarding but it does not solve phishing !

> example.com would have to say "Yup, I know that Message-ID".

yes, that is the idea. Already answered in previous mail

> [Few (if any) MTAs enforce unique Message-IDs on inbound email.]

well...a good point to investigate

> submarine with window screening comes to mind as a suitable analogy ;-)

ok, not bandaid.....let´s reinvent SMTP with 3 way handshake ! that is
a more suitable analogy for my idea !

---

From: Steve Atkins <[email-masked]>
Date: Thu, 17 Nov 2011 14:29:43 -0800

> The TTLs of those records can be adjusted to compensate for queued

The obvious flaw with a simple version of this proposal is replay
attacks. Once someone has sent me an email, I can send email "from"
them just by reusing the message-id.

One obvious fix to that is to allow each message-id to be used and
validated exactly once. The first MTA to see that message-id considers
the message to be valid, anyone after that doesn't. That's very similar
to the goodmail token approach.

Another obvious fix would be to include a timestamp and invalidate
messages that are "too old" - but the window in which you'd expect
to see legitimate delayed messages is longer than the time you'd expect
replay attacks to be attempted.

Another obvious fix is to tie the message-id to the content of the message
in some way, by including a hash of the message and a subset of the
headers. That would work quite well, but still leads to the operational
problem of requiring a tight binding between the sending MTA and the
DNS server for that domain, and the significant problem of it defeating
the caching that DNS provides, leading to both high traffic DNS queries
and a likelihood of DNS lookup failures (and hence delayed or lost email).

One way to work around that would be to look up not a unique token from
the message-id for a simple match, rather to look up a substring of the
message-id to retrieve a cryptographic key that could be used to validate
the rest of the message-id. That key entry would be cached well by
DNS and would work pretty well. Replace message-id with DKIM
headers and you have DKIM.

The less obvious flaw is that you're trying to combat phishing by "protecting"
the From: header by requiring it be authenticated, and requiring all
email using that From: domain be sent through a small number of
"trusted" MTAs.

That's exactly what ADSP attempts to do, and it's been shown to
cause fairly serious loss of legitimate email (because it's extremely
rare that you really know all legitimate email is sent from just the
servers youo think it is). And it's been shown not to work terribly
well against phishing, as there's not enough information in a
From: header to tell you if the mail is legitimate. If I register the
domain paypal-billing.com, send mail From: it, with valid-by-your-scheme
message-id headers then it's 100% legitimate email as far as
recipients are concerned. And it is legitimate - it's just not from
paypal.

Cheers,
  Steve

---

From: Michael Thomas <[email-masked]>
Date: Thu, 17 Nov 2011 14:30:01 -0800

On 11/17/2011 02:19 PM, Christian Grunfeld wrote:
> 2011/11/17 John Levine<[email-masked]>:
>    

Ask google if an improperly dkim-signed paypal.com mail sets off
alarms. I'll help you out: it does.

Mike

---

From: Seth <[email-masked]>
Date: Thu, 17 Nov 2011 18:27:59 -0500 (EST)

> domain. The last one is only true if the mail was sent by paypal.com

No, it's true only if _any_ message was sent by paypal.com with that
Message-ID.  Copying a Message-ID isn't difficult, nor is getting
legitimate mail from paypal.

Seth

---

From: Chris Lewis <[email-masked]>
Date: Thu, 17 Nov 2011 18:36:24 -0500

> same as SPF and DKIM. They are more or less good if all implement them !

Ah, sorry, no.  If the sender doesn't implement SPF or DKIM, you find 
out that it doesn't support it by not having an SPF or DKIM record, 
thus, you don't apply either.

In your proposal there is no way to tell whether a given server 
implements it, AND the result for "FAIL" is indistinguishable from the 
result for "NOT IMPLEMENTED".

In other words, you can only turn this feature on if everybody 
implements it.  At the same time.

That won't happen.

You have to add other transactions to be able to query if the server 
supports this feature.  If the forwarder doesn't (by mal or mis 
feasance), then it doesn't do anything useful.

> DKIM is good for forwarding but it does not solve phishing !

Actually, it does, if the From is signed (which it usually is), and if 
the receiver knows (by some outside arrangement) to throw away anything 
that fails.

What it doesn't do is solve "froggery" (myriads of look-alike domains, 
which your proposal does not solve) nor does it solve the fact that most 
of the time the user doesn't even notice what's in the From: line (nor 
does your proposal).  Remember that many mail readers (outlook included) 
DO NOT SHOW YOU THE EMAIL ADDRESS in the From: line by default.

Sigh.

APWG just published a report that said that less than 20% of phishes had 
the correct phished-for domain in the From line...  Cutwail is pushing a 
lot of such things right now.

---

From: Christian Grunfeld <[email-masked]>
Date: Thu, 17 Nov 2011 21:18:12 -0300

> Ah, sorry, no.  If the sender doesn't implement SPF or DKIM, you find out
> that it doesn't support it by not having an SPF or DKIM record, thus, you
> don't apply either.
>
> In your proposal there is no way to tell whether a given server implements
> it, AND the result for "FAIL" is indistinguishable from the result for "NOT
> IMPLEMENTED".

> In other words, you can only turn this feature on if everybody implements
> it.  At the same time.
>
> That won't happen.
>
> supports this feature.  If the forwarder doesn't (by mal or mis feasance),
> then it doesn't do anything useful.

ok, put a TXT RR in the DNS telling "I am using the proposal" as SPD and DKIM do

> Actually, it does, if the From is signed (which it usually is), and if the
> receiver knows (by some outside arrangement) to throw away anything that
> fails.

anything that fails what? a signed email does not solve forgery of the
headers IN THE ORIGIN ! it solves forgery IN TRANSIT !
I can send an email to this list with my gmail account (wich goes
properly signed) claiming that is from you ! do you want me to try ?

> What it doesn't do is solve "froggery" (myriads of look-alike domains, which
> your proposal does not solve) nor does it solve the fact that most of the
> time the user doesn't even notice what's in the From: line (nor does your
> proposal).  Remember that many mail readers (outlook included) DO NOT SHOW
> YOU THE EMAIL ADDRESS in the From: line by default.

my proposal does not need the user see the address...it checks
silently on behalf the user.

---

From: "John Levine" <[email-masked]>
Date: 18 Nov 2011 06:59:07 -0000

>DKIM does not protect you against this !

I get the impression that you haven't read the very, very, very
extensive discussion on the DKIM list about how DKIM and/or ADSP do or
do not "protect" the address on the From: line, and what if anything
to do about it.  Please do so.  The archives are here:

http://mipassoc.org/pipermail/ietf-dkim/

It would also be worth taking a look at the archives of the earlier
MARID WG which also considered similar issues.  Its archives are here:

http://www.imc.org/ietf-mxcomp/mail-archive/

R's,
John

---

From: Paul Smith <[email-masked]>
Date: Fri, 18 Nov 2011 10:26:54 +0000

On 17/11/2011 23:27, Seth wrote:
> Christian Grunfeld<[email-masked]>  wrote:
>

So, you put the message-id AND recipient in the source's database

If evil spammer/phisher gets Paypal to send him a message, then reuses 
that message-id in their own messages out, it would work with the 
original idea, but if you add the recipient to the data, you defeat that.

---

From: Paul Smith <[email-masked]>
Date: Fri, 18 Nov 2011 10:25:11 +0000

> silently on behalf the user.

Unfortunately, all it will check is that the from address is OK-ish 
according to the sender's domain.

It doesn't check that that message that says 'your paypal account has 
been compromised, please enter your login details on 
http://badpeople.com/giveusyourmoney.php' is actually from Paypal. The 
'From' address, and the return path could be [email-masked]', 
and some people would still click on that link and enter their login 
details.

The biggest reason phishing works is that some (many?) people become 
very stupid as soon as you put a computer in front of them. Good luck 
with finding a solution for that... :) Ask them in the street for their 
password, and they'll tell you where to go, but send them an email with 
a 'keyword' in (eg 'Paypal', or their bank's name), and they'll tell you 
everything you want to know.

Actually, as an anti-forgery technique, which may help spam filters when 
determining reputation, I think your idea would help (I know others 
would disagree), but as a specifically anti-phishing technique, I think 
human stupidity will beat it.

---

From: Martijn Grooten <[email-masked]>
Date: Thu, 17 Nov 2011 22:24:07 +0000

> string? ....now you can query what you want on a DNS server...what is

Sorry, I meant rePlay attack. You get hold of one 'valid' messagge-id and use that for all your phishing messages.

> domain. The last one is only true if the mail was sent by paypal.com

So if I were to use Gmail to send mail on behalf of a example.com, a domain which I hypothetically have an email address at (something which has worked well for years), I'd have to validate the message ID of every email I want to send at example.com?

Martijn.

---

From: Martijn Grooten <[email-masked]>
Date: Fri, 18 Nov 2011 19:26:05 +0000

> lot of such things right now.

Indeed. What the OP tries to solve is something that is only relevant to those phishing emails that:
- are not already blocked by spam filters;
- are not already thrown away as 'phishing' (or simply: as 'not my bank') by the recipient;
- show the From: address in the victim's MUA;
- do use the phished-for domain (and not a subtle variant thereof) in the From:;
- have a victim that actually checked that From: line and concluded it was legitimate because of that.

How many emails are we talking about annually? And, putting the fact that it may not work well or have unwanted side-effects aside, what would the cost of these changes, extra lookups etc. be?

Martijn.

---

From: Chris Lewis <[email-masked]>
Date: Fri, 18 Nov 2011 20:00:46 -0500

> So, you put the message-id AND recipient in the source's database

I don't want to think about the privacy issues with that.

---

From: Bill Cole <[email-masked]>
Date: Sat, 19 Nov 2011 01:54:04 -0500

> see and "trust" the From: paypal.com mail header !

The absurd idea of publishing valid MID's in DNS does nothing to stop this. 
The average user doesn't see MID's either, so a phish message can have an 
evil.com Message-ID and a paypal.com From header.

The reason the idea is absurd (beyond being utterly worthless as a practical 
matter) is that MID headers are frequently generated by MUA's that do not 
have any way to communicate the MID to the DNS authority for the domain part 
they use and frequently use domain parts that they really should not. For 
example, <[email-masked]> is a MID I recently used on a 
perfectly valid piece of email. Thunderbird generated it. A few seconds 
later I used another MUA to send an identical piece of mail with the MID 
<[email-masked]>. If I logged into GMail and 
sent another copy, it would be some very long local part @mail.gmail.com. I 
could legitimately send an identical message (except for the MID) with my 
GMail address as the From via another SMTP submission system and it would 
get a MID in the domain of the authentication identity I use with that 
sending system OR in gmail.com OR under the hostname of the submission host 
depending on which MUA I use. For many years I used a MUA which by default 
used an IP literal as the domain part of all of its MID's, which can be a 
reasonable tactic in some common circumstances.

At no point does any current MUA have to inject a new DNS record anywhere 
when it constructs a message. A MID is supposed to be globally unique, but 
it isn't really possible for any MUA to assure this perfectly since there is 
no strict standard for how a MID domain is selected or how a MID local-part 
is generated. There is no certainty that a MUA will submit mail through a 
system that will be related to the domain part of the MID.

The next problem is scale. It is not uncommon for a middling company to 
generate many thousands of messages daily with peak rates of multiple 
messages per second, while changing the externally visible DNS zone for the 
domain used in the MID and From on most messages very rarely, perhaps less 
than once in a year. Such an organization would need to make major 
infrastructural changes to deploy your idea, and might well see many (3? 5?) 
orders of magnitude more DNS query traffic on a domain they would now need 
to provision in a radically different way.

This is a concept that shares the fundamental flaws of SPF because it 
presumes that people will change how they send mail to adhere to an 
authentication protocol they know nothing about. It supplements that by 
requiring all MUA's to change and by requiring all domain owners whose users 
send mail to deploy new DNS infrastructure which will in many cases require 
new functionality (authenticated dynamic updates) which most domains do not 
currently use. All for a form of authentication that is inherently weak and 
does nothing that existing authentication mechanisms could do if it weren't 
so meaningless.

You will note that at no point have I suggested that you are crazy.

---

From: Bart Schaefer <[email-masked]>
Date: Sat, 19 Nov 2011 04:32:13 -0800

On Nov 19,  1:54am, Bill Cole wrote:
}
} The reason the idea is absurd (beyond being utterly worthless as a    
} practical matter) is that MID headers are frequently generated by     
} MUA's that do not have any way to communicate the MID to the DNS      
} authority for the domain part                                         

Wouldn't it be the MTA that would be responsible for grabbing fields
out of the header during transmission and adding them to DNS?  This
would require that the MUA or MSA only talk to an MTA that has that
capability for the domains in both the MID and From, which is a whole
different kettle of fish.

It'd be possible for a 3rd-party service to provide both the database
and the DNS for this, but that's essentially re-inventing Goodmail,
which already failed in the attempt.

---

From: Christian Grunfeld <[email-masked]>
Date: Sat, 19 Nov 2011 11:55:30 -0300

> system that will be related to the domain part of the MID.

I think you really dont know what is a MUA and a MTA ! MUAs SHOULD
submit emails to an MTA who is in charge of the transport for a
domain. Call them smarthosts or whatever you want.
MTAs for a domain should be close related to domain NS and could be
authorized to dinamically update records.

> to provision in a radically different way.

you really dont know ! nowdays for EVERY sent email there is a DNS
query...authoritatlvely or cached anywhere answered ! but there is ONE
!....SPF makes one more....DKIM makes one more...my idea makes one
more......how do you absurdely get 3 o 5 orders of magnitud more? do
you know what an order of magnitud is?

> You will note that at no point have I suggested that you are crazy.

sorry but you are stup.....

---

From: Paul Smith <[email-masked]>
Date: Sat, 19 Nov 2011 16:31:06 +0000

On 19/11/2011 06:54, Bill Cole wrote:
> The absurd idea of publishing valid MID's in DNS does nothing to stop 
> this. The average user doesn't see MID's either, so a phish message 
> can have an evil.com Message-ID and a paypal.com From header. 

You're missing the point here.

If you sent a message like:

Message-Id: <[email-masked]>
From: [email-masked]

The receiving MTA would look for something like 
'12345-gmail.com._msgids.paypal.com'

It looks up the message ID in a zone related to the From address, NOT 
the zone related to the message id.

(I'm not saying there aren't other reasons against the original idea, 
but this isn't one of them)

Personally, I like the idea of a callback to validate that mail is what 
it says it is, but I'm not sure about using DNS as the callback mechanism.

> authentication protocol they know nothing about.
This is what I have problems with. Email at the moment is fundamentally 
flawed, because people send mail assuming totally lax authentication. 
10-15 years ago people used open relays and '% forwarding' all the time, 
but that changed, and people adapted. During the changeover period, lots 
of messages bounced, and people got frustrated (I had to deal with them 
all the time), but people quickly learned the 'right way' to do things 
(because they HAD to), and things improved.

  As long as the 'new authentication' method is implemented strongly, 
then people will adapt. IMV, the reason SPF doesn't work ATM is because 
even people who check it don't reject mail when they should. This lets 
badly configured systems 'sort of work' rather than failing totally. 
'sort of working' systems often stumble on with people getting 
frustrated, and maybe blaming the recipient rather than the sender. 
Failing totally would make the senders sort their stuff out.

Any new authentication method will have to make people change the way 
they work. The challenge is really finding one which will work 
effectively enough to make it worthwhile to need the changes. Rejecting 
solutions because they require changes to the way people work would have 
meant everyone would still be running open relays nowadays.

---

From: Bill Cole <[email-masked]>
Date: Sat, 19 Nov 2011 13:07:54 -0500

> different kettle of fish.

It would also be a radical narrowing in the definition of legitimate mail 
submission.

As it stands, it is a sound practice to use unrelated domains for the SMTP 
envelope sender and From/Sender/Reply-To/Message-ID headers. SPF was created 
with the hope that mail transport routing could be controlled enough to 
allow repudiation of mail at the transport layer, but that has proven to be 
a failure in general application. Only a small number of special-function 
domains can safely use "-all" in their SPF records.

Shorter: This mechanism would force some people to change how they submit 
mail. SPF has demonstrated that to be a crippling flaw.

> which already failed in the attempt.

We are at the point in mail authentication where there is very little space 
left for truly new mechanisms that aren't ultimately equivalent to existing 
limited successes like SPF and DKIM or past failures like Goodmail.

Anyone who wants trustworthy mail authentication and is willing to sacrifice 
the inherent functional and overhead costs of that ought to know where to 
find S/MIME and PGP. With SPF and DKIM broadly but not universally deployed 
it is hard to seriously consider proposals that buy slightly more 
authentication at the price of slightly constricting how people use email 
and requiring mail providers to set up substantial new infrastructure. From 
what I can see, there is very little space for concrete improvement from 
mail authentication without requiring smarter users and/or the end to 
accepting mail from random well-meaning strangers. The former is a pipe 
dream, the latter would be a tragedy.

---

From: "John Levine" <[email-masked]>
Date: 19 Nov 2011 19:05:13 -0000

>the zone related to the message id.

I think we need a new category of FUSSP for anything that requires the
sender to maintain a public per-message database of all the messages
it's sent.  To put it mildly, this really REALLY doesn't scale.

R's,
John

PS: I'm not saying this is the only reason that looking up the message-id
isn't a good idea.

---

From: Steve Atkins <[email-masked]>
Date: Sat, 19 Nov 2011 11:44:28 -0800

> it's sent.  To put it mildly, this really REALLY doesn't scale.

As a thought experiment, if every sender offered a POP3-esque service
rather than sending mail by SMTP, it would require about the same amount
of resources as the receivers are providing today to manage that same
traffic, and that's not unreasonable.

A database of just recent messages sent would require less resources than
that.

Cheers,
  Steve

---

From: Bill Cole <[email-masked]>
Date: Sat, 19 Nov 2011 14:47:55 -0500

Please fix your MUA to respect Reply-To headers.

Christian Grunfeld wrote, On 11/19/11 9:55 AM:
>> example,<[email-masked]>  is a MID I recently used on a
> I think you really dont know what is a MUA and a MTA !

I understand both fairly well, but I appreciate your concern for my 
understanding. It is unfortunate that there seems to be a language barrier 
preventing you from coherently explaining my ignorance to me.

> MUAs SHOULD
> domain.

That is something many people have been arguing about for many years. If you 
dig through the Google Groups archive you can find me going into the 
question at length in the late 1990's. I was wrong then because I thought 
that a mechanism equivalent to SPF would force people to use well-defined 
submission and transport paths tightly bound to the domains they use in 
their email addresses. SPF in widespread deployment has proven me wrong on 
that point. It is somewhat quaint to see someone still insisting on that 
sort of binding.

> Call them smarthosts or whatever you want.

It would be useful for you to think about how you are using the words 
"should" and "could."

In a previous job I managed mail servers for the US subsidiary of a major 
global conglomerate headquartered in Germany. Due to control fetishists in 
Berlin and Frankfurt and some corporate asset shuffling between parent 
companies, some of the domains used by US employees had their authoritative 
DNS in very stable zones on machines in Germany. At one point a US IT 
manager insisted that we put SPF records in for one of those domains. That 
took 6 weeks, because no one in the US had been informed when 
responsibilities were shuffled in DE. Less than a year later when we were 
migrating our mail servers to a new set of machines with new IP's, it took 3 
weeks to get the SPF records changed because it was August (when apparently 
virtually everyone in Germany is on vacation.) In getting that change done, 
I learned that the authoritative DNS was running on hardware and software 
that had not been updated since 1997, almost a decade of stability. The 
person who told me this was very proud of it.

The point of that story: in the real world, there are generally functional 
email and DNS systems that do not work anything like the way you think they 
should now and really could not be adjusted to work as you want them without 
significant modifications to the software, hardware, and humans involved.

> you know what an order of magnitud is?

Yes, or I would not have used the term. The above paragraph convinces me 
that we have a communication problem due to language.

It is not true today that every email message requires the creation of a new 
DNS record. This is not even true for domains that have deployed SPF and 
DKIM. It is quite possible for a domain to be stable for years with 
thousands of messages per day flowing from its users.

Query traffic for an authoritative DNS server due to mail can be trivial. 
Yes, there are queries as a result of each message sent, but today those are 
queries for records that can have TTL's of a day or longer. This means they 
are cacheable so that the authoritative server has query traffic that is a 
function of the diversity of recipients (i.e.how many DNS caches their MX 
hosts use in aggregate) rather than the number of unique messages. For large 
mail systems this can mean that the queries to their authoritative servers 
tops out in the low thousands per day that is attributable to mail. If there 
is to be a mechanism that requires a unique query per message by the MX host 
for a recipient, the authoritative server will get no help at all from the 
existence of DNS caches. I've worked with domains whose mail targets were 
focused enough that they got less than a thousand SPF queries per week for 
over a million messages sent, so 3 orders of magnitude boost in queries is 
definitely possible if there were to be a query per message. 5 OoM may be a 
stretch, but maybe not for a big bulk sender.

> sorry but you are stup.....

OK, that's not a language issue. We're done here.

---

From: Bart Schaefer <[email-masked]>
Date: Sat, 19 Nov 2011 12:23:08 -0800

On Nov 19, 11:44am, Steve Atkins wrote:
}
} As a thought experiment, if every sender offered a POP3-esque service
} rather than sending mail by SMTP

How is that different from DJB's IM2000?

Cf. http://en.wikipedia.org/wiki/StubMail

---

From: Steve Atkins <[email-masked]>
Date: Sat, 19 Nov 2011 12:27:07 -0800

> Cf. http://en.wikipedia.org/wiki/StubMail

One was put forward as a thought experiment, to help estimate
system resource usage, and will never be deployed.

The other will just never be deployed.

Cheers,
  Steve

---

From: "John Levine" <[email-masked]>
Date: 19 Nov 2011 20:31:18 -0000

>How is that different from DJB's IM2000?

It's more like RSS.  But I take your point.

I think there's a difference due to the different update and query
patterns.  Anything that requires a high update rate and global
consistency is really hard and doesn't parallelize well.  That's why
ATM networks and airline reservations still use million dollar
mainframes rather than farms of PCs.

R's,
John

---

From: Chris Lewis <[email-masked]>
Date: Sat, 19 Nov 2011 15:35:46 -0500

> traffic, and that's not unreasonable.

It's called webmail, and I think it's true that you have to build out 
rather more infrastructure to support it than with a typical perimeter 
MTA/internal POP3-esque service.  The security implications are rather 
different as well.

Note also that per message database _traffic_ would scale with the total 
amount of email (around 80-95% spam), whereas POP and webmail 
infrastructure scales with the ham.

The per message database queries in DNS wouldn't be cacheable whereas 
about everything else in DNS is.  3-5 orders of magnitude increase in 
DNS traffic to the authoritatives isn't unreasonable in the least.

With large infrastructures doing messages to the tune of 100s or 1000s 
per second, usefully doing DNS updates at that frequency (and timely 
enough to be there when the receiver wants to query) seem entirely 
implausible to me.  Or you delay the email on the sending side to wait 
for the database to update... Ick.

A few thousand entries per second, with a hold-time of at least 4 days 
also seems a bit much for DNS to cope with - because _all_ your 
authoritatives would have to know them, and know them real fast, 
preferably before the email is sent.

---

From: Christian Grunfeld <[email-masked]>
Date: Sat, 19 Nov 2011 21:06:28 -0300

> preventing you from coherently explaining my ignorance to me.

I do not need to explain coherently your ignorance to you since you're
doing pretty well job yourself.

> It would be useful for you to think about how you are using the words

In my "bubble world" MUAs MUST submit emails to an MTA. Maybe not in
yours...that was why I´ve used "should".
SPF records tell the world which systems can send email for a
domain...so (as I said) MTAs are closely related to DNS servers for
that domain. With "closely related" I mean they must know each
other..then dynamic updates "could" be easily implemented !

> significant modifications to the software, hardware, and humans involved.

you 're talking about the Internet about 15 years ago. As someone
said...when using open relays was a "good" practice ! do not confuse
stability with no improvement !
No human intervention is needed to get dynamic updates to work. You
only need that NS trust you and accept your updates. And obviously if
those conditions are met you dont even need that NS are close to
you....they can stay in germany!

> thousands of messages per day flowing from its users.

It is not true, right ! but I did not say that. I think you're
confusing a DNS query with a creation of a RR !
I said that today for every email there is at least one DNS query. If
your server checks for a fqdn HELO or a correct map between IP and
names you have a query. If your server checks SPF you have another. If
your server checks DKIM you have another. Its true that some of them
can be cached.

> .... I've worked with domains whose mail targets were

Well, you've worked in many places !  you got SPF queries in the
manner that others implement it or not. It is not an indicator of
query volumes !

> OK, that's not a language issue. We're done here.

sorry about that

---

From: Paul Smith <[email-masked]>
Date: Sun, 20 Nov 2011 00:07:02 +0000

On 19/11/2011 20:35, Chris Lewis wrote:
> Note also that per message database _traffic_ would scale with the 
> total amount of email (around 80-95% spam), whereas POP and webmail 
Yes, unless it has such a beneficial effect that spammers give up...  
:)  (or at least stop forging messages from those domains protected by 
this system)

Also, obviously, message IDs could easily be constructed to spread the 
load among DNS servers

> DNS traffic to the authoritatives isn't unreasonable in the least.
Only the last part of the DNS queries wouldn't be cacheable AFAICT.

So, if the message ID was [email-masked]', translating to a 
lookup of something like '12345.bibble55.gmail.com_msgids.gmail.com', 
only the '12345' bit isn't cacheable. The resolution down to the 
bibble55.gmail.com._msgids.gmail.com. would be cacheable AFAICS, and 
would allow partitioning of the data & queries to the 'bibble55' 
authoritative DNS server. So, root servers and Google's 'normal' DNS 
servers wouldn't have significantly more load than now (if any at all), 
it would just be the msgid DNS servers which would be loaded.

The message IDs can easily be generated so that a particular DNS server 
is not overloaded. Even so that the zones go out of use after a few 
days, so the higher level DNS server just returns a non-existent domain 
(with a big TTL) for the 'bibble55' zone when they have all "expired", 
rather than having that DNS server still handling checks on old message IDs.

In fact, using DNS would appear to enable scaling to massive levels 
quite well because of its built in delegation capabilities.

> preferably before the email is sent. 

I'm not sure how that would be such a big problem. The DNS server 
obviously wouldn't be a basic Bind setup (having to reload zone files 
etc), or with ddns updates, it would be a "custom" DNS server feeding 
off a backend database which knows about the message data at the same 
time as the sending MTA (or even when the messages are queued for sending)

It is also possible that the message IDs could be generated 
cryptographically, so the DNS server doesn't even need to have a 
database of message IDs, but can check, with a reasonable level of 
confidence, that the message ID is valid.

Also, given that 90% of the email we receive via Yahoo's servers is 
spam, if the spam is cut, then the large infrastructures should be 
sending a lot less spam out, so the problem will be smaller ;)

My big problem with this idea is that I'm not sure how it would really 
help more than other ideas. I don't see the use of DNS as being that big 
an issue, it's just a UDP callback check in essence. It's massively more 
efficient than a CBV check, at least as efficient as some SPF checks 
(using macros), and miniscule compared to the actual SMTP transaction.

---

From: Bart Schaefer <[email-masked]>
Date: Sat, 19 Nov 2011 16:43:28 -0800

On Nov 19,  9:06pm, Christian Grunfeld wrote:
} Subject: Re: [Asrg] antiphishing idea
}
} 2011/11/19 Bill Cole <[email-masked]>:
} 
} > I learned that the authoritative DNS was running on hardware and software
} > that had not been updated since 1997, almost a decade of stability. The
} > person who told me this was very proud of it.
} 
} you 're talking about the Internet about 15 years ago.

No, he's talking about the internet 4 or 5 years ago.

"not since 1997" + "almost a decade" = approximately 2006.

} do not confuse stability with no improvement !

In fact that's *exactly* what he means.  Don't confuse "time has passed"
with "the situation has changed."

---

From: Chris Lewis <[email-masked]>
Date: Sat, 19 Nov 2011 20:17:09 -0500

On 11-11-19 07:06 PM, Christian Grunfeld wrote:
> 2011/11/19 Bill Cole<[email-masked]>:
>

Where's the C&C warning?

Bill can be somewhat abrasive, but ignorant in this area, he's most 
definitely not.

It's a bad idea to continue.

---

From: "John Levine" <[email-masked]>
Date: 20 Nov 2011 01:22:03 -0000

>doing pretty well job yourself. ...

I have moderated future posts from this guy due to his repeated
personal attacks.

We welcome robust debate, but not personal invective.

Do keep in mind that this list has members who have been running email
systems for a very long time, and it is rarely a winning strategy to
impugn the competence of someone who may well have been running mail
systems before you could read.

R's,
John

---

From: Andrew Kirch <[email-masked]>
Date: Sat, 19 Nov 2011 20:27:02 -0500

Wow, for once it wasn't me!

Andrew

---

From: Chris Lewis <[email-masked]>
Date: Sat, 19 Nov 2011 20:36:48 -0500

> Yes, unless it has such a beneficial effect that spammers give up... :)

We wish ;-)

> (or at least stop forging messages from those domains protected by this
> system)

APWG's latest stats seem to indicate that only 20% of phishes currently 
use the "right" domain.  Having that 20% switch to what the other 80% 
already do doesn't gain much.  The ROI seems pretty low.

[I'm not sure I believe that it's really as low as 20%, but the point 
still holds.]

> Only the last part of the DNS queries wouldn't be cacheable AFAICT.

That's all it needs to swamp current authoritive loading guidelines... 
Every outbound email results in a single uncacheable query, whereas 
under normal circumstances, _none_ do.

> it would just be the msgid DNS servers which would be loaded.

Right.  You could conceivably provide a NS coresident with every 
outbound MTA.  You could even build the NS _into_ the MTA.  However, 
even then, once you start factoring in the operational considerations 
(crash/stop/restart, load balance, availability, sheer size of zone) it 
still remains more than a little daunting.

> (using macros), and miniscule compared to the actual SMTP transaction.

Right.  For the reasons I put up near the beginning, the ROI seems 
extremely low.  Forces a small number of phishers to switch to a 
technique that the rest of the phishers are already using.