Subject: CSV specification revision available

From: Dave Crocker <[email-masked]>
Date: Mon, 14 Jun 2004 19:07:11 -0700

Folks,

We finally have a new version of the Client SMTP Validation
specification.   The changes are massive.

It is now 4 documents.

The master document is based on the CSV specification; it is now the
'systems integration service' specification. It carefully describes
motivation and scope.  An example in the Appendix shows a simple mode
of use that would be beneficial.

The new documents cover component functions of authentication,
authorization and accreditation.

Until they come out as internet drafts, you can get copies at:

      <http://brandenburg.com/current.html/#csv>

d/

ps. I am going on vacation for 5 weeks, as of tonight, and do not know
what my Internet access will be. Might be great, might not. In my
absence, of course my co-authors have my proxy. Since I won't b here,
I've sent the as draft-crocker-marid-*, rather than seeking official
working group status at this point, simply to ensure that they get
issued right away.

----- 
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA USA <tel: +1.408.246.8253>; <fax: +1.408.850.1850>

---

From: John Leslie <[email-masked]>
Date: Mon, 14 Jun 2004 23:10:48 -0400

>       <http://brandenburg.com/current.html/#csv>

   If you have trouble getting them there, feel free to try:

<http://www.jlc.net/MARID/CSV/>

--
John Leslie <[email-masked]>

---

From: Matthew Elvey <[email-masked]>
Date: Tue, 15 Jun 2004 00:02:32 -0700

><http://www.jlc.net/MARID/CSV/>
That new URL helped. 
Comments:  I don't get why draft-crocker-marid-csvhna-00.html says in
"2.1 Reverse DNS"
that "For authentication of sending SMTP clients, Reverse DNS can be 
used by itself"
This doesn't make sense to me.
e.g. Spammer controls 345.0.0.0/24 (including having rDNS delegated from 
its ISP), and makes 345.0.0.5 resolve in rDNS to mx34.aol.com, and sends 
spam with EHLO = mx34.aol.com.
This fails to protect aol.com or tie the abuse to a responsible party. 
Reverse DNS does not appear to be a good way to tie a domain to an IP.  
Also, allowing it doesn't make CSV easier to implement, does it?
"2.2 Forward DNS Lookup"
is adequate to the task; there's no need for 2.1, AFAICT.
An accreditation service used by CSV cannot accredit aol.com as a 
non-spammer if CSV does not allow aol to protect its use in HELO from 
this abuse.
Also, I don't see how 3.2 SMTP Auth protects aol.com from this same abuse.
RE. 3.1 StartTLS:  *IF* STARTTLS is used *AND* the sending server's cert 
is CA signed, then that makes sense.

(Yes, I plan to rip out part of SPF and propose it as a replacement for 
draft-crocker-marid-csvhna-00.html and 
draft-crocker-marid-csvcsa-00.html in CSV, and create a chart comparing 
the pieces chosen and the reason for each choice, all if I find the time.)

draft-crocker-marid-csvcsa-00.html won't work with wildcards, right?

---

From: John Leslie <[email-masked]>
Date: Tue, 15 Jun 2004 07:58:28 -0400

> that "For authentication of sending SMTP clients, Reverse DNS can be 

   We need to admit up-front that this document was the last one written,
and didn't get the same amount of scrutiny as the others. :^(

   That said, the whole intent of the Host Name Authentication document
was to survey the field for things currently being done; not to
recommend which of them to use.

> This fails to protect aol.com or tie the abuse to a responsible party. 

   You're right. This will be corrected.

   In truth, most actual use of Reverse DNS sets out to do both forward-
and reverse-DNS queries, looking for a match between them. In fact, that
is mentioned a few paragraphs earlier.

   I plead guilty to not proofreading the 2.1 section carefully enough.

> Also, allowing it doesn't make CSV easier to implement, does it?

   The host-name authentication function actually recommended is found
in the Client SMTP Authentication document, section 4 and following:
namely that the response to the SRV query should ordinarily list the
IP address(es) authorized to perform the MTA function; and that a match
of the actual IP address of the established SMTP connection to one of
these satisfies the authentication requirement.

   I shall ensure that either the overview document or the HNA document
makes it clear what is being recommended; and that the HNA document
explains the weakness of using reverse-DNS without verifying the
forward-DNS match.
   
> is adequate to the task; there's no need for 2.1, AFAICT.

   I agree 2.1 should not be mentioned without a warning of its weakness.

> non-spammer if CSV does not allow aol to protect its use in HELO from 

   You are correct that omission of host-name authentication, or the
use of inadequate host-name authentication renders accreditation by
domain-name virtually useless.

> is CA signed, then that makes sense.

   I'm not sure I understand your question here. Could you clarify?

> the pieces chosen and the reason for each choice, all if I find the time.)

   Sounds like a good execrcise.

> draft-crocker-marid-csvcsa-00.html won't work with wildcards, right?

   CSA was designed to validate the EHLO string, which should never
involve wildcards, IMHO. Thus "working with wildcards" was never a
design goal. (Please note that the Domain Name Accreditation methods
_are_ designed to work with wildcards.)

---

From: Tony Finch <[email-masked]>
Date: Tue, 15 Jun 2004 14:15:31 +0100

> specification.   The changes are massive.

The new documents are much clearer, however they are still vague on
important details, they take an awfully long time to get to the point, and
they contain an amazing amount of irrelevant digression. This is a shame
because I greatly prefer this approach to SPF and its ilk, since it does
not confuse email addresses and IP addresses.

I have a few comments:

draft-crocker-marid-smtp-validate-01

Section 1, Address-based Authentication:

It's probably worth explaining why this doesn't work so well in the
current Internet, particularly the use of NATs and dynamically-allocated
addresses. An IP address may be used by more than one host at a time or
over a period of time. E.g. if I blacklist a virus-infected zombie, have I
just blacklisted a NATted network? Will the zombie get through again after
it has obtained another DHCP lease? (in which case I'll blacklist the
whole DHCP range eventually)

Section 3.3

The CSV spec makes a nice distinction between authentication and
authorization. So why is the CSV authorization step defined by a document
called "Client SMTP authentication"? (Actually this appears to be a typo
since the document calls itself "Client SMTP authorization".)

draft-crocker-marid-csvhna-00

Section 2 DNS-based Mapping

"Often when contacted by a remote host, a host uses a reverse-DNS query to
get the name of the remote host." This should be changed to "names" plural
-- there can be multiple PTR records for a given IP address.

"Although [forward + reverse DNS consistency checking] has known
limitations, it is considered sufficient for many basic uses." This
comment needs a citation to a document contining a proper explanation of
the limitations.

I'm very dubious of the weaker DNS checking suggested in 2.1 and 2.2. I
think there should be much more discussion of the possible attacks and
defences, and why they are under consideration when full foward + reverse
checking is available.

The whole section needs to be more explicit about how the process works
and what the actions of the checker should be in reaction to various kinds
of failure.

3.1  StartTLS

Again this needs to be more explicit about the authentication process.

3.2 SMTP Auth

As I understand it this draft is intended to be independent of SMTP.
Therefore SASL should be cited instead.

In addition to this, SASL is generally used for user authentication not
hostname authentication. This section needs to be more explicit about how
this unusual use works in practice.

There also needs to be some comment on the limits to scalability of this
option, since it requires pre-existing bilateral agreements between all
senders and all recipients.

4 Independent Policy Services

"In effect, the question about domain name use is a question about the
reputation and accountability of the domain name administrator."

No, it is a question about the reliability of the host. This entire
section refers to a part of the CSV process covered by the DNS draft.

draft-ietf-marid-dsvcsa-00

Sections 4 & 5

These sections overlap rather a lot. They also omit to specify what the
server should do if the client's CSA records has invalid data. What should
go in the (reserved) port field? What does the SRV target mean for a
client? i.e. what should go in that field? Does it have to be the same as
the EHLO domain name if the client is authorized? If the client is not
authorized, this draft specifies that the EHLO name should go in the
target. Does it still have to have address records in accordance with RFC
2782?

6  Domain administrator advice

Paragraph three appears to be a discussion of SMTP server policy, not DNS.
It probably belongs in the DNA draft.

The protocol specified in this draft appears to be (I'm reading between
the lines here) that the EHLO domain name stated by the SMTP client is
used to look up a SRV record. This record states in the weight field if
that name may be used by an SMTP client, and it also contains a target
name. The DNS reply should contain additional data, being the IP address
records associated with this target name. One of these must match the
client IP address from the SMTP connection in order for the EHLO domain
name to be considered valid. Or maybe (according to HNA) the IP addresses
associated with the EHLO domain itself must match the connection IP
address. Or maybe the reverse DNS must agree too. This is not clear. It's
also not clear how this relates to the RFC 2821 nix on EHLO checks.

draft-crocker-marid-csvdna-00

There's no discussion of the obvious attack of a client recommending
accreditation services that all say the client is wonderful.

-- 
Tony Finch  <[email-masked]>  http://dotat.at/

---

From: John Leslie <[email-masked]>
Date: Tue, 15 Jun 2004 11:39:44 -0400

Thanks, Tony, for your extensive comments.

Tony Finch <[email-masked]> wrote:
> 
> The new documents are much clearer,

   Thank you. (There's been a lot of discussion leading to them.)

> however they are still vague on important details, they take an awfully
> long time to get to the point, and they contain an amazing amount of
> irrelevant digression.

   We mostly adopted the attitude that if any of us were confused, more
explanation was needed.

> I have a few comments:

   Some of these I need to defer to Dave Crocker. Alas, it will be
several days before we know _whether_ he'll have useful Internet access
at his vacation spot.

   Please note, when I "defer to Dave" it doesn't mean I'm unwilling to
discuss the issue -- merely that I want Dave to be responsible for
changes to the document in question.

> draft-crocker-marid-smtp-validate-01

   (This is the one document we tried to keep short.)

> whole DHCP range eventually)

   We discussed most of this, and Dave chose the terse wording:
} 
} Increased topological, transfer and access complexities...

   I'm going to defer to Dave whether to expand that.

> since the document calls itself "Client SMTP authorization".)

   This is a typo. It will be fixed.

> -- there can be multiple PTR records for a given IP address.

   I agree with you, but I'm going to defer to Dave on changes.

> comment needs a citation to a document contining a proper explanation of

   There wasn't time to chase references. I'm confident we'll add some.

> and what the actions of the checker should be in reaction to various kinds

   I agree with you, but I'd much prefer to defer to Dave on actual wording.

> Again this needs to be more explicit about the authentication process.

   I'm not sure I agree. RFC3207 is cited. But I'll defer to Dave.

> Therefore SASL should be cited instead.

   Defer to Dave.

> this unusual use works in practice.

   Defer to Dave.

> senders and all recipients.

   I agree, but I'll defer to Dave.

> section refers to a part of the CSV process covered by the DNS draft.

   I'm afraid we'll have to agree to disagree here. We intend for the
domain-name to be closely bound to those who administer the host --
not the machine itself. And we're more concerned about quick response
to problems than we are about the actual configuration of the host.

> These sections overlap rather a lot.i

   Indeed they do. We wanted to introduce the concept before the details.

> They also omit to specify what the server should do if the client's
> CSA records has invalid data.

   True. We're going to have to confer on what to say about these cases.

> What should go in the (reserved) port field?

   Zero. (I'm sure this was in a previous draft!)

> What does the SRV target mean for a client? i.e. what should go in
> that field?

   It should ordinarily be a domain-name (typically the same as the
EHLO string) that resolves to the correct list of IP addresses.

> Does it have to be the same as the EHLO domain name if the client is
> authorized?

   No, but it ordinarily will be.

> If the client is not authorized, this draft specifies that the
> EHLO name should go in the target. Does it still have to have
> address records in accordance with RFC 2782?

   We really don't care: the list of IP Addresses (if any) will be
ignored.

   I agree some more words are needed. We'll confer on that.

> It probably belongs in the DNA draft.

   It got written here: we'll confer on whether there's a better place
for it.

> records associated with this target name.i

   Correct.

> One of these must match the client IP address from the SMTP connection
> in order for the EHLO domain name to be considered valid. Or maybe
> (according to HNA) the IP addresses associated with the EHLO domain
> itself must match the connection IP address. Or maybe the reverse DNS
> must agree too. This is not clear.i

   For various reasons, we agreed not to talk about "authentication"
here. We consider it to be an "optimization" that the match of an IP
address here might satisfy the "authentication" need.

   Thus, there is no need for any IP checking in order for the _name_
to be authorized. I agree the wording could be clearer. We'll confer
on this.

> It's also not clear how this relates to the RFC 2821 nix on EHLO checks.

   I do believe we've been over this before...

   RFC2821 specifically allows error responses to EHLO, but forbids one
very particular case:
] 
] An SMTP server MAY verify that the domain name parameter in the EHLO
] command actually corresponds to the IP address of the client.
] However, the server MUST NOT refuse to accept a message for this
] reason if the verification fails

   We are not refusing "for that reason" (though, of course, it's
already quite common to refuse for that reason).

   I quite agree that a clarification of this issue probably belongs
in the published MARID RFC.

> accreditation services that all say the client is wonderful.

   I must disagree. See section 3, specifically the discussion of
"negative weighting", for example.

---

From: Tony Finch <[email-masked]>
Date: Tue, 15 Jun 2004 16:53:43 +0100

> to problems than we are about the actual configuration of the host.

OK, but how does the section relate to authenticating that the host you
are talking to is permitted to use the name it has claimed? There's no
description of a mechanism for doing that.

> "negative weighting", for example.

Ah yes. This should probably be cross-referenced in the security
considerations section, and flagged as an important consideration if the
SMTP server pays any attention to the client's suggestions.

Otherwise thanks for reading my comments. I look forward to the next
versions of the documents.

---

From: "Matthew Elvey" <[email-masked]>
Date: Tue, 15 Jun 2004 11:33:49 -0800

Glad to see the spec get refined a bit; perfection in an ID isn't
expected; some refinement discussion snipped.
This is how IETF specs are supposed to be developed - in open
discussion. 

On Tue, 15 Jun 2004 07:58:28 -0400, "John Leslie" <[email-masked]> said:
>> ...

Sounds good. (IMO, it should be mentioned as supplemental, if at all, as
KISS makes the spec easier to read, and that's critical.)

>    I'm not sure I understand your question here. Could you clarify?

I'm saying two things. 
1)Just as rDNS doesn't tie an IP to a domain for our purposes, perhaps
neither does SMTP Auth.  So it perhaps shouldn't be part of the spec, if
the purpose is just to be a component of CSV.
2)Just as rDNS doesn't tie an IP to a domain for our purposes, STARTTLS
might not either - i.e. STARTTLS should be used to validate the identity
of the connection initiator, not the connection acceptor.  Typically,
STARTTLS validates the connection acceptor to the initiator.  In other
words, it typically (e.g. in https:// or for secure SMTP) is used to
ensure that when you go to a site, you're communicating with who you
think you're communicating with.  It typically doesn't ensure that the
site is communicating whith whom it thinks it's communicating with - the
web surfer doesn't have an identity tied to a domain; the site (the
connection acceptor) is tied to a domain via the CA.  Hopefully that's
more clear, not less clear.

I guess if either these two methods are mentioned here but not relevant
to CSV, that should be stated.

>    Sounds like a good execrcise.

:)

> _are_ designed to work with wildcards.)

Oh, right. There's an MX wildcard on elvey.com.  
But I expect no server is going to send EHLO matthew.elvey.com.  
Or even EHLO elvey.com, unless I set up my own server. 

Thinking aloud about something:
Well, I just noticed that my posts ARE often from a server that seems to
be sending 
EHLO elvey.com, e.g.
Received: from elvey.com (adsl-63-195-86-147.dsl.snfc21.pacbell.net
        [63.195.86.147])
        by mail.messagingengine.com (Postfix) with ESMTP id D227ABDDD8D;
        Wed, 26 May 2004 22:17:21 -0400 (EDT)
But this doesn't matter because this is me sending to a server that I've
authenticated to with STARTTLS.  It will not be doing CSV validation on
email I send it; it trusts me. 
It's not the the hop that we're concerned with.  My mail server doesn't
do something silly like put elvey.com in the EHLO just because it's in
the from for this hop:
Received: from out2.smtp.messagingengine.com ([66.111.4.26])
        by ietf-mx with esmtp (Exim 4.12) id 1BTAUD-0005Mh-00
        for [email-masked]; Wed, 26 May 2004 22:19:13 -0400
and I don't think others do either.

---

From: Dave Crocker <[email-masked]>
Date: Wed, 16 Jun 2004 08:24:25 +0800

Tony,

First of all, MANY thanks for such a quick review!

TF> The new documents are much clearer, however they are still vague on
TF> important details,

please note where i've asked for clarification about what you want to
have different.  if you did not list something in your note, please
send details.

TF> they take an awfully long time to get to the point, and
TF> they contain an amazing amount of irrelevant digression.

Given the amount of effort we put into fleshing out the text and
working on clarity and completeness, it is not likely that any of its
irrelevance or digressiveness will be obvious to us.  So, again, your
comments say exactly what you think needs to be changed, right?

In any event, none of us expects these drafts to be in their final
form.  In fact there are aspects we have internal disagreements about,
but decided it would better to pursue public discussion rather than
wait longer to figure out a consensus among the 3 of us.

I'm in transit, and do not yet know what my connectivity will be for
the next 5 weeks.  Therefore, John L. will be doing the heavy lifting of
responding to comments, so I'll just touch on any of your comments
that seem to go at core difficulties or issues.

TF> Section 3.3

TF> The CSV spec makes a nice distinction between authentication and
TF> authorization. So why is the CSV authorization step defined by a document
TF> called "Client SMTP authentication"? (Actually this appears to be a typo
TF> since the document calls itself "Client SMTP authorization".)

Indeed, a typo.  We, ourselves, have had a very, very tough time
keeping the two functions clearly separate.  The typo is residue.

TF> I'm very dubious of the weaker DNS checking suggested in 2.1 and 2.2. I
TF> think there should be much more discussion of the possible attacks and
TF> defences,

Perhaps.  THere is a challenge to balance between specification and
pedagogy.  At the least, we felt is likely to be sufficient to make
clear that we do not see it as a useful technique for any type of
really serious authentication.

TF> and why they are under consideration when full foward + reverse
TF> checking is available.

reverse is often not available, as we noted.  so the question is
whether forward, by itself, is useful.  Our conclusion is that for
this very constrained use, it is.  It won't surprise anyone that we
won't be surprised if the community concludes otherwise

TF> 3.2 SMTP Auth

TF> As I understand it this draft is intended to be independent of SMTP.
TF> Therefore SASL should be cited instead.

Well, we do have a bit of a challenge, given that the actual use of
the document is part of an smtp-based service.

TF> In addition to this, SASL is generally used for user authentication not
TF> hostname authentication. This section needs to be more explicit about how
TF> this unusual use works in practice.

interesting point.

TF> 4 Independent Policy Services

TF> "In effect, the question about domain name use is a question about the
TF> reputation and accountability of the domain name administrator."
TF> No, it is a question about the reliability of the host. This entire
TF> section refers to a part of the CSV process covered by the DNS draft.

reliability is often part of the input to an accreditation process.

TF> draft-ietf-marid-dsvcsa-00

TF> Sections 4 & 5

TF> These sections overlap rather a lot. They also omit to specify what the
TF> server should do if the client's CSA records has invalid data.

What do you mean by "invalid" data?  How is invalidity assessed?

TF> The protocol specified in this draft appears to be (I'm reading between
TF> the lines here) that the EHLO domain name stated by the SMTP client is
TF> used to look up a SRV record. This record states in the weight field if
TF> that name may be used by an SMTP client, and it also contains a target
TF> name. The DNS reply should contain additional data, being the IP address
TF> records associated with this target name. One of these must match the
TF> client IP address from the SMTP connection in order for the EHLO domain
TF> name to be considered valid. Or maybe (according to HNA) the IP addresses
TF> associated with the EHLO domain itself must match the connection IP
TF> address. Or maybe the reverse DNS must agree too. This is not clear. It's
TF> also not clear how this relates to the RFC 2821 nix on EHLO checks.

wow.  what a really excellent summary!  wish we had written it, and thoroughly
appreciate that you did.  it will be a nice addition to the doc.

TF> draft-crocker-marid-csvdna-00

TF> There's no discussion of the obvious attack of a client recommending
TF> accreditation services that all say the client is wonderful.

that's not an attack.  i would fully expect a client to cite only
those with positive assessments.

the question is what the receiving smtp server thinks of those
accreditation services.

d/
--
 Dave Crocker <mailto:[email-masked]>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>

---

From: Dave Crocker <[email-masked]>
Date: Wed, 16 Jun 2004 08:21:38 +0800

Tony,

TF> Section 3.3

TF> 3.2 SMTP Auth

interesting point.

TF> Sections 4 & 5

d/

---

From: Tony Finch <[email-masked]>
Date: Wed, 16 Jun 2004 11:12:38 +0100

> comments say exactly what you think needs to be changed, right?

Yes, I think the high-level structure of the specification is up to you; I
was reviewing it mainly from the point of view of trying to understand
how to implement it.

If I were going to make suggestions about how to structure the spec, I'd
throw away HNA and the CSV overview. I'd improve the CSA spec with a more
precise explanation of the authentication process (in a separate section).

If you want to keep HNA in something like its current form, then I suggest
that it should describe each authentication method with reference to a
subject domain that is to be authenticated and what other protocol
information it is compared with. e.g. the forward DNS method is to look up
the A or AAAA records for the subject domain and ensure that one of the
addresses is the same as the client IP address from the TCP connection.
The TLS method is to require a client certificate with a common name that
corresponds to the subject domain.

Actually that doesn't work very well because CSA's input into the forward
DNS process is the CSA SRV target domain, not the EHLO domain. In the TLS
case perhaps the subject domain is the output of the process rather than
an input.

The problem with a strict separation between authentication and
authorization in the presentation of CSV is that the protocol does not
separate them as described. In fact if my understanding of the protocol is
correct (see below) then the description in the CSV draft that
authorization follows authentication is wrong. The process is interleaved:
look up the authorization record; look up the authentication record; check
authentication; check authorization.

The DNA spec is not really tied to CSV in the way your document structure
suggests -- it's useful as the next step after any domain authentication
system (SPF, CID, DK, etc.).

> Our conclusion is that for this very constrained use, [forward-only
> checking is sufficient].  It won't surprise anyone that we won't be
> surprised if the community concludes otherwise

I didn't work out that that was what your protocol requires until after
reading both HNA and CSA. I think most of the discussion in HNA is
relevant only in a rationale document, not a spec. (There are no useful
SHOULDs or MUSTs in it!) However the spec should probably explain why a
full forward-and-reverse DNS consistency check isn't necessary.

> TF> HNA section 4 Independent Policy Services
>

> What do you mean by "invalid" data?  How is invalidity assessed?

Outside the spec. E.g. a non-zero port, or a weight that isn't 0, 1, or 3,
or a priority that isn't 1, etc.

> appreciate that you did.  it will be a nice addition to the doc.

I'm glad you like it :-) The question still remains about the
authentication process. Does it use the addresses of the client EHLO
domain? (As I understand it, no.) Or does it only use the CSA SRV target
domain? (AIUI, yes.) Does it use both? if so, how? (AIUI, no.) What about
reverse DNS? (AIUI, no.)

---

From: John Leslie <[email-masked]>
Date: Wed, 16 Jun 2004 12:47:10 -0400

Matthew Elvey <[email-masked]> wrote:
> 1) Just as rDNS doesn't tie an IP to a domain for our purposes, perhaps
>    neither does SMTP Auth. 

   True. SMTP AUTH does not tie to an IP address.

   It does, however, authenticate that you're talking with an SMTP client
worthy of some level of trust (it gave an appropriate response to the
challenge you issued). It didn't seem a stretch to think that this might
sometimes prove it trustworthy enough to give a correct EHLO string.

   Having said that, it's hard to imagine the case where host name
authentication would be the only thing missing _and_ SMTP AUTH would
be in use.

> So it perhaps shouldn't be part of the spec, if the purpose is just to
> be a component of CSV.

   I believe Dave included it for completeness of background, and it
really isn't part of our proposal.

> 2) Just as rDNS doesn't tie an IP to a domain for our purposes,
>    STARTTLS might not either - i.e. STARTTLS should be used to validate
>    the identity of the connection initiator, not the connection acceptor.

   Agreed: if STARTTLS doesn't validate the initiator, it's useless for
host name authentication.

   However, we realize there _will_ be cases where the SRV lookup doesn't
return the matching IP address, but local policy may recognize STARTTLS
as "sufficient authentication".

> to CSV, that should be stated.

   I can't quite agree they're "not relevant"; but I agree that a warning
label is appropriate. ;^)

---

From: Douglas Otis <[email-masked]>
Date: Wed, 16 Jun 2004 10:42:42 -0700

> sometimes prove it trustworthy enough to give a correct EHLO string.

I think this can be restricted to only consider open systems. 

> host name authentication.

It can authenticate both ends.  It normally does not. But this could be
used in an open system.  It is just too expensive to be used in this
manner.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 17 Jun 2004 10:06:32 +0800

Tony,

TF> If I were going to make suggestions about how to structure the spec, I'd
TF> throw away HNA and the CSV overview. I'd improve the CSA spec with a more
TF> precise explanation of the authentication process (in a separate section).

CSA is about authorization, not authentication.

The reference to the Additional Information mechanism, within CSA, is
as a convenience rather than really being part of the spec.

TF> If you want to keep HNA in something like its current form, then I suggest
TF> that it should describe each authentication method with reference to a
TF> subject domain that is to be authenticated and what other protocol
TF> information it is compared with. e.g. the forward DNS method is to look up
TF> the A or AAAA records for the subject domain and ensure that one of the
TF> addresses is the same as the client IP address from the TCP connection.
TF> The TLS method is to require a client certificate with a common name that
TF> corresponds to the subject domain.

Adding detail like this, to make VERY clear how each one gets used,
sounds like an excellent idea.

TF> Actually that doesn't work very well because CSA's input into the forward
TF> DNS process is the CSA SRV target domain, not the EHLO domain. In the TLS
TF> case perhaps the subject domain is the output of the process rather than
TF> an input.

Huh?  CSA _is_ the forward DNS process.  Take the EHLO domain, do a
lookup on _client._smtp.<ehlo domain> and get back the authorization
SRV.

TF> The problem with a strict separation between authentication and
TF> authorization in the presentation of CSV is that the protocol does not
TF> separate them as described.

Yes it does.

TF> In fact if my understanding of the protocol is
TF> correct (see below) then the description in the CSV draft that
TF> authorization follows authentication is wrong.

The text makes clear that the actual excecution might occur
differently.    Refer to "A proposal or its implementation" in the
Introduction.

TF> The process is interleaved:
TF> look up the authorization record; look up the authentication record; check
TF> authentication; check authorization.

There is no one, right sequence. Folks will choose different
sequences, for a variety of reasons. This provides yet-another example
of the reason it is essential to be clear that separate functions are
indeed separate in the specification.

TF> The DNA spec is not really tied to CSV in the way your document structure
TF> suggests -- it's useful as the next step after any domain authentication
TF> system (SPF, CID, DK, etc.).

we specified it, so it's tied to this spec.  on the other hand, yes,
any component of this could be replaced by some other method of
satisfying the component requirement.

the challenge is to make sure that it indeed does the right
authentication, for example. alghouth spf, cid, and dk may well do
nice authentications, it's not clear to me that any of them have the
right semantics for what CSV is requiring.

>> What do you mean by "invalid" data?  How is invalidity assessed?
TF> Outside the spec. E.g. a non-zero port, or a weight that isn't 0, 1, or 3,
TF> or a priority that isn't 1, etc.

ahh.  right.  good catch. thanks.

>> appreciate that you did.  it will be a nice addition to the doc.

TF> I'm glad you like it :-) The question still remains about the
TF> authentication process. Does it use the addresses of the client EHLO
TF> domain? (As I understand it, no.) Or does it only use the CSA SRV target
TF> domain? (AIUI, yes.) Does it use both? if so, how? (AIUI, no.) What about
TF> reverse DNS? (AIUI, no.)

i think the answer needs to depend on a step-through analysis of
threats.  what will be robust against administrative attacks.  that
is, what will be open to having a bad guy create dns records to permit
deception, and what won't.  in my current, jet-lagged, vacation mental
state, i'm not even close to being able to perform the exercise.

TF> I don't think it's worth making heavy weather of the distinction,
TF> certainly not to the extent of forcing an order of presentation of the CSV
TF> algorithm that has no relation to the order of execution. People are used
TF> to authentication and authorization being very tightly bound; keeping them
TF> apart isn't necessary if it is clear which part of the protocol is
TF> performing which security function.

The "used to" has two responses, both problematic:

1. That may well contribute to the reason folks are so bad at
distinguishing between these two, very different functions and why we
need to be thoroughly, painfully, crystal-ly clear about the
differences between them.  To that extent, the anti-spam work is
working in territory that has been sadly lacking much history.

2. In fact authorization is rarely discussed within Internet standards
specifications.

d/

---

From: Tony Finch <[email-masked]>
Date: Thu, 17 Jun 2004 13:58:13 +0100

> CSA is about authorization, not authentication.

OK. But the point of using CSA as opposed to any other lookup mechanism is
it gives you authn and authz in one protocol exchange. And the other
authn mechanisms described in HNA are too unweildy for use on the scale of
the Internet.

I think that CSA is elegant and simple, but I think its specification is
being WAY overdone. Three documents to describe an enhancement to forward
and reverse DNS consistency checking?! People will get bored with the
abstract discussion of the principles of security protocol design and not
bother to read as far as the bit that explains how to implement it.

I'm afraid I'm getting a bit frustrated because I doubt this will get
anywhere near deployment.

> lookup on _client._smtp.<ehlo domain> and get back the authorization

OK, but that's different from the normal A or AAAA lookup for the bare
EHLO domain which HNA appears to describe. And "CSA is about
authorization, not authentication."

---

From: Dave Crocker <[email-masked]>
Date: Thu, 17 Jun 2004 22:44:38 +0800

Tony,

>> CSA is about authorization, not authentication.
TF> OK. But the point of using CSA as opposed to any other lookup mechanism is
TF> it gives you authn and authz in one protocol exchange.

Although that is a possible and an appealing scenario, it was
certainly not "the point" behind the CSA SRV design.

In fact, I'm expecting people to look for much stronger
authentication, at least in some cases.

TF> And the other
TF> authn mechanisms described in HNA are too unweildy for use on the scale of
TF> the Internet.

not clear.  certainly something as heavyweight as client-side
authentication with TLS has not.  and I remain skeptical about it, but
the security guys keep banging on that door and, at some point,
someone is likely to open it.

TF> I think that CSA is elegant and simple, but I think its specification is
TF> being WAY overdone. Three documents to describe an enhancement to forward
TF> and reverse DNS consistency checking?!

Well, it does rather more than that.

First, it does not rely on forward/reverse and, in fact, allows much
stronger alternatives for authentication.

Second, the accreditation step is entirely new and explores territory
with essentially no Internet standards history or large-scale,
distributed operation testing.

It well might turn out to be appropriate to compress the
specifications, once folks are VERY clear about the service and have
good consensus on it.  However at this stage, it is proving almost
impossible to get coherent discussion of these mail-reception control
techniques, with any real clarity about the component functions.
Unless and until we get there, dividing the topics up to be almost
completely separate should aid that coherence.

TF> People will get bored with the
TF> abstract discussion of the principles of security protocol design and not
TF> bother to read as far as the bit that explains how to implement it.

That's why I like your suggested summaries of the total procedure.  It
permits easy scanning to extract the core characteristics, without
having to first read all the detail.

>> lookup on _client._smtp.<ehlo domain> and get back the authorization
TF> OK, but that's different from the normal A or AAAA lookup for the bare
TF> EHLO domain which HNA appears to describe. And "CSA is about
TF> authorization, not authentication."

I had not understood that you meant the SECOND forward lookup.  The
first produces the CSA SRV record.  And, yes, the second produces the
A+ record of the SRV target.

d/

---

From: Matthew Elvey <[email-masked]>
Date: Fri, 18 Jun 2004 14:58:46 -0700

>sometimes prove it trustworthy enough to give a correct EHLO string.
It's a huge stretch.  I must strenuously object.
E.g., here's a simple solution to the spam problem that has the same 
problem. (Yeah, nominally that's not the problem we're solving; I'm just 
making the point clear.)
Solution: all non-spam must include the header X-This-is-spam: false.
Only accept mail with this header.
Spammers would [include the header|use SMTP AUTH] faster than non-spammers.
They are solutions that impose requirements that spammers can meet more 
readily than non-spammers.
These are, I think, examples of what Dave Crocker is calling an 
'administrative attack'.

Actually that point bears amplification/restatement:

(Without the reputation services that are being contemplated,) all the 
MARID proposals impose requirements that spammers can meet more readily 
than non-spammers.  It's been said that SPF validation currently 
correlates positively with spamminess.

>as "sufficient authentication".
"_will_"?  I'd say "might".  I expect STARTTLS will not take off.  (But 
since it can authenticate the initator, if TCP traffic from an IP not 
really being from that IP becomes common and IPsec doesn't, I could see 
it taking off). (BTW, TLS used like this is sort of a naiive hashcash 
implementation - it's expensive, as Otis points out - though he may be 
referring to the cost of the CA-signed cert; I'm thinking of the CPU 
cost of starting a TLS connection.)

>label is appropriate. ;^)
Ok, I can live with that, grudgingly.

---

From: Matthew Elvey <[email-masked]>
Date: Fri, 18 Jun 2004 20:49:41 -0700

Dave, John, Doug:
Have you considered and rejected or not considered adding the NBB idea 
that I threw out a while back to the CSV spec? 

For convenience:

The NBB idea is:
Take CSV, and add a new requirement: mail that has failed (as in
there is a MARID record AND the sending IP isn't there AND there's no
?all) a 2821.FROM check MUST NOT be bounced; instead it MUST either be
refused at SMTP time, or accepted and destroyed.   In other words, DON'T
require SRS, but DO require that mail that goes via non-SRS systems not
lead to bounces to  systems that didn't originate the original message.

What will this requirement break? Well, let's be up front. Fundamentally 
it breaks (for a very small subset, as I'll show!) what I think is an 
oft-broken requirement: no mail be destroyed (or it could specify that 
such mail goes to postmaster, like double-bounces, but that's a broadly 
violated requirement already (e.g. some major ISPs dump mail they think 
is spam into /dev/null.) It doesn't break SRS-compliant forwarders AND 
it doesn't break non-SRS forwarders, except if the final recipient 
decides it wants to bounce or refuse a message, that bounce or refusal 
won't get back to a sender IF she has MARID records. It doesn't break 
mailing lists or greeting card sites or require that they change. Unlike 
SPF, it DOESN'T require that every end user use an MTA authorized for 
the 2822.From they're using! If a legitimate user uses an unauthorized 
MTA, her mail will still get through, but any email that she sends to 
invalid addresses (eg. typos) or that bounces for some other reason 
won't get back to her. A further enhancement would be that SMTP servers 
that are unable to get a message a trusted user sends out for any reason 
would be allowed to bounce the message back to the user. If everyone 
implements SRS, it breaks nothing, and if no one does, the stuff it 
breaks is, I argue, not critical.

---

From: Roy Badami <[email-masked]>
Date: Sat, 19 Jun 2004 13:25:53 +0100

>>>>> "Matthew" == Matthew Elvey <[email-masked]> writes:
    Matthew> The NBB idea is: Take CSV, and add a new requirement:
    Matthew> mail that has failed (as in there is a MARID record AND
    Matthew> the sending IP isn't there AND there's no ?all) a
    Matthew> 2821.FROM check MUST NOT be bounced; instead it MUST
    Matthew> either be refused at SMTP time, or accepted and
    Matthew> destroyed.

Personally I'm very strongly against any mandate for accepting and
destroying messages.

I don't believe that this or any other IETF WG should be contemplating
removal of the mandate in RFC 1123 5.3.3 and RFC2821 6.1 that once
mail has been accepted it MUST be either delivered or bounced.

NBB would seem to go against this long established principle that
blackholes are bad.  The problem is not with messages that are
genuinely forged, but with messages that are mistakenly determined to
be forged as a result of a configuration error.  Obviously rejection
at the SMTP level is best (but this may just generate a bounce
upstream anyway), but if this is not possible then it is vital that
these messages are bounced, rather than discarded, to allow the
configuration error to be detected and corrected.

NBB is not going to prevent the problem of joe jobs.  Nor even would
SPFv1.  All NBB can do is avoid trying to make the joe job problem
worse as a result of MARID/LMAP checks.  But we need (and will have)
other ways of preventing joe jobs, whether by checking a token in the
envelope address, checking the Message-ID, or something else.

Once we have protection against unwanted bounces (and we need that
anyway) NBB gains us nothing.

If we throw away the principle of reliable mail delivery for short
term expediency, it will be very difficult to get it back, we'll
regret that decision for a long time...

Or in summary: the cure is worse than the disease...

   -roy

---

From: John Leslie <[email-masked]>
Date: Sat, 19 Jun 2004 08:53:11 -0400

> It's a huge stretch.  I must strenuously object.

   Noted. (It really doesn't make much difference, so I won't argue.)

> <snip>
> "_will_"?  I'd say "might". 

   I'd say "will". (I didn't say it would be common.)

> I expect STARTTLS will not take off.
> <snip>

   Noted.

> Ok, I can live with that, grudgingly.

   Thank you.

   BTW, the WG versions of the CSV IDs have been submitted and approved
by Marshall. Meanwhile, they're available at:

http://www.jlc.net/MARID/CSV/

   The "intro" has been rewritten: you may be happier with it now. If not,
feel free to propose actual wording to improve it.

---

From: "Alan DeKok" <[email-masked]>
Date: Sat, 19 Jun 2004 10:59:01 -0400

> Personally I'm very strongly against any mandate for accepting and

  This is already happening on the net today, due to deficiencies in SMTP.

> mail has been accepted it MUST be either delivered or bounced.

  That requirement is impossible to implement in practice.  SMTP makes
*no* provisions for verifying that a bounce path exists.  Therefore,
once a message has been accepted for delivery, it is *impossible* to
bounce it in the general case.  The MTA can try, but there's no
guarantee that the MAIL FROM parameter will be resolvable, or that the
host will even know about the message which caused the bounce.

  All of the RMX/SPF/etc. proposals can be looked at as giving the
recipient more information with which to validate the bounce path.
This property is independent of their other effects.

> Obviously rejection at the SMTP level is best (but this may just
> generate a bounce upstream anyway), but if this is not possible then
> it is vital that these messages are bounced, rather than discarded,
> to allow the configuration error to be detected and corrected.

  Bounced where?  How can you tell that the bounce path is valid, and
that you're not spamming the MAIL FROM site with bounces from forged
messages?

  Alan DeKok.

---

From: Matthew Elvey <[email-masked]>
Date: Sat, 19 Jun 2004 11:56:00 -0700

>Personally I'm very strongly against any mandate for accepting and
I'm simply being honest about the suggested disposition of such 
messages, where other proposals avoid *explicitly* saying that in 
situation S, accepting and destroying a message is appropriate.   But 
it's implicit, even when there are statements such as  "An SPF email 
system MAY choose to reject or discard email on the basis of local 
policy" and "Policy decisions regarding particular messages are outside 
the scope of SPF." (See SPF quote below as well.)

Is this acceptable? :

 The NBB idea is:

Take CSV, and add a new requirement: mail that has failed (as in
there is a MARID record AND the sending IP isn't there AND there's no
?all) a 2821.FROM check should be treated according to local policy for 
such messages.  
In other words, DON'T
require SRS, but DO enable mail that goes via non-SRS systems that would
lead to bounces to  systems that didn't originate the original message 
be treated according to local policy.

Are you for other proposals that avoid explicitly saying that in 
situation S, accepting and destroying a message is appropriate, even 
though it's obvious that this is what they're endorsing? 

>be forged as a result of a configuration error.
>
Problems would have to occur simultaneously:
* the mail is sent in direct violation of MARID checks, and
* the responsible domain admin has said -all (or the equivalent in 
different syntax), not ~all or ?all.
* the mail has gone through a forwarding system and
* the forwarding system has not implemented SRS and
* the forwarding system (e.g. mailing list) isn't whitelisted.

>  Obviously rejection
>NBB is not going to prevent the problem of joe jobs.  
>
Why not?  Your statement lacks an argument.

>Nor even would SPFv1. 
I think that's incorrect too.
 I believe Meng has said that he'd be happy if this is all that SPF+SRS 
does. I interpret this to mean Meng disagrees with you on this.
"SPF was originally designed to prevent joe-jobs." - http://spf.pobox.com/
This also proves my point earlier - proposals just aren't being up front 
about it, but clearly the intention is to make it safe to accept and 
destroy a class of messages. (SPF is supposed to work fine on MUAs.)

> All NBB can do is avoid trying to make the joe job problem
No, we have some solutions that ALLOW joe jobs, but make them not 
directly end-user-visible (if the receiving server doesn't die under the 
load of the joe-jobs it is receiving).

The end system does the accepting and destroying of a message that you 
say is not appropriate in other situations.

Also they don't work when users are able to send messages from wherever 
(without forging) using their usual email address.
With what I'm proposing, they can still do this.

>anyway) NBB gains us nothing.
Obviously.  The point of NBB is to address this problem.  So if it's 
solved, there's no point.  Circular reasoning.  The question is why 
my/another solution is better.

---

From: Roy Badami <[email-masked]>
Date: Sun, 20 Jun 2004 22:13:12 +0100

>>>>> "Alan" == Alan DeKok <[email-masked]> writes:

    Alan> Roy Badami <[email-masked]> wrote:
    >> Personally I'm very strongly against any mandate for accepting
    >> and destroying messages.

    Alan>   This is already happening on the net today, due to
    Alan> deficiencies in SMTP.

I agree it's happening; I remain to be convinced that it's a good
thing...

    Alan>   That requirement is impossible to implement in practice.

RFC1123 and 2821 do contain a note that this may not be possible in
all circumstances.

    Alan>   Bounced where?  How can you tell that the bounce path is
    Alan> valid, and that you're not spamming the MAIL FROM site with
    Alan> bounces from forged messages?

Bounced to the MAIL FROM.  I agree that you can't tell that it's
authentic.  But silently dropping messages when an administrator
misconfigures their MARID record is going to make debugging difficult.

IMHO the correct place to drop unwanted bounces is at the MTA
receiving the bounce, since it is in the best position to determine
whether it relates to a message originated by that address.

	-roy

---

From: Roy Badami <[email-masked]>
Date: Sun, 20 Jun 2004 22:35:42 +0100

>>>>> "Matthew" == Matthew Elvey <[email-masked]> writes:

    Matthew> Are you for other proposals that avoid explicitly saying
    Matthew> that in situation S, accepting and destroying a message
    Matthew> is appropriate, even though it's obvious that this is
    Matthew> what they're endorsing?

No; I'm against blackholing messages under any circumstances.  We'll
end up with an unreliable mail transport and we'll lose an important
debugging aid.

    Matthew> Problems would have to occur simultaneously: 
    Matthew> * the mail is sent in direct violation of MARID checks, and 
    Matthew> * the responsible domain admin has said -all (or the equivalent
    Matthew>   in different syntax), not ~all or ?all.
    Matthew> * the mail has gone through a forwarding system and 
    Matthew> * the forwarding system has not implemented SRS and 
    Matthew> * the forwarding system (e.g. mailing list) isn't whitelisted.

I'm more concerned with problems like misconfigured MARID records....

    >> NBB is not going to prevent the problem of joe jobs.
    Matthew> Why not?  Your statement lacks an argument.

Because the proposal endorses SMTP level rejection (which I'm all in
favour of).  In many circumstances this will cause the upstream MTA to
bounce the message to the MAIL FROM.

    >> Nor even would SPFv1.
    Matthew> I think that's incorrect too. 

Because again SPF mandates SMTP rejections.  Consider forged mail sent
through an open relay.  At the moment, only the undeliverable mail
will bounce to the MAIL FROM.  SPFv1 causes more of the forged mail to
be rejected and actually makes the joe job problem worse...

    Matthew> I interpret this to mean Meng disagrees with you on this.
    Matthew> "SPF was originally designed to prevent joe-jobs." -

I've never believed this, but that doesn't mean I'm against SPF and
other MARID schemes.  They're valuable for other reasons.

The only way SPFv1 will prevent joe jobs is that if it's so successful
and widely deployed that spammers completely give up attempting to
send forged mail since they know it zero chance of reaching its
recipient.  I don't see that happening in the medium term, if ever.

    Matthew> No, we have some solutions that ALLOW joe jobs, but make
    Matthew> them not directly end-user-visible (if the receiving
    Matthew> server doesn't die under the load of the joe-jobs it is
    Matthew> receiving).

Yes, that's true.

    Matthew> The end system does the accepting and destroying of a
    Matthew> message that you say is not appropriate in other
    Matthew> situations.

Correct.  It's not ideal, but it's the lesser evil.  And it happens at
a place where useful logging can occur.  And I believe it's going to
happen anyway.

    Matthew> Also they don't work when users are able to send messages
    Matthew> from wherever (without forging) using their usual email
    Matthew> address.  With what I'm proposing, they can still do
    Matthew> this.

Schemes that use a cryptographic checksum (whether in the MAIL FROM,
Message-ID or an extension header) can be made to work in these
circumstances.

    Matthew> Obviously.  The point of NBB is to address this problem.
    Matthew> So if it's solved, there's no point.  Circular reasoning.
    Matthew> The question is why my/another solution is better.

Because NBB will only prevent forged bounces from arriving in my
mailbox if *everyone* on the Internet implements it (and not even
then, given upstream bounces).

Given that's going to take a long time, I'm going to want to implement
one of the other schemes to give me protection now, rather than years
from now.

The kinds of scheme I describe are starting to be deployed, and I
expect them to start becoming commonplace...  Given this kind of
scheme is going to have to be deployed anyway, NBB adds little value
and removes an important reliability principle and debugging tool.

    -roy

---

From: Roy Badami <[email-masked]>
Date: Sun, 20 Jun 2004 22:47:26 +0100

I replied:

    Roy> Because NBB will only prevent forged bounces from arriving in
    Roy> my mailbox if *everyone* on the Internet implements it (and
    Roy> not even then, given upstream bounces).

Or, put another way:

If I implement NBB on *my* MTAs, it does nothing to make *my* life
better, but I make a very small step towards making *everyone else's*
life better.  Once when enough people do likewise, my like gets a lot
better.

If I implement one of the bounce discarding schemes, *I* benefit
immediately and substantially.  So there's a much stronger incentive
for me to implement.

    -roy

---

From: Matthew Elvey <[email-masked]>
Date: Sun, 20 Jun 2004 15:54:36 -0700

>life better.  Once when enough people do likewise, my like gets a lot
Not true.  If recipients of x %, e.g. 90% of the mail out there 
implement NBB, then as soon as you publish MARID records for your 
domain, you'll get around x %, e.g. 90% fewer joe jobs. 

Meng's latest SPF plan (in which he has borrowed my idea of CSV 
semantics using SPF records) makes it much easier to implement SPF than 
the plan in the current RFC draft.  Implementation of SPF by nearly all 
legit senders will be much more feasible soon. Yay! 

Anyway, if SPF  doesn't advocate NBB, then at least in the case of 
post-SMTP SPF implementations, the SPF-Received header will show a fail, 
and the would-be joe job victim can potentially detect and take that 
into account when deciding the disposition of the resulting bounce.  But 
in the (hopefully more common) case of SMTP+SPF receivers, there may not 
be a way to tell that the bounce was the result of an SPF "fail".  
Perhaps that can be remedied, e.g. a via a fixed (grep-able) string 
included in all exp replies.  

BTW, you are not accurate when you say SPF 'endorses' SMTP rejection.  
The spec says "An SPF email system MAY choose to reject or discard email 
on the
   basis of local policy." It also says "MTAs MAY reject the message 
using a permanent
     failure reply code.  (Code 550 is RECOMMENDED. ...)"  That's MAY, 
not SHOULD or MUST.

I would be interested to hear what others think - should SMTP+SPF 
receivers reject or discard messages that SPF "fail" (e.g. hit -all, not 
"error" or "unknown")?  (Perhaps it makes sense to reject for a while by 
default and then sunrise discard as the default.)

---

From: Roy Badami <[email-masked]>
Date: Mon, 21 Jun 2004 00:11:29 +0100

>>>>> "Matthew" == Matthew Elvey <[email-masked]> writes:
    Matthew> Not true.  If recipients of x %, e.g. 90% of the mail out
    Matthew> there implement NBB, then as soon as you publish MARID
    Matthew> records for your domain, you'll get around x %, e.g. 90%
    Matthew> fewer joe jobs.

Agreed.  But I have to wait for a substantial proportion of the
Internet to implement NBB before I begin to see the benefits.

    Matthew> Meng's latest SPF plan (in which he has borrowed my idea
    Matthew> of CSV semantics using SPF records) makes it much easier
    Matthew> to implement SPF than the plan in the current RFC draft.
    Matthew> Implementation of SPF by nearly all legit senders will be
    Matthew> much more feasible soon. Yay!

It's an interesting idea; do you know if an ID is forthcoming?

    Matthew> BTW, you are not accurate when you say SPF 'endorses'
    Matthew> SMTP rejection.  The spec says "An SPF email system MAY
    Matthew> choose to reject or discard email on the basis of local
    Matthew> policy." It also says "MTAs MAY reject the message using
    Matthew> a permanent failure reply code.  (Code 550 is
    Matthew> RECOMMENDED. ...)"  That's MAY, not SHOULD or MUST.

I misremembered.  But I'd still call that an endorsement.  It's not
exactly a SHOULD NOT.

    Matthew> I would be interested to hear what others think - should
    Matthew> SMTP+SPF receivers reject or discard messages that SPF
    Matthew> "fail" (e.g. hit -all, not "error" or "unknown")?
    Matthew> (Perhaps it makes sense to reject for a while by default
    Matthew> and then sunrise discard as the default.)

I'm dead against that.

For the record, this is my current position:

I think SMTP-time rejection should be used whenever possible.  I'm
currently of the position that when it's not possible it's reasonable,
and even disirable, to originate a bounce, despite the high
probability that the MAIL FROM may be forged.  I'm willing to be
convinced otherwise on this point though...

I'm absolutely dead against the idea of lying in the SMTP response
codes, and claiming you've accepted the message when you've already
determined you're going to do know such thing.  That way lies madness.

I can see why it might be expedient to do this (AFAICS it's the only
way to prevent upstream bounces to forged addresses) but as I said
before (IMHO) the cure is worse than the disease...

       -roy

---

From: Matthew Elvey <[email-masked]>
Date: Sun, 20 Jun 2004 17:43:40 -0700

>It's an interesting idea; do you know if an ID is forthcoming?
I just posted to spf-discuss for the first time:
http://archives.listbox.com/[email-masked]/200406/0958.html
I read the thread about this:
http://archives.listbox.com/[email-masked]/200406/0886.html
The latest I-D is dated May '04, and doesn't reflect this "Unified SPF".
We'll see if it's in SenderID, which is what the SPF - CID merger effort 
is being called, I think.
This thread makes me think it will.

I'm hopeful that spf code out there will soon check by default as per
http://spf.pobox.com/slides/unified%20spf/0428.html.

That slide is from the slideshow at
http://spf.pobox.com/slides/unified%20spf/index.html
that Meng pointed me to.

>    Matthew> and then sunrise discard as the default.)
That was clear.  I'm halfway won over, hence the 'what *others* think'.
I was going to post there about NBB, but am holding off; if no one likes 
the idea here, why bring it up there?

I figure we'll hear back after the weekend re.

---

From: Roy Badami <[email-masked]>
Date: Mon, 21 Jun 2004 02:18:19 +0100

>>  It's an interesting idea; do you know if an ID is forthcoming?

    Matthew> The latest I-D is dated May '04, and doesn't reflect this
    Matthew> "Unified SPF".  We'll see if it's in SenderID, which is
    Matthew> what the SPF - CID merger effort is being called, I
    Matthew> think.  This thread makes me think it will.

If the name SenderID refers to draft-ietf-marid-core (which was my
assumption) then it's clearly not in -00.  If it refers to something
else, then until and unless that proposal is submitted to this WG then
I would guess this is probably the wrong forum to discuss it further.

    >>  I'm dead against that.
    Matthew> That was clear.

No doubt :-) Still, it's good to state one's position as precisely as
possible for the benefit of clarity of discussion...

Actually, on reflection I'd like to make one further point:

I strongly believe that this point shouldn't be glossed over by any
MARID standard.  You asked a couple of messages back what I think of
proposals that implicitly suggest blackholing messages.  I think
ambiguity like that would be the worst possible outcome for MARID.  If
MARID is going to fundamentally change this (or any other) long
established principle of Internet e-mail then it needs to be upfront
about it.

I'll go further and suggest that MARID needs to do one of the
following: either

1.  Make it clear that all mail that's not accepted MUST be rejected
    at SMTP time or bounced to the MAIL FROM where possible (as per
    existing RFCs); or

2.  Explicitly update RFC2821 6.1

Even a change from MUST to SHOULD (which would seem entirely
reasonable to me in the context of MARID) is a change to RFC2821 and
should be documented as such.

But having a MARID RFC that is inconsistent with RFC2821 (without
updating it) would seem be be unhelpful...  Having one that is
implicitly so, even more so...

    -roy

---

From: Dave Crocker <[email-masked]>
Date: Mon, 21 Jun 2004 14:12:03 +0800

John,

JL>    It does, however, authenticate that you're talking with an SMTP client
JL> worthy of some level of trust

Given what follows in the rest of the exchange, let me suggest
somewhat different wording, in order to make sure that the difference
between authentication and accreditation are completely clear, here.
(I know John know it, but want to make sure this thread says if VERY
explicitly.  This is a topic that has been extremely confusing in this
forum.):

A successful SMTP Auth gives the receiving smtp server a basis for
believing that it knows who the sending smtp client is. With that
assurance about the identity of the client, the server can proceed to
assess the authorization (permission to be an smtp client) and
accreditation (degree of trust to give) appropriate for the client.

JL>    Having said that, it's hard to imagine the case where host name
JL> authentication would be the only thing missing _and_ SMTP AUTH would
JL> be in use.
>> So it perhaps shouldn't be part of the spec, if the purpose is just to
JL>    I believe Dave included it for completeness of background, and it
JL> really isn't part of our proposal.

CSV does not attempt to carefully restrict the combinatorial outcomes
of the component sequences.  (In english:  No doubt there are silly
combinations that can occur; we didn't worry ab out it.)

The HNA stuff really only attempt to document exsiting mechanisms that
can be used for the necessary authentication.  That's why it made
sense to move it to the CSV appendix rather than keep it in a separate
document.

JL>    However, we realize there _will_ be cases where the SRV lookup doesn't
JL> return the matching IP address, but local policy may recognize STARTTLS
JL> as "sufficient authentication".

In order to avoid confusion, I am inclined to use language like "there
will be cases where the DNS server does not return the matching IP
address as Additional Information to the SRV lookup.  It really isn't
the SRV record that is returning the address(es) and even the DNS
server is not obligated to.  It's only a (very useful) efficiency hack.

d/

---

From: Tony Finch <[email-masked]>
Date: Mon, 21 Jun 2004 10:18:32 +0100

> accreditation (degree of trust to give) appropriate for the client.

SMTP AUTH doesn't preclude lying in the HELO line. The use of consistent
SASL credentials from a particular sender does not imply they use a
consistent HELO domain. SMTP AUTH does not authenticate the data that CSA
and DNA use to look up the authorization and accreditation, therefore it
isn't suitable for use with CSA and DNA -- unless some additional
mechanism is documented that fixes this bug.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 24 Jun 2004 18:42:17 +0800

Matthew,

ME> Have you considered and rejected or not considered adding the NBB idea
ME> that I threw out a while back to the CSV spec? 
...
ME> Take CSV, and add a new requirement: mail that has failed (as in
ME> there is a MARID record AND the sending IP isn't there AND there's no
ME> ?all) a 2821.FROM check MUST NOT be bounced;

CSV provides a system for obtaining information about a sending SMTP
client.  It makes no attempt to dictate how the receiving SMTP server
should use that information.

This is not an accident.

For one thing, that would cross the line into a much fuzzier area, as
some of the discussions about this already demonstrated.

For another, my own experience is that specifications that are too
broad get complicated, take a long time, and ultimately are not very
useful.

ME> instead it MUST either be
ME> refused at SMTP time, or accepted and destroyed.   In other words, DON'T
ME> require SRS, but DO require that mail that goes via non-SRS systems not
ME> lead to bounces to  systems that didn't originate the original message.

SRS?  you mean re-writing stuff?  I did not think that CSV said
anything at all about address re-writing.  At least, I sure hope we
didn't...

d/