Subject: Re: [Asrg] 3. Requirements Document

From: Kee Hinckley <[email-masked]>
Date: Thu, 25 Sep 2003 10:20:55 -0400

At 8:39 AM -0400 9/25/03, [email-masked] wrote:
>  > Looking over the (as-yet-undefined) definitions I see that there is a
>>  1.3.31 for "Sender" but I don't see anything that would distinguish
>>  between the originator of the message and the organization handling the
>>  delivery.  This is important because the two roles may be separate, in
>>  which case the best practices for each will be somewhat different.
>>
>
>Hmm, that's a good point. I think we should make it clear that the
>"Sender" and "Receiver" of the email refer to the human beings or
>software agents which originally sent the email, not the MUA or MTA that
>was used. If that is true then what does "Message Originator" refer to?
>
>In general, we need to look over the definitions and decide which are
>relevant and which are not, and make clear the distinctions between
>different aspects. We might also need another section in this document
>which outlines the mail delivery model and plugs in all of the
>definitions, so the reader of the document can readily see the
>definitions "in action".
>
>EW> There is a model presented and the points Kee raises are valid.  I feel
>that the distinctions that are raised may be relevant to a discussion of
>delivery methods (these types of disctions were revised in a later draft,
>which I will forward now) and origination agent (MUA) and "Sender" are
>presented as distinct definitions.

I may just be misunderstanding you, but I'm still not sure I've made the point.

The majority of non-spam bulk mail on the internet is not being sent 
by the entity that composed the mail and owns the list.  It's being 
sent by third-party companies on their behalf.  So in a typical 
situation you have a company that shows up with a database of email 
addresses, they sign up with a bulk sender, and the sender gives them 
an interface to compose their messages.  The bulk sender usually 
makes an attempt to clean the user's list and make some determination 
of whether it's a "valid" list.  But at some point they send out the 
email on behalf of the entity that composed the message.

Of course it can be far more complicated than that, since there will 
be automated messages (subscription notifications, replies, support 
email, commerce notifications) that might go directly from the bulk 
mailer without the interaction of the list owner.

There is also a tremendous amount of behind-the-scenes activity going 
on here that I think most people have no idea about.  Legit bulk 
senders expend considerable effort working with major ISPs to make 
sure their email doesn't get blocked.  ISPs have trust levels 
associated with different IP addresses.  Bulk senders use separate IP 
addresses for known good customers and new customers....

Anyway.  One of the major issues in looking at consent is that there 
is a gap in the verification process.  When a customer hands a list 
to a bulk sender there is no way currently to verify that the 
customer has in fact followed any BCP for consent.

I believe the document needs to call out each of those roles 
separately, and any system needs to consider the proper actions for 
situations in which the list owner and list sender are separate roles.

Also, another item that may want to go in the definition list is 
e-pending, in which a company takes a list of off-line addresses and 
figures out (perhaps) what the appropriate email address is for those 
addresses.  Needless to say this throws an interesting monkey-wrench 
into the "can send email if you have a prior business relationship" 
model.

>EDW> This as well was changed in a later draft.  Unfortunately, these
>subsequent drafts were being discussed off-list.  The document which I
>forward may be a closer fit to the ideals of the list at present.
>
>BTW, more people are needed to work on this, can the volunteers step
>forward?

Those two statements are related.  I think if you want volunteers you 
need to keep the discussion on list.  It may make things more 
difficult with side-tracks and interruptions, however it will 
generate more interest and participation.  (How much it will help 
find people to write things up though, I don't know.  Certainly my 
participation on this list goes in waves, depending on my 
workload--that's not what you'd want from a document owner.)

Also you have another fundamental problem.  You've backed off to a 
level of abstraction where agreement is possible--but that also means 
it's far enough away from the problem that people don't see its 
relevance.  My personal feeling is that the most valuable thing that 
could come out of this group is a strong statement of consent, with a 
particular goal of defeating the U.S. Congress', and DMA's attempt to 
make opt-out the standard model.  We all know that won't work.  But a 
united front from the internet groups behind a common definition of 
what levels of consent must be required would help.  Along with that 
model, could come a set of BCPs for senders, receivers, spam filters 
and ISPs.  There are other groups working on portions of that (see 
http://www.isipp.org/standards.html, a document that came out of the 
Summit I and Summit II conferences that brought major senders and 
receivers together to talk about email deliverability).  I think that 
ASRG (in it's newer, mellower form) could contribute to that process.
-- 
Kee Hinckley
http://www.messagefire.com/         Next Generation Spam Defense
http://commons.somewhere.com/buzz/  Writings on Technology and Society

I'm not sure which upsets me more: that people are so unwilling to accept
responsibility for their own actions, or that they are so eager to regulate
everyone else's.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 25 Sep 2003 08:39:57 -0400

Kee, Yakov and List participants:

The latest draft of the Requirements may be accessed at:

http://www.infobro.com/anon-FTP/infoSource/IRTF/ASRG/draft-irtf-asrg-require
ments-xx-05.txt

more below...

Original Message:
-----------------
From: Yakov Shafranovich [email-masked]
Date: Wed, 24 Sep 2003 15:58:19 -0400
To: [email-masked], [email-masked], [email-masked]
Subject: Re: [Asrg] 3. Requirements Document

Kee Hinckley wrote:

> http://www.irtf.org/asrg/draft-irtf-asrg-requirements-xx-01.txt
> 

>> 2.7  Highly Effective
>>
>>  The proposal MUST provide an effective solution as determined
>>  by the [TBD] algorithm.  That blocks all appropriate messaging.
>>
>> 2.7.1     Rationale:
>>
>>  Although proposals may differ in approach all proposals will be
>>  evaluated using an appropriate application of the [TBD]
>>  algorithm or one of its derivatives.  The [TBD] algorithm
>>  provides a standard and objective measure for evaluation of the
>>  approach.
> 
> 
> I don't think I agree with this.  In particular, a set of orthogonal 
> small changes might as a whole be more likely to be implemented, and 
> more effective, than any single change.  Are we saying that small 
> incremental changes are ruled out simply because they are only partially 
> effective?  What happened to the idea of driving spammers into a corner?
> 

This has been mentioned before on the list (gmane.org is down so I can't 
find the article). We need to change or take out this requirement. 
HOWEVER, it would be very useful to provide another requirement on how a 
proposal fits within the consent framework (or any other framework) so 
we can see how it works in combination with other systems. We need 
proposal authors to outline how their solutions would work in 
combination with other systems.

EDW> Again the later (-05) draft attempts to address these concerns.  I
will forward that to you, Kee and the list so you can mull it over.

> 2.8 (Persistently Effective) might also want to be clarified.  I agree 
> that the solution needs to be persistent, but in the sense that although 
> spammers may find ways to avoid the measure, the measure itself cannot 
> be be trivially defeated.  Again, the idea is to paint spammers into a 
> corner.  If a particular technique can be avoided, but only by exposing 
> spammers in some other useful way, then I would deem it to be 
> persistently effective.
> 

Agreed.
EDW> Agreed.  See -05 draft.

>> 2.10 Single Solution
>>
>>  The proposal SHOULD provide a comprehensive solution that
>>  impacts the greatest number of problems caused by [spam], and
>>  usable or accessible by the greatest number of MTS users.
>>
>> 2.10.1    Rationale:
>>
>>  A single solution that will work for all people is the optimum
>>  solution, designers should strongly consider methodologies or
>>  approaches that impact the largest number of MTS users.  A
>>  solution that is optimal will also impact all of the problems
>>  associated with [spam], such as [list of problem terms].
> 
> 
> Again, this seems wrong to me.  This isn't a technical problem that you 
> solve, it is a battle in which you need to contain the adversary. I 
> think a search for a single overriding solution runs the serious risk of 
> missing smaller steps which are far more likely to be adopted and, in 
> aggregate, more effective.
> 
> 

This has been mentioned before on the list, and needs to be removed. See 
this list message from Paul:

https://www1.ietf.org/mail-archive/working-groups/asrg/current/msg06657.html

EDW> Indeed.

 -e

--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .

---

From: Jonathan Morton <[email-masked]>
Date: Thu, 25 Sep 2003 18:49:49 +0100

> You've backed off to a level of abstraction where agreement is 
> possible--but that also means it's far enough away from the problem 
> that people don't see its relevance.

I have to concur with this.  Even though it's reasonable to assume most 
list members are technical folk, that does not imply in the slightest 
that they have experience or even interest in "formal methods".  I paid 
enough attention in Formal Methods lectures to pass the exam, but I 
wouldn't feel at all comfortable working under such a thing, let alone 
defining one.

With regard to the Requirements and Technical Considerations documents, 
it could simply be that most of the non-comments are because readers 
don't see anything wrong with the documents as they stand.

> My personal feeling is that the most valuable thing that could come 
> out of this group is a strong statement of consent, with a particular 
> goal of defeating the U.S. Congress', and DMA's attempt to make 
> opt-out the standard model.

Agreed.

--------------------------------------------------------------
from:     Jonathan "Chromatix" Morton
mail:     [email-masked]
website:  http://www.chromatix.uklinux.net/
tagline:  The key to knowledge is not to rely on people to teach you it.

---

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 25 Sep 2003 19:24:30 -0400

> http://www.infobro.com/anon-FTP/infoSource/IRTF/ASRG/draft-irtf-asrg-require

I wanted to add some more comments as well, see inline below.

> 
> Anti Spam Research Group                                        R. Brand
> Internet-Draft                             Procinct Security Corporation
> Expires: Mmmmmmm D, YYYY                                     E. Williams
>                                                Information Brokers, Inc.
>                                                             May 29, 2003
 >........

Don't forget to update the dates, when a new draft is published.

> .......
> 1.3.1     Anti-spam (Requires consensus definition - RCD)
> 

What exactly does this mean? Anti-spam tools, anti-spam people?

> 1.3.2     Bulk E-mail (RCD)

Are we seeking a definition of spam here?

> 1.3.3     Caller (RCD)
> 
> 1.3.4     Callback (RCD)
> 
> 1.3.5     Challenge/Response System (RCD)
> 

Are the above three terms refering to C/R systems with C/R being a 
special case of callback?

>  
> 1.3.8     Commercial E-mail (RCD)
> 

What's the difference between this and "bulk email" above? Do we need both?

> 
> 1.3.14    Message Content
> 
>  That part of a messaging transaction that contains data as
>  described by [4],[9].  Message content consists of 
>  RFC2822/RFC822 compliant data.
> 

Is this refering specifically to RFC 822 headers and message body to the 
exclusion of the SMTP transaction?

> 
> 1.3.25    Non Consent Based Communication (spam) (RCD)
> 

Again what's the difference between this, "bulk email" and "commercial 
email" above?

> 1.3.27    Opt-In (RCD)
> 1.3.28    Opt-Out (RCD)

Are we looking for a way to have a machine-readable proof of "opt in" 
and "opt out" here or does this refer to the transaction that happens 
between the human receiver and sender? Also, would "consenting" to 
something imply "opt in" and non-consent or revoking consent imply "opt 
out"?

> 
> 1.3.36    Unsolicited Bulk E-Mail (RCD)
> 
> 1.3.37    Unsolicited Commercial E-Mail (RCD)
> 

Again, we have give definitions here: bulk email, commercial email, UBE, 
UCE and non-consented email (spam). Do we really need all five? Do we 
need to define all five, or should we just leave the definition to the 
users as the charter has done?

---

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 25 Sep 2003 19:32:00 -0400

Kee Hinckley wrote:
> ......... 
> need to keep the discussion on list.  It may make things more difficult 
> with side-tracks and interruptions, however it will generate more 
> interest and participation.  (How much it will help find people to write 
> things up though, I don't know.  Certainly my participation on this list 
> goes in waves, depending on my workload--that's not what you'd want from 
> a document owner.)
> 

You are definatly right! I think that with the subject numbering system 
SOME form of order is achieved, so there it is easier to follow then it 
has been before. I will endeveor to keep discussions on-list as much as 
possible. HOWEVER, as RFC 2014 states certain technical issues that 
devolve into "rat holes" would have to be decided off-list.

> Also you have another fundamental problem.  You've backed off to a level 
> of abstraction where agreement is possible--but that also means it's far 
> enough away from the problem that people don't see its relevance.

What would you suggest to remedy this problem?

>  My 
> personal feeling is that the most valuable thing that could come out of 
> this group is a strong statement of consent, with a particular goal of 
> defeating the U.S. Congress', and DMA's attempt to make opt-out the 
> standard model.  We all know that won't work.  But a united front from 
> the internet groups behind a common definition of what levels of consent 
> must be required would help.   Along with that model, could come a set of
> BCPs for senders, receivers, spam filters and ISPs.  There are other 
> groups working on portions of that (see 

Defining "consent" is a major issue, I will reply to that in a separate 
posting.

---

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 25 Sep 2003 19:45:03 -0400

> wouldn't feel at all comfortable working under such a thing, let alone 

Well guys, as much as we may hate doing everything formally, sometimes 
we have to. We are definatly open to any suggestions as to how to make 
this process a bit easier, but I don't think we can get away from doing 
this :) We need some document or model to evaluate proposals, and the 
requirements document is one way of doing it. If you can think of other 
ways, please let us know.

> don't see anything wrong with the documents as they stand.

Well the requirements document is missing definitions, especially one 
for "consent". There are some other strong issues, such as whether to 
replace SMTP.

>> opt-out the standard model.

I will be starting a separate thread of defining consent where we can 
start on this.

---

From: Jonathan Morton <[email-masked]>
Date: Fri, 26 Sep 2003 01:00:24 +0100

>> I have to concur with this.  Even though it's reasonable to assume 
>> most list members are technical folk, that does not imply in the 
>> slightest that they have experience or even interest in "formal 
>> methods".  I paid enough attention in Formal Methods lectures to pass 
>> the exam, but I wouldn't feel at all comfortable working under such a 
>> thing, let alone defining one.
>
> this process a bit easier, but I don't think we can get away from 
> doing this :) We need some document or model to evaluate proposals, 
> and the requirements document is one way of doing it. If you can think 
> of other ways, please let us know.

I don't think the Requirements and Technical Considerations documents 
are excessive, in themselves.  We do have to be relatively careful to 
avoid needlessly duplicating information between them, otherwise the 
definitions are likely to get desynced.

Actually, I'm not entirely sure that they aren't already duplicated (in 
content, if not intent or audience).  I wrote my Email Authentication 
proposal against the TCs, and that lets me see quite a lot of 
redundancy in the Reqs.

Anyway, my point is that there's a perception of bureaucracy, if you 
say "we must do this [boring paperwork] *before* we can consider that 
[which might be practical]".  I appreciate that is probably not your 
intent.  It might be useful to step back and clarify the overall 
situation for us.

---

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 25 Sep 2003 20:17:18 -0400

Jonathan Morton wrote:

> proposal against the TCs, and that lets me see quite a lot of redundancy 
> in the Reqs.
> 

They definatly DO duplicate each other and I have made that point to 
both authors off-list. Some syncing is needed.

> Anyway, my point is that there's a perception of bureaucracy, if you say 
> "we must do this [boring paperwork] *before* we can consider that [which 
> might be practical]".  I appreciate that is probably not your intent.  
> It might be useful to step back and clarify the overall situation for us.
> 

I changed the subject to reflect the subject matter a little bit better. 
You are correct - this is definatly not my intent to create bureaucracy. 
As a matter of fact, RFC 2014 goes out of its way to state that the IRTF 
and its RGs should be much less bureacratic and more flexible.

The spam problem is pretty big and there are many aspects to it. The 
problem has been that everyone has a proposal on how to solve the 
problem. There are probably hundreds of those on the Internet and 
several dozens in the ASRG at least. We cannot seriously look and 
evaluate all of them without having a common baseline for evaluation. 
This boring paperwork is one approach to that.

However, aside from all of that, the ASRG is intended to coordinate spam 
research - many other aspects of it beside proposals. Ideally this 
should be a central point for anti-spam research, giving people a place 
to connect with other interested parties, allowing research and papers 
to be peer-reviewed and commented on, etc. We just don't see the 
non-proposal stuff mentioned too often.

Please let me know if you need further clarification and I welcome any 
suggestions for improving the ASRG.

Yakov

---

From: Jonathan Morton <[email-masked]>
Date: Fri, 26 Sep 2003 01:29:07 +0100

> something imply "opt in" and non-consent or revoking consent imply 
> "opt out"?

I think that section of the document is attempting to define terms.  
With that in mind, I'll see if I can supply some possibly-appropriate 
text here:

-- snip --

1.3.27 Opt-In

An opt-in communication is one which the recipient has explicitly 
requested.  For example, most e-commerce websites will provide a 
checkbox when asking for personal information, which indicates your 
willingness to receive marketing information from them (and, in 
practice, their immediate associates).  For contrast with "double 
opt-in" as described below, this practice is sometimes referred to as 
"single opt-in".

NB: many e-commerce forms have the checkbox ticked by default, and 
require you to manually unset it before submitting the form.  This 
barely qualifies as opt-in - some parties may legitimately feel that it 
qualifies as an opt-out system.

Mailing lists, as commonly used for announcements and discussions, 
typically operate a "double opt-in" system.  This is designed to verify 
that the e-mail address given by the subscriber actually belongs to 
him, and therefore that the subscriber is in a position to opt-in for 
that address.

Double opt-in is a standard feature of mailing list software, such as 
Mailman and Majordomo, and is normally enabled by default.  It is 
widely accepted that double opt-in is the strongest way to preserve 
privacy in a bulk-communications environment.

Typically, personal and business e-mail conversations can be considered 
as opt-in communications - normally, one party will invite the other by 
either online or offline means.  It is normally expected, as a matter 
of common courtesy, that a simple business conversation is not 
considered opting-in for indiscriminate marketing material, or similar.

1.3.28 Opt-Out

An opt-out communication is one where the recipient's address is sent 
mail without their explicit consent, but which contains language 
similar to "to remove yourself from this list and stop receiving 
communication from XYZ Corp, do this".

The biggest problem with this model is that there are literally 
thousands of entities who might want to send unsolicited information, 
and the recipient must explicitly say "go away" to each and every one 
of them.  There cannot be any reasonable expectation that recipients 
can keep up with this system.

Many of these entities will, additionally, interpret the opt-out 
request as confirmation of a "live" address, and even if they comply 
with the request themselves, may pass on the address to other 
unsolicited-mail senders.

---

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 25 Sep 2003 20:36:32 -0400

Kee Hinckley wrote:
> Anyway.  One of the major issues in looking at consent is that there is 
> a gap in the verification process.  When a customer hands a list to a 
> bulk sender there is no way currently to verify that the customer has in 
> fact followed any BCP for consent.
> ........
> 
> My 
> standard model.  

I would like to use this thread for defining what consent actually 
means. The ASRG charter (http://www.irtf.org/charters/asrg.html)  states:

"The definition of spam messages is not clear and is not consistent 
across different individuals or organizations. Therefore, we generalize 
the problem into "consent-based communication". This means that an 
individual or organization should be able to express consent or lack of 
consent for certain communication and have the architecture support 
those desires."

It seems that there are two different aspects of consent: consent 
between the two parties in the real world, and how those decisions are 
expressed by machines. We need to define both of these.

Additionally, the charter states:

"The research group will investigate the feasibility of: (1) a single 
architecture that supports this and (2) a framework that allows 
different systems to be plugged in to provide different pieces of the 
solution. "

We also need to determine whether it is viable to have a consent-based 
architecture such as the one defined in the consent framework 
(http://www.solidmatrix.com/research/asrg/asrg-consent-framework.html).

---

From: Jonathan Morton <[email-masked]>
Date: Fri, 26 Sep 2003 01:42:37 +0100

> (http://www.solidmatrix.com/research/asrg/asrg-consent-> framework.html).

I'm afraid that document is not very readable.  :-(  It's about 50%  
capital letters, and scans about as well as the fine print on an  
insurance policy.

---

From: Yakov Shafranovich <[email-masked]>
Date: Thu, 25 Sep 2003 20:46:52 -0400

Jonathan Morton wrote:
>> We also need to determine whether it is viable to have a 
>> consent-based  architecture such as the one defined in the consent 
>> framework  (http://www.solidmatrix.com/research/asrg/asrg-consent-> 
>> framework.html).
> 
> 

I have to get around to converting it to ASCII (if any can volunteer, 
please). Capital letters for model terms have been used by the IETF in 
other similar models, and I followed that convention. But even without 
this document, the question remains - is a consent architecture viable?

---

From: Phil Miller <[email-masked]>
Date: Mon, 29 Sep 2003 20:03:51 -0400

[email-masked] wrote:
> <http://www.infobro.com/anon-FTP/infoSource/IRTF/ASRG/draft-irtf-asrg-requirements-xx-05.txt>

TOC: 'Rationale*:*' -> 'Rationale' or just remove Rationale from TOC in 
post-processing

1.3.11,12: These should be removed, since they are specific to a certain 
form of solution.

1.3.21: For clarity, perhaps you could say "A [MTA] is an *entity* 
within the [MTS] that participates ... or other MTA*s*.

1.3.22: This diagram is not inclusive of the degenerate case of a 
message sent between 2 users on the same end-point MTA. For example, I 
have certainly received spam from other Yahoo users in my Yahoo email 
account.

1.3.24: Why not collapse this to just MSA, and consider a local store 
maintained by the MUA as part of the MUA. Thus, MSA would refer only to 
POP3/IMAP/etc. servers. We could stand to deprecate the term 'Message 
Store' and become consistent in the process

1.3.26: How about "An MTA that is configured to accept and relay 
messages on behalf users that are not explicitly authorized to use it in 
this manner."

1.3.27,28: Perhaps irrelevant to the current discussion? This is not 
meant to be a general glossary of consent-based electronic messaging, 
but we could use a superset of this list for such a thing.

1.3.32: Considering the way the existing MTS works, this should be 
reworded to not exclude interaction with MUAs, but to emphasize the 
primary role as a non-endpoint (Sender/Receiver) MTA. Perhaps just 
stating "An MTA that is neither the Sender nor Receiver in a particular 
messaging transmission sequence, but participates in it."

1.3.33: "A Relay MTA which enforces a specified policy to conditionally 
relay messages as part of a messaging transmission sequence."

1.3.36,37: Unncessary as relevant definitions. Perhaps worth mention as 
current Non Consent Based Communication.

1.4: Much of this 1st paragraph should be incorporated into 1.3.22, as 
it is continuing the definition of the MTS. The last sentence should 
probably read "The MTS uses various transport, envelope, and message 
technologies that are standardized and well known."
Bullet a: "Which component of the MTS is the platform for the proposed 
solution,"
b: "Whether the proposed solution makes no changes to existing protocols 
[i.e. policy enforced with existing protocol], is an extension of 
existing protocols, or is an entirely new protocol,"
c: "Whether the proposed solution uses existing technologies, or depends 
on new technologies."

2.3.1: "For successful adoption by the user community*,* proposals must 
provide support for typical and atypical messaging situations." The next 
phrase, "involving 2 or more parties" is superfluous. The final sentence 
is obtuse.

Please keep us up to date on draft revisions. Although I haven't the 
time to generate these documents, I can definitely comment on them.

Phil Miller