Subject: [Asrg] 3. Requirements - Consent Framework

From: Jonathan Morton <[email-masked]>
Date: Wed, 1 Oct 2003 06:41:06 +0100

I think I have got a handle on Yakov's proposed Consent Framework, or 
at least what it attempts to achieve.  The actual document remains 
sadly opaque to me.  I'll try to summarise my understanding below.  
Yakov, feel free to correct me if I'm completely off-base.

The "Consent Framework", while presently documented in an extremely 
formal and opaque manner, is nothing more than a "system of systems", 
aimed at combining the best-of-breed proposals (the exact ones to be 
determined at some time in the future) into a single, cohesive, 
implementable e-mail system.  Preferably, of course, leveraging the 
existing SMTP infrastructure, because that makes implementation several 
orders of magnitude easier.

I believe another goal is for additional components to be added to the 
framework as and when necessary, and therefore the system will be 
governed by some extensible protocol and/or message formatting 
standards.

The main components of the framework might be:

- Sender identification
- Whitelist for stable sender/recipient relationships
- Monetary payment (using tokens issued by trusted authorities, eg 
e-stamps)
- Computational payment (generating tokens autonomously, eg hashcash)
- Trust directories (identifying which payment and/or authentication 
authorities are trustworthy)
- Message category labelling
- BCPs for effective recipient filtering based on above facilities
- BCPs for senders (of various categories) for efficient use of above 
facilities

Based on the above, it might be a good idea to split my own "E-mail 
Authentication" paper into two parts, one dealing with the 
authentication itself, and one dealing with establishing the 
trustworthiness of the MAA (Mail Authentication Agent).  Then both 
would fit neatly into positions in the above list, along with other 
people's hashcash, stamp authority, labelling, and whitelist schemes.

With any luck, that should make things a heap clearer for everyone 
(myself included), unless of course Yakov is going to point out how 
incorrect I am.  :-)

--------------------------------------------------------------
from:     Jonathan "Chromatix" Morton
mail:     [email-masked]
website:  http://www.chromatix.uklinux.net/
tagline:  The key to knowledge is not to rely on people to teach you it.

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Yakov Shafranovich <[email-masked]>
Date: Wed, 01 Oct 2003 02:10:10 -0400

Jonathan Morton wrote:
> I think I have got a handle on Yakov's proposed Consent Framework, or at 
> least what it attempts to achieve.  The actual document remains sadly 
> opaque to me.  I'll try to summarise my understanding below.  Yakov, 
> feel free to correct me if I'm completely off-base.
> 

Just a minor correction on the subject line, the consent framework has a 
separate item number. Also, I am basing the framework on the charter, so 
I don't know if it is correct to call it mine.

> orders of magnitude easier.

Sorry for being "formal and opaque" but I followed the similar model 
used for IMs defined by the IMPP WG (see RFC 2778). I have been 
considering for a long time whether to rewrite the document to be less 
formal and opaque, and recent discussions especially the thread with 
Gordon has been strongly pointing in that direction.

Jonathan is correct - the consent framework seeks to tie in all of the 
existing and future solutions into one system. The central tie-in point 
to the whole framework is the notion of consent which is presently 
absent from the email system while being present in some form in other 
messaging systems such as Instant Messaging (IM). From the charter 
(http://www.irtf.org/charters/asrg.html):

"Therefore, we generalize the problem into "consent-based 
communication". This means that an individual or organization should be 
able to express consent or lack of consent for certain communication and 
have the architecture support those desires."

and

"The research group will investigate the feasibility of: (1) a single 
architecture that supports this and (2) a framework that allows 
different systems to be plugged in to provide different pieces of the 
solution."

What the consent framework tries to do is to introduce this notion in a 
standard way with protocols and formats, that can be carried over all 
anti-spam tools. But I think Jonathan has expressed this in much better 
way than me :)

> governed by some extensible protocol and/or message formatting standards.
> 

Correct.

> - Message category labelling

The charter summarizes it into three components:

o Consent Expression Component: This involves recipients expressing a 
policy that gives consent or non-consent for certain types of 
communications
o Policy Enforcement Component: This involves subsystems within the 
communication system that enforce the policy.
o Source Tracking Component: This component provides deterrence to 
parties that consider violating the policy by facilitating 
identification and tracking of senders that violate the policy

Consent expression component interfaces with the user to specify what 
settings the user wants. Policy enforcement components are the anti-spam 
tools that actually process the mail based on those settings and input 
provided by source tracking components. Most of the items mentioned 
above would be source tracing components with the whitelist possibly 
being the policy enforcement component.

Sorry for being "formal" but I am following the terminology of the charter.

> - BCPs for senders (of various categories) for efficient use of above 

BCPs are basically the policy documents to make sure that the framework 
is used properly - kind of like manuals.

> trustworthiness of the MAA (Mail Authentication Agent).  Then both would 
> fit neatly into positions in the above list, along with other people's 
> hashcash, stamp authority, labelling, and whitelist schemes.
> 

Jonathan - your explanation has been extremely heplful. As a matter of 
fact, it seems that I will need additional help rewriting the framework 
document, specifically on how to rewrite it so "normal" humans can 
understand it :) If you wish to help with the writing and the editing, 
please contact me.

Yakov

---

From: Jonathan Morton <[email-masked]>
Date: Wed, 1 Oct 2003 10:43:00 +0100

>> existing SMTP infrastructure, because that makes implementation 
>> several orders of magnitude easier.
>
> existing and future solutions into one system. The central tie-in 
> point to the whole framework is the notion of consent which is 
> presently absent from the email system while being present in some 
> form in other messaging systems such as Instant Messaging (IM).
>
> What the consent framework tries to do is to introduce this notion in 
> a standard way with protocols and formats, that can be carried over 
> all anti-spam tools. But I think Jonathan has expressed this in much 
> better way than me :)
>
>> I believe another goal is for additional components to be added to 
>> the framework as and when necessary, and therefore the system will be 

Excuse me while I reshuffle these...

> identification and tracking of senders that violate the policy

This is covered by "sender identification" and "trust directories".

> communication system that enforce the policy.

The remaining mechanisms in my list (except the BCPs) fill this role.

But really, I think the distinction between the two is pretty slim.  
I'd rather have one Consent Verification Component with all the 
mechanisms put together.

> policy that gives consent or non-consent for certain types of 

And this, I think, is the part which hasn't really been addressed yet - 
at least, not by itself.  I think the assumption has been that it'll 
mostly live in the MUA, and is therefore safe to leave as a 
implementation detail.  Are you suggesting that the Policy Enforcement 
should mostly live on the MTA, and therefore the recipient's MUA must 
express consent (really, filtering preferences) by communicating with 
the MTA?

To be honest, I'm not sure that the MTA is usually in a position to 
receive filtering commands from the recipient's MUA.  An MUA usually 
retrieves mail using POP3 or IMAP, not SMTP, which rather disconnects 
it from the inbound MTA - which in large ISPs is frequently 
disconnected from the outbound smarthost which the MUA *does* connect 
to via SMTP.

However, a logically separate MFA (Mail Filtering Agent) would perhaps 
be plausible - SpamAssassin run in "spamd" mode might be an example of 
this role.  The MFA might be attached to the MTA, or be a plugin to the 
MUA, or even reside on a separate server (potentially run by the ISP or 
a third party) and be called by the MUA.

Also, I think there's a fourth component missing from your list - the 
sender has to attach the relevant metadata/tokens to the message at (or 
near) origin.  Most of the mechanisms I suggest must interact with this 
component, in addition to the others.  This component probably has to 
include the concepts of mail-forwarding and listservs.  Perhaps it 
might be called the Message Introduction Component?

Or was this what you originally meant by the Source Tracking Component? 
  If so, I think the "mechanisms" as I listed would be orthogonal to the 
"components" that you speak of - so instead of placing them into one 
category or the other, we have to see which mechanisms need part of 
their implementation in each component.

> Most of the items mentioned above would be source tracing components 
> with the whitelist possibly being the policy enforcement component.

No...  only two of them attempt to "trace" the source of the message.  
The remainder are used to find out about the "worth" of the message 
itself.  The final filtering decision is presumably based upon 
information from all of the mechanisms.

For example, the lack of strong traceability of a particular message 
would normally cause the whitelist information to be treated as less 
reliable, but the lack of a strong whitelisting would probably be 
offset if it carried a (monetary or computational) payment token.  
Also, if the credentials carried by the message were found to have been 
generated by a discredited authority, this would probably reflect 
negatively on the message as a whole, whatever it carried.

> BCPs are basically the policy documents to make sure that the 
> framework is used properly - kind of like manuals.

Still an essential part of the system.

> fact, it seems that I will need additional help rewriting the 
> framework document, specifically on how to rewrite it so "normal" 
> humans can understand it :) If you wish to help with the writing and 
> the editing, please contact me.

Yes, I think I could do that.

But in particular, I hope that more list members now realise that the 
Consent Framework is in fact an important and useful document.  There 
are a lot of proposals coming in which try to cover multiple aspects of 
the system at once (usually doing one well and others poorly), and I 
suspect this is due to ignorance of the framework and it's intentions.

It would be overall better if proposal authors were more aware of the 
Consent Framework, and were thus able to design their mechanisms as 
single-purpose plugins to it, instead of jack-of-all-trades 
megaproposals.

---

From: "Jon Kyme" <[email-masked]>
Date: Wed, 01 Oct 2003 12:09:08 +0100

>disconnected from the outbound smarthost which the MUA *does* connect 

A policy enforcement component might be implemented at the same point (or
perhaps *by*) an upstream MTA. In fact anything that does this job is, I
suppose, a MTA (or MDA) by definition.

This is how things are done in a WebMail system that I'm familiar with.
Message disposition is implemented by the delivery component of the MTA
software (somewhat) under the control of the user. A problem is that there
is no *standard* way to convey recipient policy back towards (not
necessarily *to*) the sender. So it's only possible to configure Policy
(MTA filtering) via the proprietary MUA software (in this example). If
someone chooses to use the account via "pine" (for instance) they lose the
ability to configure their "Spam Protection" / other filtering. i.e - this
can only be done via the web interface.

Also, it's not easy to propagate user policy beyond the terminal MTA (not
even to 2ndary MXs) since you're likely to be talking about heterogenous
MTA software.

A standard policy / consent expression mechanism would allow any MUA
implementing it to communicate with any other MTS component which also
implements it.

>a third party) and be called by the MUA.

MFA = instance of "policy enforcement component" surely?

--

---

From: Yakov Shafranovich <[email-masked]>
Date: Wed, 01 Oct 2003 16:32:40 -0400

> MFA = instance of "policy enforcement component" surely?

I don't think it makes a difference where exactly the policy enforcement 
component resides - in the MTA itself or as another system. As per charter:

"Policy Enforcement Component: This involves subsystems within the 
communication system that enforce the policy. THE OVERALL FRAMEWORK MAY 
INVOLVE MULTIPLE SUBSYSTEMS WITHIN THE POLICY ENFORCEMENT COMPONENT.
"

No reason to complicate it by including another acronym.

---

From: Yakov Shafranovich <[email-masked]>
Date: Wed, 01 Oct 2003 16:46:42 -0400

Jonathan Morton wrote:
> But really, I think the distinction between the two is pretty slim.  I'd 
> rather have one Consent Verification Component with all the mechanisms 
> put together.
> 

That's open to discussion. Keep in mind that the charter is only 
suggesting, not dictating, as stated:

"POSSIBLE components of such a framework may include:"

> retrieves mail using POP3 or IMAP, not SMTP, which rather disconnects it 
> from the inbound MTA - which in large ISPs is frequently disconnected 
> from the outbound smarthost which the MUA *does* connect to via SMTP.
> 

The choice of where it lives is left ot the implementators. It maybe a 
MUA that is also doing the spam processing, it could be a MUA 
communicating with an MTA, it could be a MUA communicating with an 
anti-spam tools, or it could be a dedicated GUI or webpage communicating 
with anti-spam tools or MTAs. All possibilies are open and we are not 
restricting to any of them.

> include the concepts of mail-forwarding and listservs.  Perhaps it might 
> be called the Message Introduction Component?
> 

You are correct - they may be part of source tracking components in 
cases such as digital signatures or e-postage. However, in other cases 
this maybe reside in the policy enforcement component. What complicates 
this more is that the sender's tagging behavior might be looked at his 
expression of consent or response to one.

You are correct about orthogonal nature of this - there are some tools 
that will only fit one category, while others will cross over.

> reliable, but the lack of a strong whitelisting would probably be offset 
> if it carried a (monetary or computational) payment token.  Also, if the 
> credentials carried by the message were found to have been generated by 
> a discredited authority, this would probably reflect negatively on the 
> message as a whole, whatever it carried.
> 

I would think that a source tracking component category would apply to 
e-postage as well, specifically when verifying whether the e-stamp is 
valid and the authority that issued it is valid as well.

Correct.

> Yes, I think I could do that.

Thanks! Please contact me off-list.

> single-purpose plugins to it, instead of jack-of-all-trades megaproposals.
> 

Awareness of the consent framework may have been bad because of the 
"opaque" nature of the document. There are some proposal guidelines in 
the works that will include a requirement to consider the consent 
framework. HOWEVER, as per my other post it is possible that there are 
other ways to look at spam solutions beside consent.

Yakov

---

From: Jonathan Morton <[email-masked]>
Date: Thu, 2 Oct 2003 03:32:18 +0100

>>> o Consent Expression Component: This involves recipients expressing 
>>> a policy that gives consent or non-consent for certain types of 
>>> communications

>> And this, I think, is the part which hasn't really been addressed yet 
>> - at least, not by itself.  I think the assumption has been that 
>> it'll mostly live in the MUA, and is therefore safe to leave as a 
>> implementation detail.  Are you suggesting that the Policy 
>> Enforcement should mostly live on the MTA, and therefore the 
>> recipient's MUA must express consent (really, filtering preferences) 
>> by communicating with the MTA?
>
> anti-spam tools, or it could be a dedicated GUI or webpage 
> communicating with anti-spam tools or MTAs. All possibilies are open 
> and we are not restricting to any of them.

OK, so you're saying that we need some kind of standard group of 
settings - I'd suggest a standard file format which can be interchanged 
between implementations - and a protocol to communicate those settings 
*where necessary*.

>> I think the "mechanisms" as I listed would be orthogonal to the 

Not necessarily.  Conceptually, an e-postage system could preserve the 
anonymity of the sender, as all the issuer really has to say is whether 
the stamp's valid or not.  Also conceptually, e-stamps could have 
intrinsic value - and could therefore be re-used for future messages by 
the recipient.  That's not to suggest such implementations would be 
easy, however...

> other ways to look at spam solutions beside consent.

Noted.

---

From: Yakov Shafranovich <[email-masked]>
Date: Wed, 01 Oct 2003 22:39:02 -0400

> between implementations - and a protocol to communicate those settings 

A standard file format and a protocol is necessary. Standard settings 
are not absolutely necessary but would probably be a set of basic stuff 
common to most systems, with the possibilities open to expansion. Both 
the format and the protocol need to be extensible as well. Keep in mind 
the example that Gordon mentioned with SPITBOL - the extension mechanism 
needs to be flexible enough to accommodate such scheme.

> the recipient.  That's not to suggest such implementations would be 

Right - anonymous digital cash, forgot about that. I think that 
anonymous digital cash has a bank signature on it also, it is just 
issued anonymously. If e-stamps actually have an intrinsic value, then 
they might belong in a different category.

---

From: "Jon Kyme" <[email-masked]>
Date: Thu, 02 Oct 2003 11:06:31 +0100

> "Policy Enforcement Component: This involves subsystems within > "
> 

Quite. Is what I meant.

--

---

From: "Pete McNeil" <[email-masked]>
Date: Fri, 3 Oct 2003 10:24:15 -0400

Apologies for my delays... I am currently heavily involved in some
critical and urgent work. I will address this at my earliest
opportunity.
_M

|-----Original Message-----
|From: Jonathan Morton [mailto:[email-masked]] 
|Sent: Wednesday, October 01, 2003 5:43 AM
|To: Yakov Shafranovich
|Cc: [email-masked]; Andrew Akehurst; Pete (Madscientist)
|Subject: Re: 4d. Consent Framework - making it clearer
|
|
|>> The "Consent Framework", while presently documented in an extremely
|>> formal and opaque manner, is nothing more than a "system of 
|systems", 
|>> aimed at combining the best-of-breed proposals (the exact 
|ones to be 
|>> determined at some time in the future) into a single, cohesive, 
|>> implementable e-mail system.  Preferably, of course, leveraging the 
|>> existing SMTP infrastructure, because that makes implementation 
|>> several orders of magnitude easier.
|>
|> Jonathan is correct - the consent framework seeks to tie in 
|all of the
|> existing and future solutions into one system. The central tie-in 
|> point to the whole framework is the notion of consent which is 
|> presently absent from the email system while being present in some 
|> form in other messaging systems such as Instant Messaging (IM).
|>
|> What the consent framework tries to do is to introduce this notion in
|> a standard way with protocols and formats, that can be carried over 
|> all anti-spam tools. But I think Jonathan has expressed this in much 
|> better way than me :)
|>
|>> I believe another goal is for additional components to be added to
|>> the framework as and when necessary, and therefore the 
|system will be 
|>> governed by some extensible protocol and/or message formatting 
|>> standards.
|>
|> Correct.
|>
|>> The main components of the framework might be:
|>> - Sender identification
|>> - Whitelist for stable sender/recipient relationships
|>> - Monetary payment (using tokens issued by trusted authorities, eg
|>> e-stamps)
|>> - Computational payment (generating tokens autonomously, eg 
|hashcash)
|>> - Trust directories (identifying which payment and/or 
|authentication 
|>> authorities are trustworthy)
|>> - Message category labelling
|>
|> The charter summarizes it into three components:
|
|Excuse me while I reshuffle these...
|
|> o Source Tracking Component: This component provides deterrence to
|> parties that consider violating the policy by facilitating 
|> identification and tracking of senders that violate the policy
|
|This is covered by "sender identification" and "trust directories".
|
|> o Policy Enforcement Component: This involves subsystems within the
|> communication system that enforce the policy.
|
|The remaining mechanisms in my list (except the BCPs) fill this role.
|
|But really, I think the distinction between the two is pretty slim.  
|I'd rather have one Consent Verification Component with all the 
|mechanisms put together.
|
|> o Consent Expression Component: This involves recipients expressing a
|> policy that gives consent or non-consent for certain types of 
|> communications
|
|And this, I think, is the part which hasn't really been 
|addressed yet - 
|at least, not by itself.  I think the assumption has been that it'll 
|mostly live in the MUA, and is therefore safe to leave as a 
|implementation detail.  Are you suggesting that the Policy Enforcement 
|should mostly live on the MTA, and therefore the recipient's MUA must 
|express consent (really, filtering preferences) by communicating with 
|the MTA?
|
|To be honest, I'm not sure that the MTA is usually in a position to 
|receive filtering commands from the recipient's MUA.  An MUA usually 
|retrieves mail using POP3 or IMAP, not SMTP, which rather disconnects 
|it from the inbound MTA - which in large ISPs is frequently 
|disconnected from the outbound smarthost which the MUA *does* connect 
|to via SMTP.
|
|However, a logically separate MFA (Mail Filtering Agent) would perhaps 
|be plausible - SpamAssassin run in "spamd" mode might be an example of 
|this role.  The MFA might be attached to the MTA, or be a 
|plugin to the 
|MUA, or even reside on a separate server (potentially run by 
|the ISP or 
|a third party) and be called by the MUA.
|
|Also, I think there's a fourth component missing from your list - the 
|sender has to attach the relevant metadata/tokens to the 
|message at (or 
|near) origin.  Most of the mechanisms I suggest must interact 
|with this 
|component, in addition to the others.  This component probably has to 
|include the concepts of mail-forwarding and listservs.  Perhaps it 
|might be called the Message Introduction Component?
|
|Or was this what you originally meant by the Source Tracking 
|Component? 
|  If so, I think the "mechanisms" as I listed would be 
|orthogonal to the 
|"components" that you speak of - so instead of placing them into one 
|category or the other, we have to see which mechanisms need part of 
|their implementation in each component.
|
|> Most of the items mentioned above would be source tracing components
|> with the whitelist possibly being the policy enforcement component.
|
|No...  only two of them attempt to "trace" the source of the message.  
|The remainder are used to find out about the "worth" of the message 
|itself.  The final filtering decision is presumably based upon 
|information from all of the mechanisms.
|
|For example, the lack of strong traceability of a particular message 
|would normally cause the whitelist information to be treated as less 
|reliable, but the lack of a strong whitelisting would probably be 
|offset if it carried a (monetary or computational) payment token.  
|Also, if the credentials carried by the message were found to 
|have been 
|generated by a discredited authority, this would probably reflect 
|negatively on the message as a whole, whatever it carried.
|
|>> - BCPs for effective recipient filtering based on above facilities
|>> - BCPs for senders (of various categories) for efficient 
|use of above
|>> facilities
|>
|> BCPs are basically the policy documents to make sure that the
|> framework is used properly - kind of like manuals.
|
|Still an essential part of the system.
|
|>> With any luck, that should make things a heap clearer for everyone
|>> (myself included), unless of course Yakov is going to point out how 
|>> incorrect I am.  :-)
|>
|> Jonathan - your explanation has been extremely heplful. As a 
|matter of
|> fact, it seems that I will need additional help rewriting the 
|> framework document, specifically on how to rewrite it so "normal" 
|> humans can understand it :) If you wish to help with the writing and 
|> the editing, please contact me.
|
|Yes, I think I could do that.
|
|But in particular, I hope that more list members now realise that the 
|Consent Framework is in fact an important and useful document.  There 
|are a lot of proposals coming in which try to cover multiple 
|aspects of 
|the system at once (usually doing one well and others poorly), and I 
|suspect this is due to ignorance of the framework and it's intentions.
|
|It would be overall better if proposal authors were more aware of the 
|Consent Framework, and were thus able to design their mechanisms as 
|single-purpose plugins to it, instead of jack-of-all-trades 
|megaproposals.
|
|--------------------------------------------------------------
|from:     Jonathan "Chromatix" Morton
|mail:     [email-masked]
|website:  http://www.chromatix.uklinux.net/
|tagline:  The key to knowledge is not to rely on people to 
|teach you it.
|
|
|