Subject: IESG Statement on Spam Control on IETF Mailing Lists

From: IESG Secretary <[email-masked]>
Date: Mon, 14 Apr 2008 08:39:38 -0700 (PDT)

The following principles apply to spam control on IETF mailing lists:

* IETF mailing lists MUST provide spam control.
* Such spam control SHOULD track accepted practices used on the Internet.
* IETF mailing lists MUST provide a mechanism for legitimate technical
participants to bypass moderation, challenge-response, or other techniques
that would interfere with a prompt technical debate on the mailing list
without requiring such participants to receive list traffic.
* IETF mailing lists MUST provide a mechanism for legitimate technical
participants to determine if an attempt to post was dropped as apparent
spam.
* The Internet draft editor, RFC editor, IESG secretary, IETF chair and
IANA MUST be able to post to IETF mailing lists. The relevant identity
information for these roles will be added to any white-list mechanism used
by an IETF mailing list.
* There MUST be a mechanism to complain that a message was inappropriately
blocked.

The realization of these principles is expected to change over time.
List moderators, working group chairs and area directors are expected to
interpret these principles reasonably and within the context of IETF
policy and philosophy.

This supercedes a previous IESG statement on this topic:
http://www.ietf.org/IESG/STATEMENTS/mail-submit-policy.txt
That statement contains justification and implementation advice that may
be helpful to anyone applying these principles.

A separate IESG statement applies to moderation of IETF mailing lists:
http://www.ietf.org/IESG/STATEMENTS/moderated-lists.txt

_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Mon, 14 Apr 2008 10:02:39 -0700

I would suggest that the IESG also think about hosting all IETF lists in
house in the future.

The main reason for this is legal, a list that is maintained by the IETF
is much more satisfactory in a patent dispute than one run by a third
party. Last thing we want is to have patent trolls dragging a third
party list maintainer into a dispute while they try to argue that the
list somehow does not count.

And yes, I am aware that the 'law', might be on our side here. The
problem is that it can cost a ridiculous amount of money to have a court
decide the most obvious and basic question you might imagine.

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of IESG Secretary
> Sent: Monday, April 14, 2008 8:40 AM
> To: IETF Announcement list
> Cc: [email-masked]; [email-masked]
> Subject: IESG Statement on Spam Control on IETF Mailing Lists 
> 
> * IETF mailing lists MUST provide a mechanism for legitimate 
> technical participants to bypass moderation, 
> challenge-response, or other techniques that would interfere 
> with a prompt technical debate on the mailing list without 
> requiring such participants to receive list traffic.
> * IETF mailing lists MUST provide a mechanism for legitimate 
> technical participants to determine if an attempt to post was 
> dropped as apparent spam.
> * The Internet draft editor, RFC editor, IESG secretary, IETF 
> chair and IANA MUST be able to post to IETF mailing lists. 
> The relevant identity information for these roles will be 
> added to any white-list mechanism used by an IETF mailing list.
> * There MUST be a mechanism to complain that a message was 
> inappropriately blocked.
> 
> List moderators, working group chairs and area directors are 
> expected to interpret these principles reasonably and within 
> the context of IETF policy and philosophy.
> 
> That statement contains justification and implementation 
> advice that may be helpful to anyone applying these principles.
>

---

From: "Daniel Brown" <[email-masked]>
Date: Mon, 14 Apr 2008 13:19:29 -0400

>  list somehow does not count.

    The question I'd have in response to that point though, Phillip,
is how the cost vs. benefit of hosting all data in-house with
consideration to changing privacy and data retention laws weighs out.

    Apologies for the run-on sentence.

-- 
</Daniel P. Brown>
Ask me about:
Dedicated servers starting @ $59.99/mo., VPS starting @ $19.99/mo.,
and shared hosting starting @ $2.50/mo.
Unmanaged, managed, and fully-managed!
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Russ Housley <[email-masked]>
Date: Mon, 14 Apr 2008 13:25:09 -0400

Phill:

When IETF lists are housed somewhere other than ietf.org, they are 
supposed to include an archive recipient so that there is an archive 
available at ietf.org (perhaps in addition to the one kept at the 
place where the list is housed).

Russ

---

From: Eliot Lear <[email-masked]>
Date: Mon, 14 Apr 2008 20:13:23 +0200

Russ,

> place where the list is housed).

I'll agree with Phill's conclusion on this one.

I think there is probably convenience value to housing the mailing lists 
at the IETF.  It allows for a single whitelist, reduction in those 
annoying monthly messages that we eventually all filter into the 
bitbucket.   Also, it  probably is easier to effect and audit policy 
(such as we have any) in terms of message retention, uniform access, etc.

Regards,

Eliot

---

From: Michael Thomas <[email-masked]>
Date: Mon, 14 Apr 2008 11:16:17 -0700

> (such as we have any) in terms of message retention, uniform access, etc.
The other things is when we start DKIM signing (HINT HINT), it will
give a single identity for reputation, etc to latch onto.

       Mike

---

From: "Frank Ellermann" <[email-masked]>
Date: Mon, 14 Apr 2008 20:58:47 +0200

Russ Housley wrote:

> When IETF lists are housed somewhere other than ietf.org,
> they are supposed to include an archive recipient so that
> there is an archive available at ietf.org

Makes sense.  I have submitted some lists to "other lists",
how is this archive recipient magic arranged ? 

 Frank

---

From: Henrik Levkowetz <[email-masked]>
Date: Mon, 14 Apr 2008 22:25:21 +0200

Hi,

> without requiring such participants to receive list traffic.

Umm -- I think I understand what this *intends* to say, but I'm not sure.

What I'm reading it as actually saying, though, is that a poster who
thinks he is a legitimate technical participant is to be provided means of
*bypassing* moderation.

A means of bypassing challenge-response could be to send a mail to one
of the list admins to forward to the list, but since moderation is (at
least normally) provided by the list admins, and essentially any human
who receives a message and is asked to forward it to the list will have
to judge whether the message is relevant and appropriate, which constitutes
moderation as I understand it, the statement above seems to imply that
there has to be some way, untouched by a human making any kind of evaluation,
to force a message to be posted to a list???

It would be rather helpful for an explanation or rationale to be provided
for a statement such as the above, which to me reads as a very categorical
statement that no kind of challenge-response, moderation, or other
reasonable guard against spam can be put in place without extraordinary
efforts at providing means to *force* a circumvention of the same.

I'm pretty sure that the third bullet above isn't intended to almost
completely nullify the first bullet, but I'm actually not sure how to
set up anything but painstaking manual inspection of every spam in order
to adhere to the third bullet as written.  None of the mechanisms currently
available, including TMDA, spam-assassin, and blocking of posts from
non-subscribers followed by manual inspection seems to fulfil this as
I read it, which leaves me at a loss.

> participants to determine if an attempt to post was dropped as apparent

Again, an umm...  I'm not sure I'm aware of an available technical solution
which out-of-the-box will ensure this is followed, without at the same time
resulting in a deluge of back-scatter.  If there was a SHOULD here, I could
imagine working over a bit of time at setting up Mailman to drop-and-archive,
but currently the solution which comes to mind is to reject, which (I believe)
potentially will result in backscatter and more work and/or junk for the list
admin.

Overall, I'm slightly surprised at how categorical several of the statements
above are, without providing rationale and background information which would
have made it possible to fully understand them.  It seems as if they are
presented as decrees from on-high which have to be followed even if they
aren't understood to be sensible or implementable...

> http://www.ietf.org/IESG/STATEMENTS/moderated-lists.txt

	Henrik

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 15 Apr 2008 09:00:07 +1200

+1 to Henrik's comments. I don't think the two MUSTs
that he comments on are algorithmically possible.

    Brian

---

From: Ned Freed <[email-masked]>
Date: Mon, 14 Apr 2008 14:11:52 -0700 (PDT)

> that he comments on are algorithmically possible.

These two MUSTs (the ability to whitelist specific posters without them having
to receive list mail and spam rejection) are both completely trivial to
implement with our software. The latter is normally done (and definitely should
be done) at the SMTP level, minimizing blowback.

I spend essentially no time these days with other messaging software but I'm
having a great deal of trouble believing this would be in any way difficult to
implement with something else. This is all stuff that lots of lists have done
for years - it doesn't even come close to rocket science.

				Ned

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 15 Apr 2008 09:27:12 +1200

> for years - it doesn't even come close to rocket science.

I think we're reading the words differently. That suggests to me
that they are ambiguous.

   Brian

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 15 Apr 2008 00:26:53 +0200

> be done) at the SMTP level, minimizing blowback.

What you describe above is feasible and reasonable.  What the posted text seems
to require isn't, in my opinion.

> for years - it doesn't even come close to rocket science.

Humm.  You're able to provide guaranteed bypass of moderation without human
evaluation, and without inspecting spam?  That's what my post was about, and
if you can do it, I'm impressed.

	Henrik

---

From: Dave Crocker <[email-masked]>
Date: Mon, 14 Apr 2008 15:32:16 -0700

Dear IESG,

> * Such spam control SHOULD track accepted practices used on the Internet.

These two bullets are well-intentioned, but have no clear meaning.  Simply put, 
there is no way of knowing whether conformance has been achieved.

For example, there are many different sets of "accepted practice" for anti-abuse 
email mechanisms, and what is particularly vexing is that what is deemed 
desirable by one constituency often is deemed as deplorable by another.

> without requiring such participants to receive list traffic.

Here, again, is something that is not a technical specification and had no clear 
criterion to permit telling when it is satisfied.

In fact, I'm not sure this bullet even has an appropriate goal.  (On reflection, 
I'm not entirely sure what the precise goal is.)

If the intent is to say that mailing lists shall not operate in a moderated 
mode, whereby all postings are subject to prior approval by the list 
administrator, then that's probably what you need to say.  But the language, 
here, seems to say that if such controls are in place, any participant can 
bypass them.  In which case, why have the control?

> participants to determine if an attempt to post was dropped as apparent

This is actually contrary to the way most list software seems to work. When 
dropping is done, it is done silently, in order to eliminate that considerable 
overhead that comes with large-volume spamming.

If the IESG has something specific in mind, then it should document how to 
achieve it for a number of the major mailing list packages.

> information for these roles will be added to any white-list mechanism used

Oops.  This is quite a good idea and I am quite sure that only one or two of the 
ones that need whitelisting are entered in the lists I administer.

But that really leads to the question of where the list of addresses to enter is 
maintained?  Given a standard list, then yes, it makes sense to have list admins 
pre-load them.

> be helpful to anyone applying these principles.

Actually, it really is essential that you add that advice, because I don't see 
it in the current draft.

> http://www.ietf.org/IESG/STATEMENTS/moderated-lists.txt

So here is the process question:

    The IESG document has shoulds and musts.  That means the intent is to be 
normative.

    Is the IESG in charge of making rules for the conduct of IETF working group 
mailing lists?

    I thought such rules were the result of an IETF consensus process.  When did 
that change?

    Please clarify.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
IETF mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/ietf

---

From: Ned Freed <[email-masked]>
Date: Mon, 14 Apr 2008 15:35:38 -0700 (PDT)

> to require isn't, in my opinion.

What I described is how I read the text. And while I sort of white the term
"whitelist" had been used, I'm having trouble figuring out how to read what's
there any other way.

> if you can do it, I'm impressed.

I guess I should be flattered, but really, I fail to see why. Guaranteed bypass
of moderation is simply an allowed-poster whitelist. And while our
implementation does allow for it, there is no requirement, expressed or
implied, that this be in any way coupled to spam filtering. (Indeed, if the
Sieve list extension is approved there will be a way to implement all of this
using nothing but standardized mechanisms.)

As for notification of spam filtering, all this requires is that a notification
be returned when a message is rejected as spam. The only issue here is that
this needs to be done at the SMTP level to prevent blowback, but again, I hope
this isn't a problem to implement with most messaging software.

				Ned

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 15 Apr 2008 01:32:07 +0200

> of moderation is simply an allowed-poster whitelist. 

So it seems to me that you've failed to see the problem.

Anybody who considers themselves a valid poster is supposed to be able to
bypass moderation, challenge-response and spam-filtering.  This would also
include a spammer who considers himself a valid poster.  At the same time,
the IETF lists MUST provide spam control.  I see this as a contradiction in
the announced text.

In other words, if one is not already on a whitelist, how does one get on
to it, automatically, without moderation and challenge-response, while
spam-filtering is still provided?

	Henrik

---

From: Ned Freed <[email-masked]>
Date: Mon, 14 Apr 2008 20:12:56 -0700 (PDT)

> bypass moderation, challenge-response and spam-filtering.

I see nothing in the requirements that says this supposed to be possible as a
unilateral action on the part of the poster. That's clearly preposterous - it
should go without saying that whitelisting arrangements are just that:
Arrangements. The requirements leave open how such arrangements are made; IMO
that's entirely appropriate.

> This would also

Only if you engage in a VERY creative reading of what's there.

> spam-filtering is still provided?

And there's that word "automatically". There is nothing in the text that says
such arrangements have to be automatic.

				Ned

---

From: "Randy Presuhn" <[email-masked]>
Date: Mon, 14 Apr 2008 20:49:57 -0600

Hi -

> From: "Ned Freed" <[email-masked]>
> To: "Henrik Levkowetz" <[email-masked]>
> Cc: "Ned Freed" <[email-masked]>; <[email-masked]>
> Sent: Monday, April 14, 2008 9:12 PM
> Subject: Re: IESG Statement on Spam Control on IETF Mailing Lists
...
> such arrangements have to be automatic.
...

> without requiring such participants to receive list traffic.

The stated requirement is that it must not "interfere with a prompt
technical debate". Clearly, that rules out anything requiring human
intervention.  What do you have in mind that would allow the
spammer^H^H^H^H^H^H^H participant to post without subscribing
and without interacting with a chair or list administrator?

Randy

---

From: Ned Freed <[email-masked]>
Date: Mon, 14 Apr 2008 20:22:23 -0700 (PDT)

> there is no way of knowing whether conformance has been achieved.

IMO this criticism is legitimate (unlike the previous criticisms I responded
to, which IMO are not). That said, I tried to come up with a way say what's
needed here and I'm afraid I could come up with nothing better for the first
point. LIke it or not, spam is a major problem and controls are necessary. But
the minute you dive into what control means you're basically stuck into a
definitional funk.

> desirable by one constituency often is deemed as deplorable by another.

Exactly. I think the right thing to do is simply remove this item entirely. I'm
not entirely happy about this resolution but I can think of nothing better.

> I'm not entirely sure what the precise goal is.)

The goal is simple: Whitelisting. There are many reasons such a facility is
needed, but one I see all the time is where someone is subscribed using a
subaddress of some sort but doesn't post using that address.

We call this a "nomail" subscription in our implementation, after the LISTSERV
command that was used to set this up.

> bypass them.  In which case, why have the control?

Wow, I just don't get it. Nothing in the text says or even implies that
any participant can bypass the controls on a whim. Why is everyone reading
into the text something that just isn't there?

I guess it is necessary to modify the text to make it clear that such
arrangements are customarily made through some sort of additional configuration
involving some kind of validity check. I have to say I'm nothing short of
astounded that this is necessary.

> overhead that comes with large-volume spamming.

The approach I prefer (and which I hope most list software is able to support)
is to bounce such messages at the SMTP level. This eliminates sufficient
blowback to be an effective approach. (And yes, I'm well aware that it doesn't
eliminate all of it. Let's please not dive down this rathole yet again.)

However, if for some reason this isn't possible, or leads to too much blowback
via relays, or whatever, there are alternatives. The obvious one is to record
the message-ids of messages rejected as spam for some reasonable period of time
so the handling of a given message can be determined.

Nothing in the requirement says that the disposition of the message has to be
communicated through an email notification.

> achieve it for a number of the major mailing list packages.

I would not object to documenting some of the possible approaches, but is this
right place for it? As for requiringing a specific approach, I strongly object
to that.

> maintained?  Given a standard list, then yes, it makes sense to have list admins

Um, Dave, you do realize the implications of providing a list on a policy page
of addresses whitelisted for all IETF lists? I think it is very sensible indeed
to omit this.

> So here is the process question:

You know, I think I'm going to pass on the process question here. I'm having
more than enough fun with the technical side.

				Ned

---

From: John Levine <[email-masked]>
Date: 15 Apr 2008 05:08:12 -0000

>> Here, again, is something that is not a technical specification and
>had no clear criterion ...
>
>The goal is simple: Whitelisting.

I had the same problem with it as everyone else.  If you say it's
supposed to mean that list managers can whitelist people, that's fine,
but in that case it desperately needs to be rewritten so it says what
it means, e.g.:

* IETF lists MUST provide a mechanism that allows list managers to
whitelist mail from legitimate technical participants to bypass
moderation and spam filters, and to allow one-way participation for
people who are too important to spend time reading or dealing with
responses to their mail.

R's,
John

---

From: Ned Freed <[email-masked]>
Date: Mon, 14 Apr 2008 22:20:57 -0700 (PDT)

> intervention.

It does nothing of the sort. This is talking about the effect the whitelist
must have, not the procedure for setting it up.

And before you argue that it can be read as applying to the setup procedure,
then you need to explain what it means to "bypass moderation" for a
whitelisting setup operation. That makes no sense whatsoever.

> What do you have in mind that would allow the

I have no such thing in mind because there's no such requirement to meet.

However, this has gone on long enough. I give up - what seems completely and
absolutely clear to me to the point of patent obviousness is apparently totally
opaque to lots of other people. So let's change the language to make it clear
to everyone and be done with it.

				Ned

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 15 Apr 2008 08:20:30 +0200

> Only if you engage in a VERY creative reading of what's there.

As has been painfully clear for some email round-trips, we obviously don't agree.

	Henrik

---

From: Tim Chown <[email-masked]>
Date: Tue, 15 Apr 2008 10:17:21 +0100

Having a single system for all WG lists has the appeal that whatever
process(es) handle the lists, it will be the same for all lists, so
you don't have to figure out how N different lists are run.

As a shameless plug, we have a free open source solution developed here
which is widely used against spam/viruses and that could do the job, see 
http://www.mailscanner.info/

-- 
Tim

---

From: "Tom.Petch" <[email-masked]>
Date: Tue, 15 Apr 2008 15:08:31 +0200

----- Original Message -----
From: "IESG Secretary" <[email-masked]>
To: "IETF Announcement list" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Monday, April 14, 2008 5:39 PM
Subject: IESG Statement on Spam Control on IETF Mailing Lists

> * IETF mailing lists MUST provide spam control.

And what is spam?  I have seen this discussed on several occasions on lists
whose prime concern is spam and there has been no rough consensus.  For example,
Phillip Hallam-Baker, on asrg, summed up one discussion well with
"OK so defining the term spam is off limits to the group because it ends up in
definitional flame wars."

For me, it is dead obvious that the 397kbyte PowerPoint file I received on an
IETF list last week is spam, and I know some IETF moderators who would not have
allowed it on to the list; but equally, I am sure that the sender would protest
his innocence.

If we do not agree what spam is, then the whole of this statement seems to me
pointless.

Tom Petch

> IETF-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-announce

---

From: "Tom.Petch" <[email-masked]>
Date: Tue, 15 Apr 2008 15:09:11 +0200

----- Original Message -----
From: "Eliot Lear" <[email-masked]>
To: "Russ Housley" <[email-masked]>
Cc: "IETF Discussion" <[email-masked]>
Sent: Monday, April 14, 2008 8:13 PM
Subject: Re: IESG Statement on Spam Control on IETF Mailing Lists

> bitbucket.

Err, no!  I have been silently discarded from IETF lists on a number of
occasions - v6ops is the worst offender - and that monthly message is an
invaluable check as to whether I have been discarded again or whether the list
has been dormant.

Tom Petch

> Also, it  probably is easier to effect and audit policy

---

From: James Galvin <[email-masked]>
Date: Tue, 15 Apr 2008 10:32:58 -0400

-- On Monday, April 14, 2008 8:58 PM +0200 Frank Ellermann 
<[email-masked]> wrote regarding Re: IESG Statement on 
Spam Control on IETF Mailing Lists --

> how is this archive recipient magic arranged ?

I can tell you what is supposed to happen.  The short answer is 
that RFC2418 tells you one of the two email addresses to subscribe 
to your "other list" to get an archive on the IETF web site. 
However, I think that guidance is at best incomplete.  The rest of 
this message is the long answer and why I think that guidance is 
incomplete.

RFC2418, IETF Working Group Guidelines and Procedures, says this
in Section 2.2 Charter:

As a service to the community, the IETF Secretariat operates a
mailing list archive for working group mailing lists. In order
to take advantage of this service, working group mailing lists
MUST include the address "[email-masked]"
(where "wg_acronym" is the working group acronym) in the
mailing list in order that a copy of all mailing list messages
be recorded in the Secretariat's archive.  Those archives are
located at ftp://ftp.ietf.org/ietf-mail-archive.  For
robustness, WGs SHOULD maintain an additional archive separate
from that maintained by the Secretariat.

In turns out that this guidance is both incomplete in practice and, 
unfortunately, wrong in one detail.

The incorrect detail is that the IETF no longer uses the domain 
lists.ietf.org.  In fact, it never used it correctly.  During the 
transition we discovered 7 domains used for mailing lists, the 
predominant one being the domain "ietf.org".  As part of the 
cut-over it was decided to use the domain "ietf.org" exclusively 
for all IETF lists.  Everything was setup that way with backwards 
compatibility in place for all existing lists that used some other 
domain.  Today, all lists are created in "ietf.org", unless they 
are IAB, IESG, or IRTF lists.

In practice, there are a few incomplete details.

First, there are actually two aliases that need to be subscribed:

    [email-masked]
    [email-masked]

Some might ask how this separation came to be?  I don't know and 
never asked.  It is continued today mostly for convenience; it's 
quite tedious to "undo" the current setup and there are more 
important things to be done.

Second, except for the guideline in 2418, there's no process or 
documentation to go with this whole model on the IT side.  Here's a 
sampling of some questions.

1. There's no maintenance of this practice.  Last year the IETF did 
an audit, for the first time in forever, and brought everything 
up-to-date.  This means that at that time all the "other lists" had 
an archive on the IETF web site.  However, we're out of date again.

2. These archives have a SPAM problem.  Well, they had a much more 
serious SPAM problem.  Early in the post cut-over process AMS (Glen 
Barney) added some SPAM protection to these archives.  However, all 
the best work has been put into getting Mailman firmly entrenched 
and protected.  My opinion is that the IETF should just create a 
mailing list for every WG and then these "other lists" should just 
subscribe the IETF list to their list.  This way there's no "extra" 
work on the IT side to protect things, particularly since Mailman 
provides some useful built-in features.  In addition, we don't need 
these "magic" aliases any more.

3. Part of the maintenance problem is that for obscure reasons the 
IETF subscriber on these "other lists" will drop off the list 
occasionally.  I'd be interested in hearing any good ideas for how 
to deal with this issue.

4. The Secretariat has to take action to make these email addresses 
work.  I realize this is probably obvious to everyone here, but in 
the interests of completeness it should be documented that if 
you're going to setup an "other list" you need to ask to have the 
archives created.  Or, perhaps, a back-office process should exist 
(does not currently) that says these archive should be 
automatically created whenever a working group is created.

5. These questions and issues are frequently asked or framed in the 
context of WGs.  However, on the IT side, they apply more broadly, 
i.e., they apply to BOFs, directorate mailing lists, and other 
private lists.  This not normally visible to the 
community-at-large, and perhaps should be at least accessible in 
some way even if it's not announced, but my comment here is that 
none of it is documented in any way.

Jim

---

From: "TS Glassey" <[email-masked]>
Date: Tue, 15 Apr 2008 07:38:17 -0700

----- Original Message ----- 
From: "Tom.Petch" <[email-masked]>
To: "Eliot Lear" <[email-masked]>
Cc: "IETF Discussion" <[email-masked]>
Sent: Tuesday, April 15, 2008 6:09 AM
Subject: Re: IESG Statement on Spam Control on IETF Mailing Lists

> invaluable check as to whether I have been discarded again or whether the 

That is addressable though tom. The real issue is the legal controls which 
the IETF is obligated by its creation under the ISOC Charter to put in 
place.  These include managing the electronic archive and legal culpability 
for changes (authorized or not) to that content as well as dealing with 
instances where individuals were 'quietly edited out' of various WG's

---

From: James Galvin <[email-masked]>
Date: Tue, 15 Apr 2008 10:59:20 -0400

-- On Monday, April 14, 2008 10:25 PM +0200 Henrik Levkowetz 
<[email-masked]> wrote regarding Re: IESG Statement on Spam 
Control on IETF Mailing Lists --

> Again, an umm...  I'm not sure I'm aware of an available
> technical solution which out-of-the-box will ensure this is
> followed, without at the same time resulting in a deluge of
> back-scatter.  If there was a SHOULD here, I could imagine
> working over a bit of time at setting up Mailman to
> drop-and-archive, but currently the solution which comes to mind
> is to reject, which (I believe) potentially will result in
> backscatter and more work and/or junk for the list admin.

There is another method, which is currently used on the IETF 
mailing lists with a public archive.

First, the current statement does point you at the older statement:

<http://www.ietf.org/IESG/STATEMENTS/mail-submit-policy.txt>

Which says this:

In other cases, it MUST be possible for the sender of a legitimate
message, whether a mailing list subscriber or not, to determine if 
it is has been dropped as apparent spam.  This can be done in 
several ways; all of these have their advantages and disadvantages.

 b.  Provide an up-to-date archive of accepted postings.
     Unfortunately, while this can show dropped messages, it doesn't
     help if the email is merely delayed, nor does it say why a
     message was dropped.  This MAY be used.

This is the method currently employed on IETF lists with a public 
archive.  Thus, the onus is on the originator of a message to make 
sure it was distributed.

I'll also note for completeness that the IETF does not reject 
messages after they have been accepted for delivery by the SMTP 
server.  Messages may be rejected by the SMTP server, in which case 
the originator should get a notice.  After that point the message 
is either delivered or, in the case of lists with public archives, 
it may be dropped.

Jim

---

From: James Galvin <[email-masked]>
Date: Tue, 15 Apr 2008 11:09:49 -0400

I'm not sure I agree.  I do agree with Henrik's comments to the 
extent that I think we need to be clear.  Obviously there's some 
ambiguity and we should clear that up.

My interpretation of the two statements is that they are 
complementary, not conflicting.  I would say that the third bullet 
is a response to the first bullet "running amok".  In other words,
if you're going to have SPAM control, you have to deal with the 
problem of false negatives.  It seems to me that all the third 
bullet is trying to say is that when individuals find themselves 
subject to a "denial-of-service" because of a false negative report 
from the SPAM control, there has to be a way for them to get 
through.

I don't know if that's what was intended.  If it was then that 
needs to be made clear.  This could be helped by explicitly 
suggesting a way around, which is to forward the message to a 
Chair, list moderator (easily visible on the Mailman listinfo page 
for the list), Area Director, or perhaps even to the Secretariat 
for forwarding to one of these people if the person is having 
trouble even getting to them.

If that's not what was intended then I agree completely with Henrik.

Jim

-- On Tuesday, April 15, 2008 9:00 AM +1200 Brian E Carpenter 
<[email-masked]> wrote regarding Re: IESG Statement on 
Spam Control on IETF Mailing Lists --

> >> * Such spam control SHOULD track accepted practices used on
> >> the Internet. * IETF mailing lists MUST provide a mechanism
> >> for legitimate technical participants to bypass moderation,
> > What I'm reading it as actually saying, though, is that a
> > poster who thinks he is a legitimate technical participant is
> > to be provided means of *bypassing* moderation.
> >
> > A means of bypassing challenge-response could be to send a mail
> > to one of the list admins to forward to the list, but since
> > moderation is (at least normally) provided by the list admins,
> > and essentially any human who receives a message and is asked
> > to forward it to the list will have to judge whether the
> > message is relevant and appropriate, which constitutes
> > moderation as I understand it, the statement above seems to
> > imply that there has to be some way, untouched by a human
> > making any kind of evaluation, to force a message to be posted
> > to a list???
> >
> > It would be rather helpful for an explanation or rationale to
> > be provided for a statement such as the above, which to me
> > reads as a very categorical statement that no kind of
> > challenge-response, moderation, or other reasonable guard
> > against spam can be put in place without extraordinary efforts
> > at providing means to *force* a circumvention of the same.
> >
> > I'm pretty sure that the third bullet above isn't intended to
> > almost completely nullify the first bullet, but I'm actually
> > not sure how to set up anything but painstaking manual
> > inspection of every spam in order to adhere to the third bullet
> > as written.  None of the mechanisms currently available,
> > including TMDA, spam-assassin, and blocking of posts from
> > non-subscribers followed by manual inspection seems to fulfil
> > this as I read it, which leaves me at a loss.
> >
> > drop-and-archive, but currently the solution which comes to
> > mind is to reject, which (I believe) potentially will result in
> > Overall, I'm slightly surprised at how categorical several of
> > the statements above are, without providing rationale and
> > background information which would have made it possible to
> > fully understand them.  It seems as if they are presented as
> > decrees from on-high which have to be followed even if they
> >> chair and IANA MUST be able to post to IETF mailing lists. The
> >> relevant identity information for these roles will be added to
> >> any white-list mechanism used by an IETF mailing list.
> >> The realization of these principles is expected to change over
> >> time. List moderators, working group chairs and area directors
> >> are expected to interpret these principles reasonably and
> >> within the context of IETF policy and philosophy.
> >>

---

From: James Galvin <[email-masked]>
Date: Tue, 15 Apr 2008 11:27:30 -0400

-- On Monday, April 14, 2008 2:11 PM -0700 Ned Freed 
<[email-masked]> wrote regarding Re: IESG Statement on Spam 
Control on IETF Mailing Lists --

> These two MUSTs (the ability to whitelist specific posters
> without them having to receive list mail and spam rejection) are
> both completely trivial to implement with our software. The
> latter is normally done (and definitely should be done) at the
> SMTP level, minimizing blowback.

To be fair, and I know Ned that you know this, it depends on where 
and how you implement specific controls.  Some software makes this 
easier than other software.  In general, the more integrated the 
components the finer granularity one gets in what you can do.

Specifically, the whitelisting has to occur either before or within 
the SPAM filtering.  If a source is whitelisted it has to bypass 
all other checks.

The IETF setup uses SpamAssassin for tagging purposes.  This is 
done outside of the SMTP service and outside of Mailman, which 
supports the mailing lists.  The whitelisting is done with TMDA, 
which is also outside of SpamAssassin and outside of Mailman.

Getting all three of these things to work together is not trivial. 
I don't mean to suggest it's rocket science, but you have to sit 
down and think about how each of them provide the various services 
they provide and get them to cooperate.  Changes in any one require 
a re-evaluation of the entire setup, just to make sure there are no 
unintended consequences.

The fact that TMDA does whitelisting means that Mailman does not 
have to do it.  This reduces the SPAM load on Mailman but it does 
not change the fact that you have to be a subscriber to get a 
message through.  If you're not a subscriber you're still going to 
get "moderated".

For Mailman to do the whitelisting it means that every mailing list 
would have to have the same database that TMDA has, which has 
40,000 entries in it.  Yes, that's right, there are 40,000 unique 
email addresses across all IETF mailing lists.  This is how Mailman 
works.

My point here is that there are choices to be made, and those 
choices have implications.  Obviously the IETF could make different 
choices, but I do think it's important to understand the advantages 
and disadvantages of different choices.

Jim

---

From: Rich Kulawiec <[email-masked]>
Date: Mon, 14 Apr 2008 14:38:56 -0400

> annoying monthly messages that we eventually all filter into the 

I'll concur with the general sentiment here, although I don't think
there's any need for DKIM or any of the other related flavor-of-the-month
technologies (SenderID, SPF, etc).

A suggestion -- to Eliot's point about monthly reminders -- would be
to consider consolidating those into a single reminder that covers all
IETF mailing lists.  This would cut down IETF-outbound mail volume as well
as per-recipient inbound mail volume, while (I think) still serving the
same function.

---Rsk

---

From: Henrik Levkowetz <[email-masked]>
Date: Tue, 15 Apr 2008 19:49:00 +0200

>      message was dropped.  This MAY be used.

If this is acceptable, I'm happy.  Unfortunately, I wouldn't have
thought this solution would have been acceptable after reading the
statement of the original posting, and as long as the IESG doesn't
indicate that it is acceptable, I'll continue to be uncertain.

So as far as I can tell, the essence of my original response remains:

The original posting would have benefited greatly by including a
bit of rationale and examples; and my suggestion would be to do
any needed revision to the older statement:
  http://www.ietf.org/IESG/STATEMENTS/mail-submit-policy.txt
and issue a new version of that, which seems to give the background,
rationale and examples missing from the latest statement.  If the
latest statement is going to be allowed to stand, at least I am
going to feel that I'm guessing far more than I'm comfortable with
regarding exactly where the line between acceptable and non-acceptable
technical solutions to spam filtering goes.

If the IESG finds itself unable to find the time needed to revise the
older document I'm also offering to provide revised text based on
that document, in the interest of having policy I feel can be read,
understood and followed without too much ambiguity.

	Henrik

---

From: "Frank Ellermann" <[email-masked]>
Date: Tue, 15 Apr 2008 23:36:12 +0200

James Galvin wrote:

Hi, thanks for the explanation, I add some notes of what
I think this means, please correct me if I got it wrong.

  [2418]
>| As a service to the community, the IETF Secretariat
>| operates a mailing list archive for working group
>| mailing lists.

Most lists I submitted to the "other lists" are in fact 
former WG lists.  I guess there is nothing to do for the
submitter / AD / list owner for former IETF WG lists, the
"official" archives are likely still subscribed. 

> Today, all lists are created in "ietf.org", unless they 

Okay.  Existing "other lists" are not necessarily hosted
by ietf.org, they can be anywhere (Harald, IMC, Pobox,
W3C, Google, Yahoo!, etc.)

> Last year the IETF did an audit, for the first time in
> forever, and brought everything up-to-date.  This means
> that at that time all the "other lists" had an archive
> on the IETF web site.

Great, that should take care of any list I ever submitted
with one or two fresher exceptions.

> However, we're out of date again.

Maybe the two archive subscriptions could be added to the 
verification procedure.  As far as I can tell it the Web
form creates a request to the chosen AD, the AD can then
accept, reject, or ignore it.  The accept state could be
split to arrange the archive subscriptions.  Lots of fun
there, but there aren't many "other list", and most have
a mailman Web interface.

> These archives have a SPAM problem.

Willing to explain the secrets of RFC 4408, but maybe not
again here... :-)

> My opinion is that the IETF should just create a mailing
> list for every WG and then these "other lists" should
> just subscribe the IETF list to their list.

The opposition can then still pretend to send mail from the 
old list to the new list, and vice versa.  The position of
the gateway on the IETF side (directly above the archives
vs. before the new list) won't necessarily change the spam
problem.  If something breaks you have a split, articles
appearing only in the old or new list.  A gateway directly
above the archives has the same problem, but archives at
least don't try to post... :-)  Or if they apparently post
it must be spam.

> Mailman provides some useful built-in features.

Unfortunately it doesn't let me say "for each subscribed
nobody@xyzzy add a new address jhjhjggjhmgc@gmail", it also
doesn't let me say "now please consider hmhmdnbdngnf@gmail
to have write access on all existing lists".

> Part of the maintenance problem is that for obscure
> reasons the ETF subscriber on these "other lists" will
> drop off the list occasionally.  I'd be interested in
> hearing any good ideas for how to deal with this issue.

Pass.  Disable password reminder, maybe ?

> The Secretariat has to take action to make these email
> addresses work.  I realize this is probably obvious to
> everyone here, but in the interests of completeness it
> should be documented that if you're going to setup an
> "other list" you need to ask to have the archives 
> created.

Certainly not obvious for me, I never did this.  I just
filled out the submission form, staying away from using
"<" + ">", because that breaks the "other lists" script.
ADs are annoyed when they have to confirm updates only
because "<" + ">" breaks the "other lists" Web page ;-)

 Frank

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Tue, 15 Apr 2008 14:36:43 -0700

I don't think it is helpful for the IETF to describe its work product as 'flavor-of-the-month'.
 
DKIM is an IETF Proposed Standard.
 
Using DKIM is thus a dog-food issue. SPF/Sender-ID on the other hand are arguably not at the same status but there is a general consensus amongst the spam community that they help the spam-fighters.

That said, I also +1 on the monthly reminders issue. I would like the monthly reminder to include a NOTE WELL section.
 
It would also enable services such as allowing mail to be redirected en-mass for a given recipient when they change jobs or email provider. Or simply decide that they would prefer to direct all their IETF mail to another account so they don't run up an incredible bill on the pager.

________________________________

From: [email-masked] on behalf of Rich Kulawiec
Sent: Mon 14/04/2008 2:38 PM
To: IETF Discussion
Subject: Re: IESG Statement on Spam Control on IETF Mailing Lists

---Rsk

---

From: Cullen Jennings <[email-masked]>
Date: Tue, 15 Apr 2008 15:01:30 -0700

Hi Henrik,

Seems this email about email still needs some more discussion - I have  
not been involved much with this much but I suspect that Chris Newman  
would probably be the best person on the IESG to work with on both  
clarifications and changes.

Cullen

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 16 Apr 2008 10:11:53 +0200

Hi Cullen,

> clarifications and changes.

Ok, I'll contact Chris off-list.

	Henrik

---

From: James Galvin <[email-masked]>
Date: Wed, 16 Apr 2008 10:35:57 -0400

-- On Tuesday, April 15, 2008 11:36 PM +0200 Frank Ellermann 
<[email-masked]> wrote regarding Re: IESG Statement on 
Spam Control on IETF Mailing Lists --

> "official" archives are likely still subscribed.

I would suggest that there needs to be a back-office process in 
place to ensure that when a working group (or BOF or any other 
list) shuts down, the IETF should make sure it has a copy of the 
complete archive.  This does not currently exist.

> there, but there aren't many "other list", and most have

Which web form are you referring to?  I think what you're talking 
about is the one for the web page that lists "other lists".  If 
that's true, I agree, there should be a back-office process that 
checks the archive subscriptions before (perhaps in parallel) with 
the list getting listed on the "other lists" web page.  Of course 
this does not address the maintenance issue of keeping the 
subscription live....

If you're talking about the mailing list request form, that's for 
getting a list hosted at ietf.org so the archives are not an issue 
in that case.

Or did you mean some other form?

> problem.

But what it facilitates is using the same mechanisms in the same 
way to control the SPAM problem.  It is an operational 
simplification that obviates a bifurcation.  Instead of a message 
coming in, getting tagged by SpamAssassin and then having to be 
directed either to an archive or Mailman, it always goes to 
Mailman.  The SPAM filtering is already part of Mailman.  If it 
goes to the archive you have to add a module or function to do the 
SPAM filtering that Mailman does.

> to have write access on all existing lists".

Well, I'm not exactly a Mailman expert but I believe these features 
are relatively straightforward to do if you are the Mailman site 
administrator and have shell access to the server.  Therefore, an 
ambitious person could setup a web interface to support them.  :-)

Jim

---

From: "Frank Ellermann" <[email-masked]>
Date: Wed, 16 Apr 2008 19:44:11 +0200

> Or did you mean some other form?

Yes, not the "request list" form, I've never used it.

I meant IETF -> lists -> note well -> other lists ->
non-WG -> non-WG posting page (five clicks deep ;-)
<https://datatracker.ietf.org/list/nonwg/update/> ->
step 1 "add new entry" -> proceed -> step 2 **THIS**.

 [gateway old <-> new list instead of list <-> archive]
> what it facilitates is using the same mechanisms in
> the same way to control the SPAM problem.  It is an
> operational simplification that obviates a bifurcation.

If it obviates bifurcation I didn't get what you were
talking about.  Maybe you want to *replace* the old 
list elsewhere by a new list at ietf.org, preserving
only the subscriptions.  

IME moving lists or groups, just renaming them, is a 
guaranteed way to lose 80% of all readers forever, and
annoying a significant part of the rest.  The "other
lists" have owners, why should they hand over their
list to the IETF ?  Change as much as lists.ietf.org
to ietf.org and it will cause havoc somewhere, and
months to figure out what's broken. 

As an example, from my POV two review lists are dead,
and I will dump review requests directly to iesg@ or
whatever it takes to get them on public record (it's
a formal thing, "tried to send" is not good enough ;-)

 Frank