Subject: Re: [feedback-report] ARF
From: Steve Atkins <[email-masked]>
To: ARF mailing list <[email-masked]>
Date: Sat, 12 Jul 2008 18:54:16 +0100
List-Id: Public forum for discussion on the feedback-report draft <abuse-feedback-report.mipassoc.org>

On Jul 12, 2008, at 4:45 PM, Jenny Gavin-Wear wrote:

> Hi all,
>
> I have been researching ARF in relation to the AOL FBL.
>
> My situation is that I am a web site developer (Coldfusion) offering  
> hosting and email services to my customers.  Some of those customers  
> are using mail lists.
>
> My problem is that AOL users incorrectly mark mail list emails as  
> spam instead of unsubscribing, despite all of the options I have  
> made available - such as one click unsubscribe.
>
> So, I need to be able to find the email address of the complainant  
> from the AOL ARF report.

You need to embed the information needed to unsubscribe the recipient  
in the email you send out. The usual way to do that is to embed some  
sort of magic cookie which will allow you to identify both the email  
address the mail was sent to and the list the subscription was  
associated with (ideally you'd also be able to identify the specific  
mailing used). This is supported by every decent mailing list manager  
(not least because you need the same data to handle bounces reliably).

>
> I am running two mail servers, (Mdaemon and Smartermail) on Windows  
> 2003 Server Standard.
>
> On researching a solution for the Windows platform I find nothing.   
> Worse still, these is very little information on the Net about the  
> ARF format and on how I might be able to parse it.

If you embed a magic cookie in the mail you send out then it's trivial  
to parse - you don't need to do anything more than pull the message/ 
rfc822 section out of the mail and look for the cookie. 999 times out  
of 1000 you don't even need to parse the MIME at all, you can just  
treat the ARF report as a block of text and look for the magic cookie  
in that block of text.

If you don't include a magic cookie of some sort, and a way of mapping  
from that cookie back to the original email address, you're making a  
lot of unneeded work for yourself.

Mail::ARF - found here http://wordtothewise.com/resources/arfdeveloper.html 
  - is a simple perl module that will parse and validate an ARF report  
and make the information (including the included original message)  
available to a perl script. There's another module in the Email::  
space too that'll do similar things. Perl runs just fine on Windows,  
and that'd be where I'd start if I needed to do a bespoke ARF handler  
on Windows.

>
> I would appreciate any information that can lead me to finding a  
> solution.  It would be my intention to make any script I might be  
> able to create publicly available.
>
> In your development of ARF as a standard, I would like you to  
> consider this, and correct me if I am wrong: AOL state that they are  
> not prepared to provide the email addresses of complainants and will  
> only provide the message ID of the email that generated the  
> complaint - from which I can then find the email address.  Wouldn't  
> it be easier to provide the email address in the ARF report in the  
> first place?

This is unrelated to the ARF standard. ARF just describes a simple,  
consistent MIME format that the data is embedded in. It's just an AOL  
policy thing.

> What could their reasoning be behind their policy?

Scomp reports are sent mechanically. It's possible, and not even that  
unusual, for an scomp report to arrive in the mailbox of someone who  
was not responsible for the original email being sent, and who does  
not have legitimate knowledge of the AOL email address that the  
original mail was sent to. AOL does not give their users email  
addresses to third parties.

By removing the email address from the report AOL is doing basic due  
diligence to reduce the likelihood that their subscribers email  
address will be given to a third party who should not have access to  
it. Any legitimate bulk emailer[1] will have embedded a cookie in  
their outgoing mail such that they, and only they, can identify the  
subscriber given access to the cookie.

This is annoying to sysadmins, occasionally, when you're sending mail  
from a badly written webapp or mail is from an individual user and you  
can't identify the recipient from the scomp report, but that's the  
tradeoff for helping to ensure that the AOL users privacy is protected  
to some degree, but it doesn't inconvenience any legitimate sender of  
bulk email, and they're the recipients of the vast, vast majority of  
legitimate scomp reports.

(Note that I've argued vehemently in the past that this behavior from  
AOL is Bad and Wrong, but I understand why they do it and their  
reasoning is quite sane and fairly compelling, even if I don't always  
agree with it :) ).

>
> I am also curious as to why the ARF report cannot be viewed using  
> standard email clients, such as MS Outlook. Can someone please  
> explain this to me?
>

ARF is a pretty standard MIME format - it uses a multipart/report  
format, containing three parts - one text/plain, one message/feedback- 
report and one message/rfc822.

Any standard email client should be able to handle that format just  
fine. multipart/* is absolutely standard and the only element you need  
to read to handle the report is message/rfc822.

I'm pretty sure that Outlook can read ARF format just fine. If it  
can't, it's a fairly serious bug. Mail.app renders the text/plain and  
message/rfc822 parts inline and makes the message/feedback-report  
section - which is only usually of any use in the case of automated  
handling of reports - available as a mime attachment.

If you want to test for yourself, send an email message to [email-masked] 
  and you'll get an ARF formated report about it sent to you  
immediately - I'd be interested in knowing how Outlook handles that.

Cheers,
   Steve

[1] Any legitimate mailer will handle bounces. Any bulk emailer will  
be sending enough email, and receiving enough rejections and bounces  
that they can't handle bounces by hand. Magic cookie based approaches  
such as VERP are the only reliable way to handle bounces mechanically.  
(That there are a few ESPs who don't do this, and try and intuit from  
the bounce messages what the associated email address might be doesn't  
affect the general observation :) ).

From: "Jenny Gavin-Wear" <[email-masked]>
To: <[email-masked]>
Date: Sat, 12 Jul 2008 16:45:20 +0100
Importance: Normal
Subject: [feedback-report] ARF
Reply-To: [email-masked]

This is a multi-part message in MIME format.

------=_NextPart_000_0047_01C8E43E.ABFFEA40
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Hi all,

I have been researching ARF in relation to the AOL FBL.

My situation is that I am a web site developer (Coldfusion) offering hosting
and email services to my customers.  Some of those customers are using mail
lists.

My problem is that AOL users incorrectly mark mail list emails as spam
instead of unsubscribing, despite all of the options I have made available -
such as one click unsubscribe.

So, I need to be able to find the email address of the complainant from the
AOL ARF report.

I am running two mail servers, (Mdaemon and Smartermail) on Windows 2003
Server Standard.

On researching a solution for the Windows platform I find nothing.  Worse
still, these is very little information on the Net about the ARF format and
on how I might be able to parse it.

I would appreciate any information that can lead me to finding a solution.
It would be my intention to make any script I might be able to create
publicly available.

In your development of ARF as a standard, I would like you to consider this,
and correct me if I am wrong: AOL state that they are not prepared to
provide the email addresses of complainants and will only provide the
message ID of the email that generated the complaint - from which I can then
find the email address.  Wouldn't it be easier to provide the email address
in the ARF report in the first place?  What could their reasoning be behind
their policy?

I am also curious as to why the ARF report cannot be viewed using standard
email clients, such as MS Outlook. Can someone please explain this to me?

Thanks in advance,

Jenny Gavin-Wear
Fast Track Online

------=_NextPart_000_0047_01C8E43E.ABFFEA40
Content-Type: text/html;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Diso-8859-1">
<META content=3D"MSHTML 6.00.6000.16674" name=3DGENERATOR></HEAD>
<BODY>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>Hi=20
all,</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>I have =
been=20
researching ARF in relation to the AOL FBL.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>My =
situation is that=20
I am a web site developer (Coldfusion) offering hosting and email =
services to my=20
customers.&nbsp; Some of those customers are using mail=20
lists.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>My =
problem is that=20
AOL users incorrectly mark mail list emails as spam instead of =
unsubscribing,=20
despite all of the options I have made available - such as one click=20
unsubscribe.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>So, I =
need to be=20
able to find the email address of the complainant from the AOL ARF=20
report.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>I am =
running two=20
mail servers, (Mdaemon and Smartermail) on Windows 2003 Server=20
Standard.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>On =
researching a=20
solution for the Windows platform I find nothing.&nbsp; Worse still, =
these is=20
very little information on the Net about the ARF format and on how I =
might be=20
able to parse it.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>I =
would appreciate=20
any information that can lead me to finding a solution.&nbsp; It would =
be my=20
intention to make any script I might be able to create&nbsp;publicly=20
available.</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>In =
your development=20
of ARF as a standard, I would like you to consider this, and correct me =
if I am=20
wrong: AOL state that they are not prepared to provide the email =
addresses of=20
complainants and will only provide the message ID of the email that =
generated=20
the complaint - from which I can then find the email address.&nbsp; =
Wouldn't it=20
be easier to provide the email address in the ARF report&nbsp;in the =
first=20
place?&nbsp; What could their reasoning be behind their=20
policy?</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>I am =
also curious as=20
to why the ARF report cannot be viewed using standard email clients, =
such as MS=20
Outlook. Can someone please explain this to me?</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>Thanks =
in=20
advance,</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN=20
class=3D406545004-12072008></SPAN></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>Jenny=20
Gavin-Wear</SPAN></FONT></DIV>
<DIV><FONT face=3DArial size=3D2><SPAN class=3D406545004-12072008>Fast =
Track=20
Online</SPAN></FONT></DIV></BODY></HTML>

------=_NextPart_000_0047_01C8E43E.ABFFEA40--