Subject: Re: [domainrep] In search of the reputon

From: John Levine <[email-masked]>
Date: 1 Sep 2010 10:02:45 -0000

>> (minor detail) I believe John was the first one to propose the split
>between small and large domains/queries.

I was passing on a summary of what we discussed at DAC.

>Oops, sorry.  If the Internet were a crime scene John would be in
>trouble, his prints are all over it.  :-)

Gee, you're a fine one to talk.

>In a nutshell:  I'm trying to decouple the definition of what you're
>asserting from the mechanism by which you assert it.  -- Nathaniel

In the large model, I think that makes sense since it's likely that
different dossiers, even from the same provider, will have different
sets of data.

The idea of the small model was to have something simple and locked down,
so it was straightforward to interoperate.  Needless to say, the more
options you have, the harder it is to interoperate, and the last thing
you want is something that in theory has a zillion options, but the
undocumented reality is that nobody supports more than two of them.

R's,
John

---

From: Nathaniel Borenstein <[email-masked]>
Date: Wed, 1 Sep 2010 09:04:51 -0400

> undocumented reality is that nobody supports more than two of them.

I instinctively agree, which is why my inclination was to argue against the NAICS codes, or almost anything else not in *my* critical path.  But I've come to believe that the few things that are important to any one person/application are not the few things that another cares about.  Collectively we want lots of different things from a reputation system, even within our one little email-management example of a reputation system.  If we design a framework with just one set of options, we'll end up with a long list of options most of which are useless for most purposes.  That's precisely what I'm hoping to preempt. 

We can solve hundreds of future problems by solving today's problem right.  There is a core protocol here that any reputation system needs to have, for the format and management of reputation information.  We could design a very simple "reputation for goodness" protocol that facilitates the interchange of statements that tell us A is rated 0 to 1.0, but not what for.  The rest of our effort, in this group, would be on a "reputation for email management" sub-protocol -- elaborating the meaning of each 0 to 1.0  rating with details in the attached MIME object.

Once we're done, other groups will be able to design protocols for rating other things -- telco/ISP quality of service, blogger ratings, pundit trustworthiness -- without any protocol design.  All they'll need to do is define the format of the auxiliary MIME object that conveys essentially all the semantic data related to the rating.

I can try writing some of this up as a draft if people think it would be worthwhile... -- Nathaniel

---

From: "J.D. Falk" <[email-masked]>
Date: Wed, 1 Sep 2010 14:47:54 -0700

(as co-chair, for whatever that's worth)

On Sep 1, 2010, at 6:04 AM, Nathaniel Borenstein wrote:

> I can try writing some of this up as a draft if people think it would be worthwhile...

Yes, please, and thank you.  I think it will help immensely to have actual drafts to review; these wide-ranging threads can get confusing.

---

From: Nathaniel Borenstein <[email-masked]>
Date: Mon, 20 Sep 2010 12:09:44 +0100

> Yes, please, and thank you.  I think it will help immensely to have actual drafts to review; these wide-ranging threads can get confusing.

I'm sorry this took almost 3 weeks; I've been traveling and have probably tried (or at least hoped) to do too much for this stage.  

What follows below is partial text for what I envision as set of five RFC's.  I'm completely flexible about nearly all of this -- from the document structure to those details I've actually put down -- but offer it up as a starting point for more focused discussion.  If people like the basic structure, I'll volunteer to serve as editor for one or two of the five documents, most likely the first two.

A few notes relative to the discussion on this list:

-- I'm hoping that the notion of "Assertions" in this specification will address Doug's concerns about being too abstract to be useful.

-- One thing I've done that hasn't been mentioned on the list is that I've changed the notion of the rater's confidence in the rating into a value that expresses the number of data points on which the rating is based.  I think that might make it easier to compare rating in formation from two different raters.  But we might want both, too.

-- Also, I don't agree with Murray's comment that AGENT-A (in my original formulation, called RATER below) and AGENT-B ("RATED") are unnecessary, because the data might be passed on to a third party after an initial query.  But perhaps they should be optional so that the initial query can be more efficient, with the information added when passed on, I'm not sure.

-- I apologize if I missed anything substantive in the discussion that I should have taken into account.  Please let me know if I misssed anything.

The draft text follows below.  Fire away.  -- Nathaniel
=================================
Outline -- Five Documents

I.  A Model for Reputation Interchange

II.  A Mime Type for Reputation Information

III.  A Vocabulary for Email Senders' Reputation

IV.  Using the DNS for Reputation Interchange (the "small query")

V.  Using HTTP/XML for Reputation Interchange (the "large query")

============
Document #1:  A Model for Reputation Interchange

This document describes the general model underlying a set of proposals for the exchange of reputation information on the Internet, and provides a roadmap to the four additional documents that collectively define a reputation interchange protocol.  It is intended to roughly follow the recommendations of RFC 4101 for describing a protocol model.

The Problem to be Solved

Traditionally, most Internet protocols have taken place between unauthenticated entities.  For example, when an email message is submitted via SMTP, the server typically trusts the self-identification of the sender, and the sender trusts that the DNS has led it to the right server.  Both kinds of trust are easily betrayed, leading to spam, phishing, and a host of other ills.

In recent years, stronger identity mechanisms have begun to see wider deployment.  For example, the DKIM protocol permits a much higher level of trust in the identity of the sending domain of an email message.  While this is a major step forward, by itself it does little to solve the problem of bad actors on the Internet.  Even if you can be sure a message comes from a domain called "trustworthy.com," you don't really know whether or not that domain is trustworthy.  As a practical matter, the bad actors seem to have adopted DKIM even more rapidly than the good ones, in the hope that some receiving domains will naively confuse a confirmation of identity with trustworthiness.

The next step, which could usefully be undertaken only in the presence of such stronger identity mechanisms, is to establish a mechanism by which mutually trusted parties can exchange information about other parties.  Such information is known as reputation information.

While the need for reputation information has been most clear in the email world, where abuses are commonplace, it is easy to think of additional uses for such information.  It could also be useful in rating the security of web sites, the quality of service of an ISP or ASP, the customer satisfaction levels at e-commerce sites, and even things unrelated to Internet protocols, such as rating plumbers, hotels, or books.  Just as human beings traditionally rely on the recommendations of trusted parties in the physical world, so too they can be expected to make use of such reputation information in a variety of applications on the Internet.

Accordingly, this protocol is designed to facilitate a wide range of reputation applications.  However, not all such reputations will need to convey the same information -- an overall reckoning of goodness versus badness can be defined generically, but specific applications are likely to want to describe reputations for multiple attributes; an e-commerce site might be rated on price, speed of delivery, customer service, etc., and might receive very different ratings on each.  Therefore this protocol defines a generic mechanism and basic format for reputation information, while allowing extensions for each application.

Information Represented in the Protocol

The basic information to be represented in the protocol is fairly simple, and should include:

-- The identity of the entity providing the reputation information

-- The level of confidence in that identity being genuine

-- The identity of the entity being rated

-- The overall rating score for that entity

-- The number of data points underlying that score.

Beyond this, arbitrary amounts of additional information might be represented for specific applications of the protocol.  Such information is called the vocabulary for that application.  The general protocol defines a syntax for representing such vocabularies, but each application must define its own vocabulary.  Thus the basic information must also include

-- The name of the application for which the reputation data is being expressed.

For example, a subsequent document will define the reputation vocabulary for the application "email-sending-domain" which will be used to combat spam and other abuses of email.  Additional documents define a MIME type for reputation data, and protocols for exchanging such data.

Information Flow in the Protocol

The basic reputation data represented in the new MIME type can be transported in any number of ways, like any MIME object.  However, it is anticipated that the typical use of the protcol will be a simple request/response.  One entity will ask a second entity for reputation data about a third entity, and the second entity will respond with that data.

It is anticipated that a few applications, at least including the email-sending-domain application, will need a small, lightweight protocol for such queries and responses, while other applications will need to be able to retrieve larger and more complex responses.  For this reason, two subsequent documents define two such protocols, one based on DNS queries and a terse representation, and one based on http queries with an XML representation.

Document #2:  A MIME Type for Reputation Information

This document defines a MIME type, application/reputon, for the representation of reputational data.  This MIME type has one optional parameter, "app,"  which conveys the specific application of reputation data in use, and usually extends the set of data values that may be included in the MIME object itself.  

The body of the MIME type consists of RFC822-style attribute/value pairs, six of which are standard for all apps:

RATER:  The identity of the entity providing the reputation information, generally expressed as a domain name.

RATER-AUTHENTICITY:  The level of confidence in that identity being genuine, on a scale of 0 to 1.

ASSERTION:  A keyword indicating the specific assertion or claim being rated.  In the absence of an app parameter, the reputon can only indicate generic goodness, with the default assertion "IS-GOOD," but each app is expected to define additional types of ASSERTION.

RATED:  The identity of the entity being rated

RATING: The overall rating score for that entity, on a scale of 0 to 1.

SAMPLE-SIZE:  The number of data points underlying that score, possibly an approximation.

Thus, the following example:

      Content-type: application/reputon

      RATER:  RatingsRUs.com
      RATER-AUTHENTICITY: 1.0
      ASSERTION: IS-GOOD
      RATED: Alex Rodriguez
      RATING: 0.99
      SAMPLE-SIZE: 50000

indicates that we are absolutely sure (1.0) that the entity RatingsRUs.com consolidated 50000 data points (perhaps from everyone in Yankee Stadium) and concluded that Alex Rodriguez is very very good (0.99) at something.  It doesn't tell us what he's good at, and while it might be playing baseball, it could just as well be dating beautiful women. 

 A more sophisticated usage would define a baseball app with a vocabulary of specific assertions, so that this example::

      Content-type: application/reputon; app=baseball

      RATER:  baseball-reference.com
      RATER-AUTHENTICITY: 1.0
      ASSERTION: HITS-FOR-POWER
      RATED: Alex Rodriguez
      RATING: 0.99
      SAMPLE-SIZE: 50000

would indicate that 50000 fans polled by the entity baseball-reference.com rate A-Rod very highly in hitting for power, whereas this example:

      Content-type: application/reputon; app=baseball

      RATER:  baseball-reference.com
      RATER-AUTHENTICITY: 1.0
      ASSERTION: CLUTCH-HITTER
      RATED: Alex Rodriguez
      RATING: 0.4
      SAMPLE-SIZE: 50000

would indicate that a similar poll indicated a somewhat weaker consensus that A-Rod tends to choke in critical situations.

In practice, most usage of reputons is expected to make use of the "app" parameter to target an application-specific set of assertions.

... TBD:  Requirements for authentication of reputation data, e.g. DKIM signing...

Document #3:  A Vocabulary for Email Senders' Reputation

This document defines the additional possible assertions and other additional fields for an app value of "email-sending-domain," used to rate email senders.  It will define such values as:

SENDS-SPAM

PERPETRATES-FRAUD

SENDS-MALWARE

... TBD ...  See "Threat Taxonomy" thread on domainrep mailing list.

Document #4:  Using the DNS for Reputation Interchange (the "small query")

... TBD ...

Document #5:  Using HTTP/XML for Reputation Interchange (the "large query")

... TBD ...

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 20 Sep 2010 15:00:26 +0100

I generally like this, but have two concerns.

1. Who'd ever say anything other than 1.0 for rater
authenticity? I guess I just don't get what that's
for since its always (?) the rater that's answering
the question.

2. I don't see the generic benefit of the #data
points.

S.

On 20/09/10 12:09, Nathaniel Borenstein wrote:
> _______________________________________________
> domainrep mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/domainrep
>

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 21 Sep 2010 09:28:24 +0200

On 20/Sep/10 16:00, Stephen Farrell wrote:
> I generally like this

+1

> but have two concerns.

This seems somewhat of a misnomer to me, since reputon exchanges 
should always be authenticated using good crypto stuff.  But then also 
terms like "reliability", or "trustworthiness" can be source of 
confusion.  If I interpret Nathaniel's sketch correctly, this number 
is the probability that the measure asserted by a rater (for their 
systems) is also correct for the system at hand.

> 2. I don't see the generic benefit of the #data

The ability to scale back a rating by the underlying sample size 
allows to add different ratings and then re-normalize the result. 
Thus it is possible to aggregate assertions made on different times or 
by different raters.

It will have to be determined whether the (same) aggregation formula 
is defined for the general model, rather than for each application, 
rater, or assertion type.

>> The basic reputation data represented in the new MIME type can be
>> transported in any number of ways, like any MIME object.
>> However, it is anticipated that the typical use of the protcol
>> will be a simple request/response.  One entity will ask a second
>> entity for reputation data about a third entity, and the second
>> entity will respond with that data.

I'd say "that /one/ typical use" will be request/response.  A rater 
may want to submit a reputon to one or more reputation trackers 
without having received a request for it, e.g. as a consequence of 
processing abuse reports.

>> This document defines the additional possible assertions and
>> other additional fields for an app value of
>> "email-sending-domain," used to rate email senders.  It will
>> define such values as:
>>
>> SENDS-SPAM
>>
>> PERPETRATES-FRAUD
>>
>> SENDS-MALWARE

Those examples are all negative.  For positive ones:

IDENTIFIES-USERS

HANDLES-REPORTS

PUNISHES-SPAMMERS

---

From: "J.D. Falk" <[email-masked]>
Date: Tue, 21 Sep 2010 14:24:18 -0700

> Those examples are all negative.  For positive ones:

Those all sound like "not-bad" characteristics, which aren't quite the same as "good"...but in any case I expect we'll see a wide range of values at first, then in practice we'll discover that a much smaller list is sufficient.  For example, a Reputation Provider whose input data involves manually testing unsubscribe processes might at first think they need a PROCESSES-UNSUBSCRIBES assertion, when in fact all they need is the default IS-GOOD because they only process one type of data.

As a general framework, I like this proposal a lot.  I think we'll need to keep SAMPLE-SIZE optional, because for commercial applications that's likely to be either secret sauce or an improbable boast.

Nathaniel, what do you have in mind for the next draft?

(A side note: trustworthy.com does not appear to live up to the name.)