Subject: [p2pi] Charter and problem statement

From: Enrico Marocco <[email-masked]>
Date: Fri, 11 Jul 2008 10:20:50 +0200

Vijay and I have put together a first draft of the charter, trying to
capture how people on this list think an IETF working group should
address the ALTO problem. It is available at the following link:
http://alto.tilab.com/docs/charter.txt
Please take a look and comment.

An updated version of the problem statement is also available:
http://www.ietf.org/internet-drafts/draft-marocco-alto-problem-statement-02.txt
Changes should reflect comments received in regards to the previous
version, including a more accurate description of how caching relates to
ALTO. We are aware that terminology may need to be revised (as pointed
out by Dimitri). As usual, you can use IETF tools to visualize diffs:
http://tools.ietf.org//rfcdiff?url1=http://tools.ietf.org/id/draft-marocco-alto-problem-statement-01&url2=http://www.ietf.org/internet-drafts/draft-marocco-alto-problem-statement-02.txt

-- 
Ciao,
Enrico
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: Stanislav Shalunov <[email-masked]>
Date: Fri, 11 Jul 2008 01:56:14 -0700

> http://alto.tilab.com/docs/charter.txt

Thank you for putting together the charter well enough ahead of the  
BoF that we have time to discuss it.

The general problem of overlay routing and how to best expose  
information for it is big and hard.

The problem of caching, on the other hand, is well-defined, easily  
scoped, and easy to agree on.  Cache tracker location is a pure and  
simple engineering problem.  Routing information exposure has a  
substantial research component.

This charter does not let the possible WG solve the easy problem -- in  
fact, it does not even contain the word "cache".  Instead, it proposes  
a particular solution (oracle) for the more complicated routing  
information exposure problem, before the problem has even been scoped.

Further, the chartered oracle location service is of exactly the same  
nature and complexity as the non-chartered cache tracker location  
service.

-- 
Stanislav Shalunov
http://shlang.com

---

From: Salman Abdul Baset <[email-masked]>
Date: Fri, 11 Jul 2008 06:15:36 -0400 (EDT)

In addition to comments by Stanislav, a question to be considered is 
whether the signaling protocol should support cooperation of oracles in 
addition to querying of oracle. Is there a need for such cooperation?

-s

On Fri, 11 Jul 2008, Stanislav Shalunov wrote:

> Thank you for putting together the charter well enough ahead of the BoF that 
> we have time to discuss it.
>
>
> The general problem of overlay routing and how to best expose information for 
> it is big and hard.
>
> The problem of caching, on the other hand, is well-defined, easily scoped, 
> and easy to agree on.  Cache tracker location is a pure and simple 
> engineering problem.  Routing information exposure has a substantial research 
> component.
>
> This charter does not let the possible WG solve the easy problem -- in fact, 
> it does not even contain the word "cache".  Instead, it proposes a particular 
> solution (oracle) for the more complicated routing information exposure 
> problem, before the problem has even been scoped.
>
> Further, the chartered oracle location service is of exactly the same nature 
> and complexity as the non-chartered cache tracker location service.
>
> -- 
> Stanislav Shalunov
> http://shlang.com
>

---

From: Enrico Marocco <[email-masked]>
Date: Fri, 11 Jul 2008 16:06:30 +0200

> information exposure problem, before the problem has even been scoped.

Ok, I see that the first change must be on the term "oracle". It wasn't
of course meant to push any particular solution; in our mind, "oracle"
could be any source of topology information, including the IDIPS
"sorting oracle" as well as the P4P iTracker or the Ono/Akamai redirect
server.

That said, I understand caches as "special" peers that can be
transparently handled by the peer selection service we are trying to
standardize (as suggested in S2.1.1 of the problem statement).
Alternatively, we would have to work on some sort of
application-dependent cache location mechanism, which, AFAICT, is not
what most people here want to pursue and I don't think is what you are
proposing. Can you suggest a rewording for the current charter to best
address caching issues?

Finally, the survey draft references several solutions for addressing
the topology information exposure problem, which have proved in the real
world to be beneficial to P2P applications in the first place (and some
to ISPs as well). I really don't want people to think that we are going
to invent a new one; instead, what we are proposing is just to define a
standard interface for existing and future peer selection optimization
services, in order to ease their adoption by different P2P and non-P2P
applications.

If you agree, but don't see it reflected in the charter, feel free to
propose alternative text.

---

From: Laird Popkin <[email-masked]>
Date: Fri, 11 Jul 2008 10:13:22 -0400 (EDT)

IMO caches are a (rather valuable) special case of the general problem of finding optimal sources of data (for data that has multiple available sources).

To elaborate on this, once the ISP has an "oracle" that communicates network policies to applications, that seems like an ideal channel for communicating information about caches. Doing so explicitly and under the direction of the ISP and the support of the p2p networks seems much better than the techniques that those guys use now (e.g. multiple proprietary cache discovery protocols, or deep packet inspection and protocol hacking). As a p2p developer, I have to say that it's far more likely that we'd support a single standard cache location mechanism (especially if it gave all of the other benefits of an "oracle") than for us to test multiple proprietary cache discovery mechanisms on every download.

- Laird Popkin, CTO, Pando Networks
  mobile: 646/465-0570

----- Original Message -----
From: "Enrico Marocco" <[email-masked]>
To: "Stanislav Shalunov" <[email-masked]>
Cc: [email-masked]
Sent: Friday, July 11, 2008 10:06:30 AM (GMT-0500) America/New_York
Subject: Re: [p2pi] Charter and problem statement

-- 
Ciao,
Enrico

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Fri, 11 Jul 2008 09:53:56 -0500

Laird Popkin wrote:
> IMO caches are a (rather valuable) special case of the general
> problem of finding optimal sources of data (for data that has
> multiple available sources).
> 
> To elaborate on this, once the ISP has an "oracle" that communicates
> network policies to applications, that seems like an ideal channel
> for communicating information about caches. Doing so explicitly and
> under the direction of the ISP and the support of the p2p networks
> seems much better than the techniques that those guys use now (e.g.
> multiple proprietary cache discovery protocols, or deep packet
> inspection and protocol hacking). As a p2p developer, I have to say
> that it's far more likely that we'd support a single standard cache
> location mechanism (especially if it gave all of the other benefits
> of an "oracle") than for us to test multiple proprietary cache
> discovery mechanisms on every download.

Laird, Stanislav: I think both of you are proposing that cache-
discovery be a part of ALTO.  I am trying to make a distinction
between /cache/ and /content/ in the context of our work.

In a P2P application-based overlay, the "best" peer (however
the network defines "best") is where the querying peer should
be sent to get the resource in question from.  If we stretch
this analogy a bit further, it does not matter that the resource
is mirrored (cached) at the "best" peer or it appears as
the only instance of resource on the "best peer" (i.e., it is
not cached.)  So in cases like this, I believe that sending
a querying peer to the "best" peer is immaterial of whether
or not the "best" peer is a caching-only server versus a
server providing original content.  The content is immutable.

In other applications like HTTP-based caching, it matters whether
or not a cache is hit or the origin server is hit.

If caching is deemed to be a part of the charter, how do we define
it for ALTO?

Thanks,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
2701 Lucent Lane, Rm. 9F-546, Lisle, Illinois 60532 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
WWW:   http://www.alcatel-lucent.com/bell-labs
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: Laird Popkin <[email-masked]>
Date: Fri, 11 Jul 2008 11:10:41 -0400 (EDT)

For this level of discussion, I agree that caches should simply be viewed as a peer in ALTO.

I would point out some differences that we'll want to discuss at some point, as I think that they'll require some mild extensions to ALTO as we've discussed it so far:
- The cache servers are never part of the ALTO client request (since the p2p network doesn't know about them) but the ALTO server needs to give out exact addresses of caches in its response, since they're peers that are unknown to the p2p network. This is different from the more abstract network locations (e.g. IP prefix, ASN) that ALTO otherwise uses (IMO, anyway) in order to protect user privacy. There isn't a privacy concern with revealing the location of an ISP-run cache server to the ISP. :-)
- Cache servers are protocol, application and perhaps even content specific, so that information needs to be (optionally) communicated in the ALTO request. For example, an ISP might operate BitTorrent and Pando caches, but not eDonkey caches, or it might operate different caches for different protocols on different hardware at different locations, or it might decide to cache some content (e.g. commercial content) but not other content (e.g. consumer generated content) that is transported using the same protocol and application.
- There might be some additional policy information that the ISP may wish to communicate. For example, an ISP that invests heavily in caching to relieve uplink congestion might want p2p apps to use the cache servers to the exclusion of peers within the network, etc. Of course, the implementation of this is up to the p2p application, but at the least the ISP should be able to communicate preferences.

----- Original Message -----
From: "Vijay K. Gurbani" <[email-masked]>
To: "Laird Popkin" <[email-masked]>, "Stanislav Shalunov" <[email-masked]>
Cc: [email-masked]
Sent: Friday, July 11, 2008 10:53:56 AM (GMT-0500) America/New_York
Subject: Re: [p2pi] Charter and problem statement

Thanks,

- vijay

---

From: Reinaldo Penno <[email-masked]>
Date: Fri, 11 Jul 2008 08:30:14 -0700

Some comments inline...

On 7/11/08 8:10 AM, "Laird Popkin" <[email-masked]> wrote:

> I would point out some differences that we'll want to discuss at some point,
> as I think that they'll require some mild extensions to ALTO as we've
> discussed it so far:
> - The cache servers are never part of the ALTO client request (since the p2p
> network doesn't know about them)

After a client knows about the IP address of the cache (which in its
interpretation might be just a very good peer), it can communicate such
'peer' to others peers in the network.

As I'm sure you know both Bittorrent and emule have source exchange between
peers, so after the cat is out of the box, how to prevent such information
from being disseminated? Or do we want to prevent it?

If the idea is for peers to use caches, such information should be
disseminated, no?

but the ALTO server needs to give out exact
> addresses of caches in its response, since they're peers that are unknown to
> the p2p network. This is different from the more abstract network locations
> (e.g. IP prefix, ASN) that ALTO otherwise uses (IMO, anyway) in order to
> protect user privacy. There isn't a privacy concern with revealing the
> location of an ISP-run cache server to the ISP. :-)
> - Cache servers are protocol, application and perhaps even content specific,
> so that information needs to be (optionally) communicated in the ALTO request.
> For example, an ISP might operate BitTorrent and Pando caches, but not eDonkey
> caches, or it might operate different caches for different protocols on
> different hardware at different locations, or it might decide to cache some
> content (e.g. commercial content) but not other content (e.g. consumer
> generated content) that is transported using the same protocol and
> application.

I believe the front end of the cache might be protocol specific. So, emule
and Bittorrent, use different protocols divide the file, hash and serve
content, but the content is the same.

Of course there might be policies involved in commercial vs. user generated
content as you point out.

> - There might be some additional policy information that the ISP may wish to
> communicate. For example, an ISP that invests heavily in caching to relieve
> uplink congestion might want p2p apps to use the cache servers to the
> exclusion of peers within the network, etc. Of course, the implementation of
> this is up to the p2p application, but at the least the ISP should be able to
> communicate preferences.

Agreed, but does not seem to within the scope of ALTO. It would probably be
within proprietary extensions.

---

From: Laird Popkin <[email-masked]>
Date: Fri, 11 Jul 2008 11:42:50 -0400 (EDT)

Comments below.

----- Original Message -----
From: "Reinaldo Penno" <[email-masked]>
To: "Laird Popkin" <[email-masked]>, "Vijay K. Gurbani" <[email-masked]>
Cc: [email-masked], "Stanislav Shalunov" <[email-masked]>
Sent: Friday, July 11, 2008 11:30:14 AM (GMT-0500) America/New_York
Subject: Re: [p2pi] Charter and problem statement

Some comments inline...

LP: Yes, Pando implements peer-exchange as well. And it's probably better for cache server addresses not to be passed around. For example, if a peer in ISP A finds out about a cache server in ISP A's infrastructure, and tells a peer in ISP B about that peer, you'll have ISP B's customers using the cache server at ISP A (or at least trying to). It'd probably be better for the client to know that it's been informed of a peer operated by their ISP so that they know not to send it around to other peers, because that'll just lead to lots of failed connections (or a cache server serving off-net).

LP: Yes, some cache servers support multiple protocols over one set of content, while others are protocol specific (e.g. HTTP caches are different from eDonkey caches), or support multiple protocols, each with their own set of content. Given that we don't know how this will all play out over time, I'd suggest giving the ISP (and ALTO server) maximum flexibility to decide what to do by having a few fields that can be used to communicate information such as application, protocol, etc., and let the ALTO server implementors figure out what to do with them based on ISP requirements. :-)

LP: As long as it's extensible (which ALTO will be) that's fine. Though hopefully really basic concepts like this can be negotiated as "open" rather than "proprietary" extensions.

---

From: Stanislav Shalunov <[email-masked]>
Date: Fri, 11 Jul 2008 13:13:06 -0700

OK, I misunderstood the use of oracle -- suspect it might be confusing  
to others in this context?  Thank you for clarifying.

I'll send proposed textual changes to the charter later.

My main concern that the solution to the easy location problem is not  
dependent on the solution for the hard information conveying problem.

Since the easy location problem needs to be solved anyway -- or else  
one can't find a routing information provider/oracle/whatever we want  
to call it -- we need a mechanism for it regardless.

I'll think about how to package and phrase this best and send it in  
another message.

Thanks,  -- Stas

-- 
Stanislav Shalunov
http://shlang.com

---

From: "Damien Saucez" <[email-masked]>
Date: Sat, 12 Jul 2008 19:09:43 +0200 (CEST)

Salman,

> addition to querying of oracle. Is there a need for such cooperation?
I think that the cooperation is important and probably that the
difficulties in the design are a matter of politic instead of technique
(business relationship).

Damien Saucez

>> scoped,
>> fact,
>> nature

---

From: Sebastian Kiesel <[email-masked]>
Date: Mon, 14 Jul 2008 17:41:16 +0200

> viewed as a peer in ALTO.

What happens if we consider caches and rare content, i.e., content where
the likelyhood is high, that there is a cache miss when we ask for it
and that the cache will never deliver a second copy of it? It might not
be wise to select a cache which is in the "opposite direction" of the
original content, even if connectivity to the cache is "better"
(according to whatever criteria). If the cache tries to fetch the
content the request may pass by the client and go through the same "bad"
link plus twice through the "good" one.

Cache ====Link1(good)==== Client -----------Link2(bad)----------- Server

This becomes even more important if we want to generalize ALTO and use
it, e.g., to select media relays for VoIP NAT traversal:

                       UserA                     UserB
                         |                         |
                        NAT                       NAT
                         |                         |
Relay1 ====L1(medium)====+---------L2(bad)---------+==L3(good)== Relay2
                         |
                        NAT
                         |
                       UserC

UserA should tell the ALTO service not only that it needs "a VoIP relay"
but also that he wants to talk to UserB.

Thanks,
Sebastian

-- 
Sebastian Kiesel            mailto:[email-masked]
Network Research Division   tel:+49-6221-4342-232   fax:+49-6221-4342-155
NEC Laboratories Europe     Kurfuerstenanlage 36, 69115 Heidelberg, Germany
--
NEC Europe Limited          Registered in England 2832014
Registered Office           NEC House, 1 Victoria Road, London W3 6BL
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: "Livingood, Jason" <[email-masked]>
Date: Mon, 14 Jul 2008 16:03:03 -0400

So are you suggesting that you may want a response to the client to
include multiple responses (cache and content) so that a client can try
them in sequence or in parallel?  I think I understand the problem of
rare content, but perhaps in the p2p area the speed of finding the
optimal resource for the content has been a lower priority than finding
a high-speed resource (since a file transfer could take several minutes
to hours).  This is quite different from a real-time application in that
way I suppose.

Jason 

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Sebastian Kiesel
> Sent: Monday, July 14, 2008 11:41 AM
> To: Laird Popkin
> Cc: [email-masked]
> What happens if we consider caches and rare content, i.e., 
> content where the likelyhood is high, that there is a cache 
> miss when we ask for it and that the cache will never deliver 
> a second copy of it? It might not be wise to select a cache 
> which is in the "opposite direction" of the original content, 
> even if connectivity to the cache is "better"
> (according to whatever criteria). If the cache tries to fetch 
> the content the request may pass by the client and go through 
> the same "bad"
> This becomes even more important if we want to generalize 
> ALTO and use it, e.g., to select media relays for VoIP NAT traversal:
> 
>                        UserA                     UserB
>                          |                         |
>                         NAT                       NAT
>                          |                         |
> Relay1 
> ====L1(medium)====+---------L2(bad)---------+==L3(good)== Relay2
>                          |
>                         NAT
>                          |
>                        UserC
>

---

From: Stanislav Shalunov <[email-masked]>
Date: Mon, 14 Jul 2008 13:33:56 -0700

Even when content is going to be downloaded by an AS exactly once,  
there can be benefits to using a cache.

Consider a network like cable where uplink capacity is scarce and  
shared.  Running through a cache can enable one to use less of it.   
The fact that the content will pass twice through a link between the  
datacenter and the ISP's backbone is no big deal -- it's a fast fiber  
connection and can handle far more traffic than the cable uplink.

In general, however, caching strategies is a rich field ("all computer  
science" and all that).  The good news that the strategy can be left  
up to the applications and the caches.  There's no interoperability  
need to make it standard.

-- Stas

-- 
Stanislav Shalunov
http://shlang.com

---

From: Enrico Marocco <[email-masked]>
Date: Tue, 15 Jul 2008 15:32:18 +0200

> link plus twice through the "good" one.

As long as caches are viewed as regular peers, they will be included in
ALTO requests/responses only if they actually have the resource the
querying peer wants to access.

---

From: Laird Popkin <[email-masked]>
Date: Tue, 15 Jul 2008 09:38:15 -0400 (EDT)

Actually, caches will probably want to be included in all ALTO responses so that they are contacted by peers, so that they can determine whether they want to cache the content (and have a data source for the cache if so).

---

From: stefano previdi <[email-masked]>
Date: Tue, 15 Jul 2008 15:41:04 +0200

Laird,

do you suggest to make the cache entity explicit in the ALTO protocol
semantic ? 

Or is it just part of the opaque protocol data the algorithm (not under
standardization at this point) will work with ?

I'd vote for the latter...

thanks.

s.

> From: Laird Popkin <[email-masked]>
> Date: Tue, 15 Jul 2008 09:38:15 -0400 (EDT)
> To: Enrico Marocco <[email-masked]>
> Cc: <[email-masked]>
> Actually, caches will probably want to be included in all ALTO responses so
> that they are contacted by peers, so that they can determine whether they want
> to cache the content (and have a data source for the cache if so).
>

---

From: Sebastian Kiesel <[email-masked]>
Date: Tue, 15 Jul 2008 16:59:05 +0200

On Mon, Jul 14, 2008 at 04:03:03PM -0400, Livingood, Jason wrote:
> them in sequence or in parallel?

I am not sure what the conclusion is, I just wanted to raise the issue:
The goal of ALTO is to provide information that cannot be acquired
easily by the peers using measurements between each other. 

> I think I understand the problem of
> to hours). 

If it might happen that candidate caches do not already have the rare
content, any recommendation should not only consider the link between
client and cache but also the link from the cache to the "upstream
source", from which the content is retrieved by the cache.

thanks,
Sebastian

---

From: Reinaldo Penno <[email-masked]>
Date: Tue, 15 Jul 2008 08:09:32 -0700

Hi,

Well, this brings a question if we are here to standardize a client-server
protocol or the data it carries as well.

As we standardize the protocol we need to go through what kind of
information it queries. Are these peers, network masks, weights, etc? How
they are represented? What is mandatory and what is optional? These can be
opaque to the protocol in the wire but certainly not to the application.

Since we are talking about applications and we need to interoperate we need
to define those. At least that was my assumption.

If we are here to just standardize a client-server protocol, there many,
many choices already available, including good ol' HTTP.

So, if caches are just peers, they would be identified as peers or a special
type of peer.

Thanks,

Reinaldo

---

From: songhaibin 64081 <[email-masked]>
Date: Tue, 15 Jul 2008 23:12:10 +0800

----- Original Message -----
From: Sebastian Kiesel <[email-masked]>
Date: Tuesday, July 15, 2008 10:59 pm
Subject: Re: [p2pi] Charter and problem statement

> source", from which the content is retrieved by the cache.

I agree with you at this point: )
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: Stanislav Shalunov <[email-masked]>
Date: Tue, 15 Jul 2008 17:05:43 -0700

> between /cache/ and /content/ in the context of our work.

Cache is a computer.  Content is a sequence of bits.  How can there be  
confusion?

> be sent to get the resource in question from.

What's a resource in this context?  Like a given file?  P2P apps  
(those that work, that is) are designed to deal with partial files and  
do so quite reasonably.  You generally don't so much get a file from  
somewhere as you trade pieces with multiple peers.

A cache is a special kind of peer simply because it's fast.  As it  
feeds to the data, you discover you can slurp it up from the cache  
that wants nothing in return.  And then you're done.

> If we stretch

OK.

> or not a cache is hit or the origin server is hit.

Agreed.

> If caching is deemed to be a part of the charter, how do we define

As long as ALTO provides a way to find it, the application -- be it a  
P2P app or a web browser -- could then decide how to use it  
appropriately.  HTTP caching has defined semantics with bypass  
pragmas, etc.  Current-gen P2P systems don't need this, because they  
are currently used only for static content delivery.  If ways are  
found to distribute dynamic content through P2P, the cache notion will  
evolve, but the mechanism to find the cache would not need to.

-- 
Stanislav Shalunov
http://shlang.com

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 15 Jul 2008 22:44:52 -0500

Stanislav Shalunov wrote:
> A cache is a special kind of peer simply because it's fast.  As it feeds 
> to the data, you discover you can slurp it up from the cache that wants 
> nothing in return.  And then you're done.

OK, excellent definition.  Now ...

> appropriately.  HTTP caching has defined semantics with bypass pragmas, 
> etc.  Current-gen P2P systems don't need this, because they are 
> currently used only for static content delivery.  If ways are found to 
> distribute dynamic content through P2P, the cache notion will evolve, 
> but the mechanism to find the cache would not need to.

In an earlier email, you had pointed out one mechanism for cache
discovery in BitTorrent: BEP 22.  What do you envision ALTO doing
above and beyond the techniques outlined in:
   - BEP 22 (namely, using PTR RR and walking down the DNS
     tree to the TLD doing SRV RRs)?
   - DNS-SD work?

Excuse me if I am being dense, but I am trying to get a good
handle on exactly what can ALTO standardize for cache discovery
that is not already covered by some existing protocol.

Thanks,

- vijay

---

From: Stanislav Shalunov <[email-masked]>
Date: Tue, 15 Jul 2008 21:40:01 -0700

On Jul 15, 2008, at 8:44 PM, Vijay K. Gurbani wrote:
> discovery in BitTorrent: BEP 22.

To clarify the status of the BEP documents: they are only meaningful  
inside the BitTorrent community.

This particular BEP is new and is not yet implemented by any client in  
the wild.  To my knowledge, no ISP has configured this either.

> What do you envision ALTO doing

The cache discovery portion of ALTO could be equivalent to BEP 22, or  
it could literally be the mechanism in BEP 22 and that would be quite  
sufficient for cache discovery.

I did not want to focus on the particular solution at this point,  
because we still haven't formed a WG, but something like BEP 22 would  
be one option.

>  - DNS-SD work?

DNS-SD would be another solution, also based on DNS.  (BEP 22 could be  
viewed as inspired by DNS-SD, but self-contained.)

There could probably be about a dozen others: special anycast address,  
etc., etc.

> that is not already covered by some existing protocol.

An ALTO document could, indeed, use an existing mechanism.  (And  
certainly whatever mechanism is used it'd be silly not to use existing  
protocol building blocks.)

The advantage of having it in an IETF standard vs, say, a BEP are:

* with an IETF standard, cache vendors would be more confident in the  
stability of the spec and would be more inclined to build caches

* ISPs would be more inclined to use and configure the discovery  
mechanism

* IETF review would likely make the mechanism technically better

* cache discovery is about more than BitTorrent, and the IETF is the  
right place to create a broad standard

The usual IETF stuff -- what we're here to do.  This is cross-vendor  
interoperability stuff, the point of standards.  While even without  
the IETF, the BEP series might provide a lightweight form of  
standardization, I'd much prefer this work to occur in the IETF.   
Particularly given that there's the ALTO BoF specifically in this  
space and given that the work is straightforward.  (As proof of  
concept, we could republish BEP 22 as an internet-draft, but again,  
it's too early for solutions drafts.)

Does this make any sense?

-- 
Stanislav Shalunov
http://shlang.com

---

From: "Stas Khirman" <[email-masked]>
Date: Tue, 15 Jul 2008 22:33:14 -0700

I would like to express some doubts:

Does "cache discovery" is in the scope of ALTO? Does it make sense to mix
together seems to be irrelevant issues?

It seems that ALTO can/have to be designed to be protocol and content
agnostic. Indeed, from traffic optimization perspective it doesn't matter if
peers are using eDonkey, BitTorrent or any other P2P ( or non-P2P) protocol
and what content they are sharing. (especially considering that some P2P
clients are capable to use multiple protocols )

Making ALTO protocol and content agnostic has many benefits:
- can be adopted by variety of P2P networks/applications
- can be adopted by non-P2P applications (such as CDN)
- eliminates all DMCA-like liability for ISP's deployment of ALTO components
- remove end-user privacy concerns

IMHO, it is possible to design and implement such "agnostic" approach as
peer selection optimization does not depend on specific protocol, but rather
on "application requirements" ( minimize delay, localize traffic , etc) ,
peers capabilities, network topology, current utilizations and other
ISP-related characteristics. I strongly believe that agnostic nature has to
be one of the requirements for ALTO core implementation. 

On the other hand, cache discovery depends on (a) protocol and (b) content.
BitTorrent cache server will not help in eDonkey traffic optimization. In
general case, given content may be preserved in multiple caches or not at
all. It seems that "cache discovery" have to be done BEFORE any attempt to
"optimize" peer list - exactly like regular peers have to be discovered
before ALTO optimization. 

Frankly, I have no opinion if "cache discovery" discussion belongs to ALTO
WG or not. But if it included, IMHO, "cache" issues have to be clearly
separated from "optimization" issues and this separation has to be
considered in future design/implementation.

Regards
Stas

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Stanislav Shalunov
> Sent: Tuesday, July 15, 2008 9:40 PM
> To: Vijay K. Gurbani
> Cc: [email-masked]

---

From: Stanislav Shalunov <[email-masked]>
Date: Wed, 16 Jul 2008 00:04:51 -0700

On Jul 15, 2008, at 10:33 PM, Stas Khirman wrote:
> agnostic.

Yes, it can, and it probably should.  Naturally it doesn't have to be  
designed this way, but it would be advantageous.

> On the other hand, cache discovery depends on (a) protocol and (b)  

Protocol, yes, in a trivially parametrizable way.  You need to plug in  
the name of the protocol.

Content, not so much, but it could be an argument that's passed in and  
then mostly ignored.

> It seems that "cache discovery" have to be done BEFORE any attempt to
> "optimize" peer list

That's application-specific.  Didn't you just argue to make it  
agnostic?  In BitTorrent, no, we would not put it before.  We'd make  
the cache discovery concurrent with everything else that's going on at  
the start of a torrent.

> WG or not.

Cache discovery was one of the themes of the workshop in Boston.   
Multiple P2P app makers, equipment vendors, and ISPs all want it to  
work.  And it's in scope for ALTO.  So while you may not have an  
opinion, others do.

> But if it included, IMHO, "cache" issues have to be clearly

Resource discovery is, of course, a prerequisite for traffic  
optimization -- you have to know whom to ask before you can ask.

I share the desire to have these in different documents, because the  
discovery piece can be very simple and would allow ALTO to quickly  
produce a deliverable that would be immediately useful.

-- 
Stanislav Shalunov
http://shlang.com

---

From: Enrico Marocco <[email-masked]>
Date: Wed, 16 Jul 2008 09:27:45 +0200

Laird Popkin wrote:
> Actually, caches will probably want to be included in all ALTO
> responses so that they are contacted by peers, so that they can
> determine whether they want to cache the content (and have a data
> source for the cache if so).

Well, to include new caches' addresses in the response, the ALTO server
needs to know the application the client is running, but I suspect not
all users will be happy to pass such information to an external entity,
maybe run by an ISP.

Once again, optional extensions to a very simple base protocol will
probably be the key.

---

From: "Stas Khirman" <[email-masked]>
Date: Wed, 16 Jul 2008 00:30:15 -0700

> From: Stanislav Shalunov [mailto:[email-masked]]

[Stas Khirman] 
I afraid your model will work only with a centralized cache where all
content located in the same single virtual node.

In general case, caching service can be distributed across ISP network - for
example, very popular content has to be pushed down, as close as possible to
the end-user (POP, CMTS, DSLAM ??), while less popular content can be cached
in more centralized manner. 

In this , general model, specific content id (info_hash for BitTorrent) is a
VERY important attribute for cache discovery. And certainly, discovery has
to be made 
(a) individually for each unique content as each discovery may return
different sets of appropriate caches
(b) before attempt to optimize peer list as cache list can be quite unique
for given content

> produce a deliverable that would be immediately useful.
[Stas Khirman] 
We are in agreement on this point.

---

From: Stanislav Shalunov <[email-masked]>
Date: Wed, 16 Jul 2008 00:47:37 -0700

> content located in the same single virtual node.

There's no harm in passing the resource id in, but a sane distributed  
design is still likely to ignore it, directing the user to the closest  
existing cache.

The good news is that the design of the caching system does not need  
to be standardized.  This is where vendors can innovate on caching  
strategies while still interoperating.

-- 
Stanislav Shalunov
http://shlang.com

---

From: Stanislav Shalunov <[email-masked]>
Date: Wed, 16 Jul 2008 00:48:32 -0700

On Jul 16, 2008, at 12:33 AM, Song Haibin wrote:
> If so, the ALTO
> client must identify the "kind" of cache it wants to get from the ALTO
> server's selection. I don't know if it violates the rule of  
> application
> agnostic here.

ALTO doesn't need to know anything about BitTorrent to make it  
possible to discover caches for a string "BitTorrent".  You could also  
ask for "HTTP" or "gopher", and again, ALTO would not need to know how  
these work.

To interoperate on the ALTO level, it is only necessary to standardize  
these handles.  (Whatever is behind that label needs to interoperate  
somehow, but it's not ALTO's job to make sure it does.  It's the job  
of whoever specified the protocol to which the label refers.)  While  
the list is potentially very long, we're trying to solve a specific  
problem, for which the practical list will consist of very few entries.

-- 
Stanislav Shalunov
http://shlang.com

---

From: songhaibin 64081 <[email-masked]>
Date: Wed, 16 Jul 2008 16:08:32 +0800

----- Original Message -----
From: Stanislav Shalunov <[email-masked]>
Date: Wednesday, July 16, 2008 3:48 pm
Subject: Re: [p2pi] Charter and problem statement

> these handles.  

Yes, that is the point.

BR
Song Haibin

---

From: Reinaldo Penno <[email-masked]>
Date: Wed, 16 Jul 2008 02:11:46 -0700

The discovery procedure based on DNS and outlined in BEP 22 is related to
Local Trackers (you could think of them as ALTO servers/service).

The cache discovery piece is done through HTTP.

In a ALTO world the DNS based procedure outlined in BEP 22 (SRV based) could
be used to find ALTO servers/service and this service could (thorough the
protocol) indicate/order a list of caches.

Thanks,

Reinaldo

---

From: Reinaldo Penno <[email-masked]>
Date: Wed, 16 Jul 2008 02:22:43 -0700

I agree. 

We seem to be stuck in caching as if it is something special but it seems to
me this is a general problem with any piece of network/topology information
an ALTO server wants to convey.

The entities involved in this protocol (P2P clients, ALTO servers, etc) need
to to be able to encode/decode and understand in a uniform fashion the
notions of weight, distance, peer, etc. The notion of passing caching
information is just one aspect of it.

Thanks,

Reinaldo

---

From: songhaibin 64081 <[email-masked]>
Date: Wed, 16 Jul 2008 17:44:26 +0800

Hi Reinaldo,

> protocol) indicate/order a list of caches.

Do you mean we use DNS SRV(or DNS-SD) to locate the ALTO servers and then use the ALTO service to rank the list of caches for a ALTO client?

The question is how to choose a specific ALTO server for a client if there are more than one?

BR
Song Haibin

---

From: Reinaldo Penno <[email-masked]>
Date: Wed, 16 Jul 2008 02:59:14 -0700

Hi,

BEP 22 suggests a procedure based on DNS to make a choice. Others are
possible. 

People have tried to solve the important problem of "locating a
server/service" in previous/current WGs with wide different levels of
success such as SLPv1/v2, Zeroconf, DNS, DHCP, etc.

DNS is usually what is featured on SIP/SIPPING/BEHAVE documents to discover
SIP/STUN servers. 

Thanks,

Reinaldo 

On 7/16/08 2:44 AM, "songhaibin 64081" <[email-masked]> wrote:

> Do you mean we use DNS SRV(or DNS-SD) to locate the ALTO servers and then use
> the ALTO service to rank the list of caches for a ALTO client?
>

---

From: stefano previdi <[email-masked]>
Date: Wed, 16 Jul 2008 16:01:24 +0200

> From: songhaibin 64081 <[email-masked]>
> 
> Hi Reinaldo,
> 
>> 

there are many ways to do that among which there's service differentiation
(based on capabilities). ALTO servers may also compete to each others...

s.

---

From: stefano previdi <[email-masked]>
Date: Wed, 16 Jul 2008 16:01:23 +0200

> From: Reinaldo Penno <[email-masked]>
> 
> I agree. 
> 

yes.

I agree.

s.

---

From: stefano previdi <[email-masked]>
Date: Wed, 16 Jul 2008 16:21:32 +0200

> opaque to the protocol in the wire but certainly not to the application.

I agree. Opaque was referred to the protocol in the sense that it doesn't
trigger any specific behavior of the protocol machinery.

> So, if caches are just peers, they would be identified as peers or a special

we need to find some form of characterization of the peer/resource and let
the application to handle it.

s.

PS: I believe we have 3 or 4 threads crossing on the same subject... ;-)

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Wed, 16 Jul 2008 10:13:03 -0500

I have re-named this thread to track it better and summarize
the discussion.

Stanislav Shalunov wrote:
> The cache discovery portion of ALTO could be equivalent to BEP 22, or it 
> could literally be the mechanism in BEP 22 and that would be quite 

Stanislav: I think we're making progress.  So, in that spirit,
let's try to flesh this out some more.

Who do you see doing cache discovery?  The ALTO server or the
individual peers?

If we take the view that a cache is a selfless peer who wants
nothing in return, then its discovery is no different than
discovering other peers.  Then this is best relegated as a
normal peer-discovery algorithm/process implemented in any
given P2P overlay.

Now, let's tackle the ALTO server doing cache discovery.  If the
ALTO server can use a standard means to discover a cache, why
can't the peer use the same means?

I believe that if cache discovery is moved to the realm of
the ALTO server, it becomes more of cache "dissemination" problem
than of cache "discovery."  In other words, the ALTO server
will be pre-configured by the ISP  -- or if it is not operated by
the ISP, will use other means -- to maintain a set of known caches.
It has been suggested that caches "register" to the ALTO server.
I think that this is probably not a good idea for a variety of
reasons.

Then, peers will query the ALTO server using a protocol yet to
be defined (let's refer to this as the TBD protocol.)  This
TBD protocol will have an extension that the peer can use to
provide the ALTO server with some hints (overlay protocol, i.e.,
BitTorrent, eDonkey; hash of the content, etc.)  The TBD
protocol will arrange it such that in a response, the querying
peer is told to contact the cache first.

With some additional work, this extension to the TBD protocol
could be used to also find other resources like VoIP relays,
and it can also be applicable for non-P2P uses like CDN.

Is this an approach that seems feasible?  If so, we can word-
smith something to this effect and add it to the draft
charter.

Thanks,

- vijay

---

From: "Stas Khirman" <[email-masked]>
Date: Wed, 16 Jul 2008 10:08:37 -0700

> Enrico Marocco
[Stas Khirman] 
As matter of fact, "application" knowledge is just a smallest part of
problem - cache discovery requires announcement of specific content request
by end-user. It will create a lot of privacy and DMCA issues and probably
will be "show-stopper" for adoption both by client application developers
and ISPs.

It is why I advocate clear separation of mechanisms for 
- peer selection optimization
- resource discovery (cache discovery as a special case)

---

From: "Stas Khirman" <[email-masked]>
Date: Wed, 16 Jul 2008 10:08:37 -0700

Hi Song,

Actually, we are in the violent agreement here : I also see a complete
equivalency between "cache selection" and "peer selection" - the problem is
not in the selection, but in discovery.

Resource discovery requires knowledge of resource type ( "BitTorrent") and
context ( content id and other) and in general case uses little or no
information about ISP topology. (I assume that application need to identify
an exact cache as cache redirection is not an option for P2P applications -
(a) not supported by today's protocols (b) creates a lot of security/privacy
issues).

Peer selection does not require knowledge about resource types and content,
but uses a lot of topology knowledge and other ISP-sensitive information.

> From: Song Haibin [mailto:[email-masked]]
> Sent: Wednesday, July 16, 2008 12:33 AM
> To: 'Stas Khirman'; 'Stanislav Shalunov'; 'Vijay K. Gurbani'
> Cc: [email-masked]
> Subject: RE: [p2pi] Charter and problem statement
> 
> Hi Stas,
> 
> I think "cache selection" would be better to describe this issue and
> should
> be included in the scope of ALTO, just like the "peer selection". If
> caches
> are deployed by the ISP (Probably true in most cases), then it meets the
> requirement that ALTO wants to expose the information that P2P application
> can't measure for traffic optimization.
> 
> Here cache is a special kind of peer, which is registered to the ALTO
> server. ALTO server must respond to peer selection queries with the
> selected
> peers and also caches. Of course, client can request resources from the
> cache that doesn't have the requested resource, as cache can fetch the
> resource from other peers/servers even other caches.
> 
> IMO, there may be kinds of caches, P2P caches, http caches and etc. I
> don't
> know if ALTO takes into account all these kinds of caches. If so, the ALTO
> server's selection. I don't know if it violates the rule of application
> agnostic here.
> 
> Best Regards!
> Song Haibin

---

From: Enrico Marocco <[email-masked]>
Date: Wed, 16 Jul 2008 19:24:16 +0200

> will be "show-stopper" for adoption both by client application developers

I don't see issues as long as disclosure of such information is optional
and up to the client. At worst, peers concerned by DMCA issues won't use
the cache discovery extension (which in any case would be useful to
peers in "controlled" P2P systems such as Pando or BBC's iPlayer). No?

---

From: "Stas Khirman" <[email-masked]>
Date: Wed, 16 Jul 2008 10:56:49 -0700

> From: Enrico Marocco [mailto:[email-masked]]
> Stas Khirman wrote:
[Stas Khirman] 
DMCA issue is rather ISP problem, not an end-user one. I'm not a layer
(happy smile), but it seems that if ISP-own equipment able to distinguish a
specific content, under specific conditions they are liable to provide
filtering and other DMCA "safe-harbor" features. It will be a case even when
a small part of clients includes content identifier into "optimization"
request (regardless if ISP has or has no installed cache servers)

My main point is that separating "optimization" and "discovery" processes we
can eliminate all those issues all together, making ALTO more flexible and
ISP less reluctant to deploy it ( or part of it).

---

From: Laird Popkin <[email-masked]>
Date: Wed, 16 Jul 2008 14:15:05 -0400 (EDT)

I agree that we should keep "cache location" and "peer location" distinct, as they have somewhat different specifics. But I think that they can both fit within the same (ALTO) framework, as they have much in common.

I agree that keeping content identity out of cache location is a good idea. That being said, cache location doesn't need to be content-specific, because one the p2p client connects to the caches, the caches can determine what they do on a content-specific basis. So ALTO could (for example) decide which cache servers to send to a user based on the user's network location and protocol (e.g. Pando caches might run on different servers from Kontiki caches or HTTP caches, etc.). But as long as ALTO can return multiple cache servers (i.e. pools) the cache servers can manage whether to cache specific content, or where to assign it, transparently to ALTO, without ALTO having to know (for cache location) what content is assigned where. This is good for the caches, as it gives them maximum flexibility, without having to implement a complex state-change mechanism between the cache servers and the ALTO server.

----- Original Message -----
From: "Stas Khirman" <[email-masked]>
To: "Enrico Marocco" <[email-masked]>
Cc: "Laird Popkin" <[email-masked]>, [email-masked]
Sent: Wednesday, July 16, 2008 1:56:49 PM (GMT-0500) America/New_York
Subject: RE: [p2pi] Charter and problem statement

[Stas Khirman]

---

From: "Stas Khirman" <[email-masked]>
Date: Wed, 16 Jul 2008 11:51:34 -0700

To summarize my points from recent threads:

"Peer selection optimization" is processes to select "optimal" subset of
peers from set of discovered peers that have specific content or part of it.
This process could/should be implemented without knowledge of specific
protocol (protocols) supported by peers and content they share, but rather
based on peers capabilities, ISP topology, utilization and other
ISP-sensitive information. 

A few important assumptions/requirements for "optimization" process:
- due necessity to deal with business sensitive information, ALTO
"optimization" process will be deployed and operated by ISPs and not by any
third-party
- in general, good optimization requires a knowledge of set of all peers to
chose from (and their capabilities/utilisation). Merging of few "optimized"
subsets may result in sub-optimal decision. That, peer set discovery process
has to be accomplished BEFORE optimization.
- Content and protocol obscurity is very important requirement for ALTO
"optimization" as it removes any concerns about end-user privacy and ISP
legal exposure, which are one of the main reasons for ISPs reluctance to
deal with P2P optimization today (P2P caching as an example) . 

So far, there are no reliable methods/protocols to satisfy "optimization"
task. I strongly believe that once offered, content/protocol agnostic ALTO
"peer selection optimization" will be quickly adopted by application
developers and ISPs.

"Resource discovery" is a process to discover [additional] peers that may be
useful in content delivery. Cache discovery is a special case of it.

A few important assumptions/requirements for "discovery" process:
- Resource discovery process does not need to use ISP-sensitive information,
such as topology, peering agreements, etc.
- Resource discovery process requires some context information, such as type
of resource, content id, etc.
- Resource discovery do not need to deliver "optimal" set of peers. As
matter of fact, it probably has no enough knowledge to do it. 
- Resource discovery and associated resources may be own and run by various
entities, not just end-user ISP. For example, group of regional ISPs may run
a shared caching service in the peering point. Resource discovery may be run
by third party unrelated to given ISP(as matter of fact, BitTorrent tracker
is a "discovery" service as well).
- "Discovery" and "optimization" services may or may not run on the same
equipment. IMHO, there are no technical needs to mix them together at all. 
- Discovery may have a number of issues with end-user privacy and ISP legal
exposure 

There are already few proprietary "discovery protocols"
(CacheLogics/BitTorrect CDP for example), but no one had been widely
adopted. Considering expressed interest, it will be beneficiary to include
open flexible "resource discovery" into ALTO scope. However, due legal
issues, some ISPs may be reluctant to deploy any kind of ALTO "discovery"
components.

Considering business, legal and technical points above, I advocate to keep
"optimization" and "discovery" as separate independent methods, protocols,
implementations, components, etc....

Stas

> Vijay K. Gurbani
> Sent: Wednesday, July 16, 2008 8:13 AM
> To: Stanislav Shalunov
> Cc: [email-masked]
> Subject: [p2pi] ALTO and caching (Was: Re: Charter and problem statement)
>

---

From: Stanislav Shalunov <[email-masked]>
Date: Wed, 16 Jul 2008 11:54:24 -0700

On Jul 16, 2008, at 10:08 AM, Stas Khirman wrote:
> It will create a lot of privacy and DMCA issues and probably

 From the client perspective, it will not tell the ISP anything it  
doesn't already know, except in encrypted mode, where it'll naturally  
be disabled.

This list has plenty of client application developers and ISPs.  There  
is no need to hypothesize what they might or might not consider a show- 
stopper or a problem at all.

-- 
Stanislav Shalunov
http://shlang.com

---

From: "Stas Khirman" <[email-masked]>
Date: Wed, 16 Jul 2008 12:07:02 -0700

> doesn't already know, except in encrypted mode, where it'll naturally
[Stas Khirman] 
I respectfully disagree - without DPI, ISPs have no knowledge about specific
content exchanged. With DPI, they have such knowledge, but can claim a
"safe-harbor" as they are not facilitating exchange (and probably restricted
to intervene due "net neutrality"). 
 
> stopper or a problem at all.
[Stas Khirman] 
It is exactly a main reason for my postings - I want to clarify this issue
in a large community of potential ALTO customers (mostly ISPs). I may be
right or wrong in my opinion, but believe in clarifying major requirements
before spending major efforts in design.

---

From: Stanislav Shalunov <[email-masked]>
Date: Wed, 16 Jul 2008 12:11:42 -0700

> Who do you see doing cache discovery?  The ALTO server or the

Peers mainly.  The ALTO server might use these same means as a  
convenience, but one can think of it as being configured with this  
information.

> normal peer-discovery algorithm/process implemented in any

Unfortunately, this won't work too well.  Consider a large BitTorrent  
swarm.  You're only going to ever unchoke a very small fraction of the  
swarm, and the cache, most probably, is not going to be in it.   
Further, the cache would need to be cued by something like DPI then to  
even know it needs to be in the swarm.  If you actively come to it  
knowing it's a very special peer, it solves both problems.

> can't the peer use the same means?

It should.

> I think that this is probably not a good idea for a variety of

I don't see why we need to standardize how ALTO servers are configured  
in this regard.  The ISP buys something that somehow makes it work.   
Maybe it uses automated discovery so you don't configured it twice.   
Maybe it's bundled with the DNS server so you don't configure it twice  
(probably not, but a theoretic possibility).  Maybe it's manually  
configured.

> peer is told to contact the cache first.

It might or might not work, depending on what an ALTO server is.  If,  
for example, an ALTO server is a URL that contains BGP dump with all  
the communities, it'll work for traffic optimization, but won't help  
you find the cache.

On the other hand, if you find the cache in the same way you find  
whatever is the ALTO server, it will definitely work, and it's easier  
to do.

> and it can also be applicable for non-P2P uses like CDN.

Yes.

-- 
Stanislav Shalunov
http://shlang.com

---

From: Laird Popkin <[email-masked]>
Date: Wed, 16 Jul 2008 15:57:58 -0400 (EDT)

A few comments:

I believe that "- due necessity to deal with business sensitive information, ALTO "optimization" process will be deployed and operated by ISPs and not by any third-party" should not be a requirement for ALTO. While some ISPs want to use proprietary information to provide guidance, and thus want to run their own P4P iTrackers (and, by analogy, future ALTO servers), other ISPs feel just as strongly that they prefer to have an independent third party (like IANA) operate between the ISPs and the P2P networks. Given this, I believe that ALTO should not mandate a specific option, but should preserve flexibility for the ISPs to choose either model.

Also, "discovery" and "optimization" are quite similar issues (because caches are p2p nodes), communicating related information between the same participants at the same time, so while a few minor details are different, they feel (to me) to be issues that should be addressed within the same framework and protocol. Given that ALTO is extensible, I'd prefer that we address cache location within ALTO than to invent two very similar protocols.

The definition of "peer selection optimization" is (IMO) incorrect, unless I am misinterpreting your terminology. Let me try to provide an example to illustrate. Consider a swarm with 10,000 peers, of which a given peer knows about 40 peers, and is connected to 10. The greatest contribution that ALTO can make is to help the p2p network determine the best 40 peers to tell a peer to connect to, because the speed and performance savings are large, because it would take a long time (perhaps 2,000 minutes) to exchange data with all 10,000 peers and determine which are best/closest. There is minimal if any value in telling a peer which of the 40 known peers is closest, because the peer can fairly rapidly (perhaps 1 minute) measure throughput with all 40 peers and determine the best/closest peers without ALTO. Given this, the assertion that "peer set discovery process has to be accomplished BEFORE optimization" is incorrect, as the ALTO optimization is used to provide the optimal peer set to be 'discovered' by a peer.

----- Original Message -----
From: "Stas Khirman" <[email-masked]>
To: "Vijay K. Gurbani" <[email-masked]>, "Stanislav Shalunov" <[email-masked]>
Cc: [email-masked]
Sent: Wednesday, July 16, 2008 2:51:34 PM (GMT-0500) America/New_York
Subject: Re: [p2pi] ALTO and caching (Was: Re: Charter and problem statement)

Stas

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Wed, 16 Jul 2008 15:08:56 -0500

> convenience, but one can think of it as being configured with this 

OK; good.  So we have peers that mainly perform cache discovery
using some protocol (more on this below.)

> in this regard.  

You misunderstood.  I was not saying that we standardize how ALTO
servers are configured (we should not go down that path, obviously.)
The point of my paragraph above was to determine if the ALTO server
is the only entity that does cache discovery, do we need to
specify a protocol at all?

So, it looks like the following statements apply (for sake of
sanity, I will take a leap of faith and assume that we have an
actual ALTO server instead of representing it a URL of a BGP dump):

Discovering caches and other resources:

   1) Peers may discover caches using a protocol X. The protocol
    X may be as simple as using well known DNS techniques to
    bootstrap the discovery process (i.e., something akin to
    DNS-SD, BEP-22, etc.)

   2) The ALTO server may be asked by the peers to discover caches
    using certain extensions to the ALTO protocol Y (Note: Y != X).
    When so asked, the ALTO server can use protocol X to discover
    those caches on behalf of the peers, and using additional
    information provided by the peer.

   3) The ALTO server may be asked by applications to discover
    other resources using certain extensions to the ALTO protocol Y.
    When so asked, the ALTO server can use protocol X to discover
    such resources on behalf of the applications, and using additional
    information provided by the application.

Yes?

Thanks,

- vijay

---

From: Stanislav Shalunov <[email-masked]>
Date: Wed, 16 Jul 2008 13:22:36 -0700

>   bootstrap the discovery process (i.e., something akin to

Yes.  This is the main and easiest case.

>   using certain extensions to the ALTO protocol Y (Note: Y != X).

Perhaps, I believe cache discovery would work fine either way, whether  
or not Y exists.  This sort of Y could produce caches, dynamic  
information about nearby peers, etc.

Yes.

>   information provided by the application.

Not sure what kind of resources, but yes, potentially.

-- 
Stanislav Shalunov
http://shlang.com

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Wed, 16 Jul 2008 15:24:09 -0500

> Not sure what kind of resources, but yes, potentially.

VoIP relays, CDN caches, etc.

- vijay

---

From: Reinaldo Penno <[email-masked]>
Date: Wed, 16 Jul 2008 13:27:28 -0700

This is fine.

---

From: "Stas Khirman" <[email-masked]>
Date: Wed, 16 Jul 2008 14:54:39 -0700

> From: Laird Popkin [mailto:[email-masked]]
> Sent: Wednesday, July 16, 2008 11:15 AM
> 
> I agree that keeping content identity out of cache location is a good
idea.

[Stas Khirman] 
Laird,

It seems that we agree on the key point - content identity in ALTO ["peer
selection"] request will be harmful. 

It will be very important to get confirmation/rejection of this assumption
from ISP's representatives.  

> That being said, cache location doesn't need to be content-specific,
> because one the p2p client connects to the caches, the caches can
> determine what they do on a content-specific basis. So ALTO could (for
> example) decide which cache servers to send to a user based on the user's
> network location and protocol (e.g. Pando caches might run on different
> servers from Kontiki caches or HTTP caches, etc.). But as long as ALTO can
> return multiple cache servers (i.e. pools) the cache servers can manage
> whether to cache specific content, or where to assign it, transparently to
> ALTO, without ALTO having to know (for cache location) what content is
> assigned where. This is good for the caches, as it gives them maximum
> flexibility, without having to implement a complex state-change mechanism
> between the cache servers and the ALTO server.
> 
[Stas Khirman] 
Lets review a few usage cases to decide if ALTO can locate optimal cache
without content identity.

1.) Cache implemented as a single virtual peer or as a set of peers with the
same cached content. Certainly, no content knowledge needed to locate a most
appropriate server. However, such architecture doesn't scale.

2.) Cache implemented as distributed set of servers where given content may
be located in some subset of nodes. If ALTO return [ordered] list of all
known servers, it will be up to client application to poll them to discover
which one has appropriate content. Sometimes it will work, but with large
list it takes significant time, especially harmful for applications that
uses cache for bootstrapped playback.

3.) Cache implemented in master-slave hierarchy. ALTO returns a pointer to a
master cache (cache director). Upon client connection, master redirects it
to a slave cache that has appropriate content. However, "master" do not have
enough knowledge to pick "ALTO optimized" slave server from set of
appropriate ( I assume that popular content may be presented in multiple
cache locations). Master can resolve this issue with a second ALTO lookup,
but it not different from separated "discovery" and "optimization" calls as
I suggested.

4.) Cache service implemented by third party CDN ( I assume you referred to
this specific case).  In this case decision what cache to be used is under
control on another party (CDN owner) who may have "optimization" policy
quite different then originating ISP (those diminishing value of ISP-own
ALTO service). Probably it is extreme speculation, but in theory, Tier-1 ISP
operated CDN may use "selection" policy that increase its own profit, but
not decrease cost of originating ISP.

---

From: "Woundy, Richard" <[email-masked]>
Date: Thu, 17 Jul 2008 22:34:48 -0400

I've been thinking about this since yesterday, and I'm not really sure I
agree with this statement.

I definitely understand that there would likely be privacy concerns if
ALTO mandated content identification in its messages. Content identities
should be an optional parameter in the ALTO message exchange (if we can
agree that it should be kept at all).

But I would think that having the content identity could help an ALTO
server make a better recommendation, at least for P2P caches.

Consider the case when the ISP is operating an ALTO server as well as a
P2P cache. If there isn't an issue of violating customer privacy, why
would it be undesirable to reveal the content identity to the ALTO
server, when it is necessary to reveal the content identity to the P2P
caching server?

In fact, the P2P caching server knows potentially more valuable
information than the content identity. ALTO would only know about
metadata for content, whereas the P2P cache would have a copy of the
actual content.

To illustrate, suppose that some content's metadata (e.g. torrent)
indicates that the file is "Spiderman" and not much more. It is possible
that the file represents a DVD rip of a Spiderman movie. It is equally
possible that the file represents a digital home video of someone's
child wearing a Spiderman costume for Halloween. Examining the actual
content (as stored on the P2P cache) would help determine which scenario
is true.

-- Rich

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
Stas Khirman
Sent: Wednesday, July 16, 2008 5:55 PM
To: 'Laird Popkin'
Cc: [email-masked]
Subject: Re: [p2pi] Charter and problem statement

idea.

[Stas Khirman] 
Laird,

It seems that we agree on the key point - content identity in ALTO
["peer
selection"] request will be harmful. 

It will be very important to get confirmation/rejection of this
assumption
from ISP's representatives.  

[Stas Khirman] 

1.) Cache implemented as a single virtual peer or as a set of peers with
the
same cached content. Certainly, no content knowledge needed to locate a
most
appropriate server. However, such architecture doesn't scale.

2.) Cache implemented as distributed set of servers where given content
may
be located in some subset of nodes. If ALTO return [ordered] list of all
known servers, it will be up to client application to poll them to
discover
which one has appropriate content. Sometimes it will work, but with
large
list it takes significant time, especially harmful for applications that
uses cache for bootstrapped playback.

3.) Cache implemented in master-slave hierarchy. ALTO returns a pointer
to a
master cache (cache director). Upon client connection, master redirects
it
to a slave cache that has appropriate content. However, "master" do not
have
enough knowledge to pick "ALTO optimized" slave server from set of
appropriate ( I assume that popular content may be presented in multiple
cache locations). Master can resolve this issue with a second ALTO
lookup,
but it not different from separated "discovery" and "optimization" calls
as
I suggested.

4.) Cache service implemented by third party CDN ( I assume you referred
to
this specific case).  In this case decision what cache to be used is
under
control on another party (CDN owner) who may have "optimization" policy
quite different then originating ISP (those diminishing value of ISP-own
ALTO service). Probably it is extreme speculation, but in theory, Tier-1
ISP
operated CDN may use "selection" policy that increase its own profit,
but
not decrease cost of originating ISP.

---

From: Laird Popkin <[email-masked]>
Date: Fri, 18 Jul 2008 06:57:17 -0400 (EDT)

I agree  with Richard - as long as content ID's are optional and abstracted (e.g. content ID's are not Torrent ID's) I don't think there's a privacy issue.

There are a number of use cases where metadata related to the content could be useful in allowing the ALTO server to make decisions. For example, an ISP might want to implement policies in their ALTO server such as:

- Don't cache consumer generated content (requires being able to differentiate between CGC and commercial content).
- Provide different guidance to different applications (requires knowledge of p2p application).
- Provide guidance based on distribution of that content (requires ID of that specific content).

It's a good point that once content is cached the ISP knows a lot about the content and its distribution. In particular, since it's participating in the p2p network it knows the IP addresses of people exchanging the content, whatever metadata is exposed about the content in the protocol (e.g. size, number of files, names). And for p2p networks that don't secure content on caches (Pando does, but some others do not) the ISP has the content on the cache server and can inspect it directly. But this is (IMO, IANAL) similar to the case of an HTTP cache or an email server - the ISP is providing a service, and privacy of use of that service is protected by policy, not by technology.

Given all of that, ALTO can certainly be applied to content that isn't cached, and it should preserve privacy for that content. :-)

----- Original Message -----
From: "Richard Woundy" <[email-masked]>
To: "Stas Khirman" <[email-masked]>, "Laird Popkin" <[email-masked]>
Cc: [email-masked]
Sent: Thursday, July 17, 2008 10:34:48 PM (GMT-0500) America/New_York
Subject: RE: [p2pi] Charter and problem statement

["peer

assumption

-- Rich

-----Original Message-----

idea.

[Stas Khirman] 
Laird,

[Stas Khirman]

---

From: "Woundy, Richard" <[email-masked]>
Date: Fri, 18 Jul 2008 18:42:14 -0400

Sounds reasonable to me -- thanks for listening, Laird. :)

Sometime in the future, we'll have to refine the list of potential ISP
policies for the ALTO server. For example, there may be some user
generated content that is cacheable by the ISP and other UGC that is
not.

-- Rich

-----Original Message-----
From: Laird Popkin [mailto:[email-masked]] 
Sent: Friday, July 18, 2008 6:57 AM
To: Woundy, Richard
Cc: [email-masked]; Stas Khirman
Subject: Re: [p2pi] Charter and problem statement

I agree  with Richard - as long as content ID's are optional and
abstracted (e.g. content ID's are not Torrent ID's) I don't think
there's a privacy issue.

There are a number of use cases where metadata related to the content
could be useful in allowing the ALTO server to make decisions. For
example, an ISP might want to implement policies in their ALTO server
such as:

- Don't cache consumer generated content (requires being able to
differentiate between CGC and commercial content).
- Provide different guidance to different applications (requires
knowledge of p2p application).
- Provide guidance based on distribution of that content (requires ID of
that specific content).

It's a good point that once content is cached the ISP knows a lot about
the content and its distribution. In particular, since it's
participating in the p2p network it knows the IP addresses of people
exchanging the content, whatever metadata is exposed about the content
in the protocol (e.g. size, number of files, names). And for p2p
networks that don't secure content on caches (Pando does, but some
others do not) the ISP has the content on the cache server and can
inspect it directly. But this is (IMO, IANAL) similar to the case of an
HTTP cache or an email server - the ISP is providing a service, and
privacy of use of that service is protected by policy, not by
technology.

Given all of that, ALTO can certainly be applied to content that isn't
cached, and it should preserve privacy for that content. :-)

----- Original Message -----
From: "Richard Woundy" <[email-masked]>
To: "Stas Khirman" <[email-masked]>, "Laird Popkin" <[email-masked]>
Cc: [email-masked]
Sent: Thursday, July 17, 2008 10:34:48 PM (GMT-0500) America/New_York
Subject: RE: [p2pi] Charter and problem statement

["peer

assumption

-- Rich

-----Original Message-----

idea.

[Stas Khirman] 
Laird,

[Stas Khirman]

---

From: "Stas Khirman" <[email-masked]>
Date: Fri, 18 Jul 2008 22:13:15 -0700

> Sent: Friday, July 18, 2008 3:57 AM
> To: Richard Woundy
> abstracted (e.g. content ID's are not Torrent ID's) I don't think there's
> a privacy issue.

[Stas Khirman] 

Laird and Richard,

Thank you for clarifying this issue - I convinced that content id does not
create additional problems in use case when ISP is using cache servers.
Indeed, if any kind of caching implemented on ISP level, they already have
privacy/DMCA questions resolved somehow. It also true for use case of P2P
CDN that use P4P architecture (ALTO server polled by pTracker ). In those
use cases, including content id in ALTO request makes no harm, and can be
quite useful for cache location and [probably] other goals (differentiated
QoS ?) 

However, I still worry about use case that seems to be more "popular" ( in
term on number of users) : due variety of [legal] reasons,  ISP does not
install caching at all, but want to use ALTO-powered transport optimization
for BitTorrent, eDonkey and similar P2P traffic (assuming that major P2P
clients adopted ALTO).

In this use case, inclusion of content id into ALTO request makes no sense
at all but opens a Pandora box of potential liabilities. Making it optional
with end-user manual configuration will not help - even if small part of
users keep it ON, it will create a sufficient reason for DMCA-like request
to filter P2P content. And it is not just hypothetical speculation - see for
example http://yro.slashdot.org/article.pl?sid=07/07/05/1512247&from=rss 
http://torrentfreak.com/isps-should-block-bittorrent-and-tpb-071226/ . 

Probably, "optionally" can be achieved if during ALTO server discovery stage
( DNS?) client receives some kind of "configuration" flag and will be
mandated NOT TO send content id. Will it pass legal scrutiny? Will be ALTO
server used by DMCA/IFPA as a proof of ISP ability to filter P2P ? I have no
idea, but assume that the better "DMCA-proof" solution is to separate
"content-sensitive" functionalities (like cache discovery) from "content
agnostic" ones.

> could be useful in allowing the ALTO server to make decisions. For
example,
> an ISP might want to implement policies in their ALTO server such as:
> 
> the content and its distribution. In particular, since it's participating
> in the p2p network it knows the IP addresses of people exchanging the
> content, whatever metadata is exposed about the content in the protocol
> (e.g. size, number of files, names). And for p2p networks that don't
> secure content on caches (Pando does, but some others do not) the ISP has
> the content on the cache server and can inspect it directly. But this is
> (IMO, IANAL) similar to the case of an HTTP cache or an email server - the
> ISP is providing a service, and privacy of use of that service is
> protected by policy, not by technology.
>

---

From: Laird Popkin <[email-masked]>
Date: Sat, 19 Jul 2008 09:04:21 -0400 (EDT)

Note that content optimization can also (potentially) be content-specific. For example, in P4P, the guidance to the p2p networks can be 'tuned' to reflect the specific distribution of content. So for both peer selection optimization and for cache location there are use cases that could utilize some form of content identifier.

Perhaps the way to phrase this is that the ALTO server MUST support operation with no content identifiers, but ALTO clients MAY send content identifiers?

----- Original Message -----
From: "Stas Khirman" <[email-masked]>
To: "Laird Popkin" <[email-masked]>, "Richard Woundy" <[email-masked]>
Cc: [email-masked]
Sent: Saturday, July 19, 2008 1:13:15 AM (GMT-0500) America/New_York
Subject: RE: [p2pi] Charter and problem statement

[Stas Khirman] 

Laird and Richard,

example,

---

From: John Leslie <[email-masked]>
Date: Sat, 19 Jul 2008 10:32:40 -0400

Laird Popkin <[email-masked]> wrote:
> 
> Note that content optimization can also (potentially) be
> content-specific. For example, in P4P, the guidance to the p2p
> networks can be 'tuned' to reflect the specific distribution of
> content.

   This is (potentially) helpful when content is sufficiently static
to flood the distribution information to all ALTO servers. This is
not the general case, and might not even be the most common case in
future scenarios.

> Perhaps the way to phrase this is that the ALTO server MUST support
> operation with no content identifiers, but ALTO clients MAY send
> content identifiers?

   I think Stas was saying that even receiving and discarding content
Identifiers could cause undesirable legal exposure to large ISPs.
In any case, I agree it could.

   I see two very different use cases for what we're discussing here.

1. Client has a list of IP addresses, and wants to know which to
   try first;

2. Client doesn't have a list of IP addresses, but has a Content
   Identifier, and wants an ordered list of IP addresses to try.

   I frankly would much prefer to develop these separately. The
second might well have reason to call upon the first; but the first
never has any reason to make use of the second. Trying to combine
both into a single protocol complicates things unnecessarily (even
before the legal complications come into play).

   Also, the two cases deal in information best known by parties
with vastly different interests in limiting distribution of that
information -- up to and including a business decision to obscure
or even falsify the information passed to certain other parties.

   The technical problems of combining both into a single protocol
look easy enough, but yield no measurable benefit. The legal and
information-protection problem are much more serious.

--
John Leslie <[email-masked]>
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: "Woundy, Richard" <[email-masked]>
Date: Sat, 19 Jul 2008 12:38:31 -0400

>In this use case, inclusion of content id into ALTO request makes no sense at all but opens a Pandora box of potential liabilities. Making it optional with end-user manual configuration will not help - even if small part of users keep it ON, it will create a sufficient reason for DMCA-like request to filter P2P content. 
 
Maybe the client could determine by out-of-band means (e.g. metadata) whether it makes sense to send the content ID in an ALTO request.
 
There is an underlying assumption here that every P2P user is going to request at least some illicit content (copyright infringing or kiddie porn). What if P2P was primarily used for "legitimate" content distribution? How would that assumption influence our requirements?
 
At the same time, I would caution against assuming that the lack of content identification in any ALTO request implies that the user is attempting something illegal. There are obviously many privacy concerns besides trying to evade one's legal system...
 
-- Rich

________________________________

From: [email-masked] on behalf of Stas Khirman
Sent: Sat 7/19/2008 1:13 AM
To: 'Laird Popkin'; Woundy, Richard
Cc: [email-masked]
Subject: Re: [p2pi] Charter and problem statement

[Stas Khirman]

Laird and Richard,

example,

---

From: Stanislav Shalunov <[email-masked]>
Date: Sat, 19 Jul 2008 12:36:21 -0700

On Jul 19, 2008, at 9:38 AM, Woundy, Richard wrote:
> Maybe the client could determine by out-of-band means (e.g.  
> metadata) whether it makes sense to send the content ID in an ALTO  
> request.

Note that, at least in BitTorrent, clients that view the network  
adversarially already have a manually triggered mechanism, encrypted  
mode, to deal with it.  Encrypted mode might well be sufficient.

>
>
> There is an underlying assumption here that every P2P user is going  
> to request at least some illicit content (copyright infringing or  
> kiddie porn). What if P2P was primarily used for "legitimate"  
> content distribution? How would that assumption influence our  
> requirements?
>
> At the same time, I would caution against assuming that the lack of  
> content identification in any ALTO request implies that the user is  
> attempting something illegal. There are obviously many privacy  
> concerns besides trying to evade one's legal system...
>
> -- Rich
>
>> there's
>> participating
>> To: "Stas Khirman" <[email-masked]>, "Laird Popkin" <[email-masked] 
>> >
>> Cc: [email-masked]
>> Behalf Of
>>> servers from Kontiki caches or HTTP caches, etc.). But as long as  
>>> ALTO
>> can

-- 
Stanislav Shalunov
http://shlang.com

---

From: "Stas Khirman" <[email-masked]>
Date: Sat, 19 Jul 2008 16:42:29 -0700

> Sent: Saturday, July 19, 2008 6:04 AM
> To: Stas Khirman
> Cc: [email-masked]; Richard Woundy
> Note that content optimization can also (potentially) be content-specific.
> For example, in P4P, the guidance to the p2p networks can be 'tuned' to
> reflect the specific distribution of content. So for both peer selection
> optimization and for cache location there are use cases that could utilize
> some form of content identifier.
> 
[Stas Khirman] 
Sorry, probably I missed your point - how content information can be used in
peer selection (traffic optimization) ? 

Let's consider a use case when ALTO has no knowledge about additional peers
(such as cache) to "enrich" list supplied by client for optimization. 

ALTO receives list of peers ( and their characteristics) from client and
should send back ordered ( and probably filtered) list of the same peers. 

What kind of information to be used in this process?

IMHO, ALTO needs to know peers characteristics (leeacher, seeder,
firewalled, etc),network topology, utilization, some traffic business rules,
but I fail to see how content information can be useful here at all....

---

From: Laird Popkin <[email-masked]>
Date: Tue, 22 Jul 2008 11:51:32 -0400 (EDT)

This is a good point. There are estimates that "legit" p2p content could rapidly grow to be much larger than "pirate" p2p content. For example, if every home on broadband watched one HD TV show a day, that would be more data than the internet moves now (or can move from any central location).

----- Original Message -----
From: "Richard Woundy" <[email-masked]>
To: "Stas Khirman" <[email-masked]>, "Laird Popkin" <[email-masked]>
Cc: [email-masked]
Sent: Saturday, July 19, 2008 12:38:31 PM (GMT-0500) America/New_York
Subject: RE: [p2pi] Charter and problem statement

 

-- Rich

[Stas Khirman]

Laird and Richard,

example,

---

From: Laird Popkin <[email-masked]>
Date: Tue, 22 Jul 2008 12:01:03 -0400 (EDT)

While not required, the ALTO server could benefit from a consistent identifier of a swarm (and thus piece of content) in a number of ways.
- Different treatment of different content (e.g. optimize some traffic, and not other traffic, or apply caches to some content and not other content).
- Build up a 'state' of a swarm so that the ALTO server can respond more rapidly than if it has to compute each response for each request.

I'll also point out that we shouldn't assume that peers (or trackers) send lists of IP's. They could (and in large swarms, should) send lists of network locations (e.g. IP prefixes, ASNs) with peer information (e.g. number of seeds and leeches) and receive back a weight set of network locations, which the p2p software would apply to specific client IP addresses. This allows for optimizations such as (for example) not needing to make an ALTO request for additional peers in the same network location, since they would all receive the same guidance.

I also think that there's a level of detail about peer state that is managed by the p2p network and need not be sent to and processed by the ALTO server. For example, Pando collects extensive profiles of peers (e.g. firewalled) and uses that to optimize which peers are connected to each other. This profiling and associated logic is constantly evolving, so it does not make sense to try to standardize and and move it into ALTO servers.

----- Original Message -----
From: "Stas Khirman" <[email-masked]>
To: "Laird Popkin" <[email-masked]>
Cc: [email-masked], "Richard Woundy" <[email-masked]>
Sent: Saturday, July 19, 2008 7:42:29 PM (GMT-0500) America/New_York
Subject: RE: [p2pi] Charter and problem statement

[Stas Khirman]

---

From: "Stas Khirman" <[email-masked]>
Date: Tue, 22 Jul 2008 09:01:59 -0700

I will not be so optimistic.... 

Just a few numbers to consider:
- From time of system deployment (2003?), Comcast sold 6 billions VoDs
views.
- iTune sold only sold 125 million TV shows and 7 million movies since its
launch (2005?)
- Mininova ( most popular BT portal) facilitated 3.3 billion torrents
downloads ( about 80% - movies and TV shows) in 12 months (May 2007-May
2008)
- ThePirateBay and IsoHunt have in total more daily visitors then Mininova
(and so - facilitated downloads)

> Sent: Tuesday, July 22, 2008 8:52 AM
> To: Richard Woundy
> This is a good point. There are estimates that "legit" p2p content could
> rapidly grow to be much larger than "pirate" p2p content. For example, if
> every home on broadband watched one HD TV show a day, that would be more
> data than the internet moves now (or can move from any central location).
> 
> >In this use case, inclusion of content id into ALTO request makes no
> sense at all but opens a Pandora box of potential liabilities. Making it
> optional with end-user manual configuration will not help - even if small
> part of users keep it ON, it will create a sufficient reason for DMCA-like
> request to filter P2P content.
> 
> Maybe the client could determine by out-of-band means (e.g. metadata)
> whether it makes sense to send the content ID in an ALTO request.
> 
> There is an underlying assumption here that every P2P user is going to
> request at least some illicit content (copyright infringing or kiddie
> porn). What if P2P was primarily used for "legitimate" content
> distribution? How would that assumption influence our requirements?
> 
> attempting something illegal. There are obviously many privacy concerns
> besides trying to evade one's legal system...
> 
> -- Rich
>

---

From: Laird Popkin <[email-masked]>
Date: Tue, 22 Jul 2008 12:05:00 -0400 (EDT)

Some content owners are optimistic that if they provide high quality content for free (ad supported) that they could not only grow the download volumes with people that don't currently download, but shift significant usage from piracy.

I'm not saying that those estimates are guaranteed to be right, just that it's possible that they could be, and that ALTO should keep that in mind when designing the protocol.

---

From: Damien Saucez <[email-masked]>
Date: Tue, 22 Jul 2008 18:22:44 +0200

Maybe it is better to discuss if ALTO has to determine legit. of content 
or not. IMHO, it is not a protocol design problem.

Damien Saucez

Laird Popkin wrote:

>>knowledge
>>>of p2p application).
>>>secure content on caches (Pando does, but some others do not) the ISP
>>>      
>>>
>>has
>>    
>>
>>>(IMO, IANAL) similar to the case of an HTTP cache or an email server -
>>>      
>>>
>>the
>>    
>>
>>>>example) decide which cache servers to send to a user based on the
>>>>        
>>>>
>>>user's
>>>      
>>>
>>>>network location and protocol (e.g. Pando caches might run on
>>>>        
>>>>
>>>different
>>>      
>>>
>>>>servers from Kontiki caches or HTTP caches, etc.). But as long as ALTO
>>>>        
>>>>
>>>can
>>>      
>>>
>>>>return multiple cache servers (i.e. pools) the cache servers can
>>>>        
>>>>
>>>manage
>>>      
>>>
>>>>whether to cache specific content, or where to assign it,
>>>>        
>>>>
>>>transparently to
>>>      
>>>
>>>>flexibility, without having to implement a complex state-change
>>>>        
>>>>
>>>mechanism
>>>      
>>>

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 22 Jul 2008 11:29:00 -0500

> or not. IMHO, it is not a protocol design problem.

Damien: ALTO will not -- and cannot -- determine the legitimacy of
the content.  This is in the draft charter (see last couple of
lines of http://alto.tilab.com/docs/charter.txt), plus this was
also discussed at the IETF/MIT workshop.

I doubt the ADs will let us progress too far if we go down the
path of enforcing content legitimacy.

Thanks,

- vijay

---

From: Nicholas Weaver <[email-masked]>
Date: Tue, 22 Jul 2008 09:33:40 -0700

On Jul 22, 2008, at 9:29 AM, Vijay K. Gurbani wrote:

> Damien Saucez wrote:
>> Maybe it is better to discuss if ALTO has to determine legit. of  
>> content or not. IMHO, it is not a protocol design problem.
>

However, can ALTO be used as an oracle to find the peers distributing  
illegitimate content, once an illegitimate content identifier has been  
discovered by the legitimate content owner?

Should this be considered a requirement in any direction (MUST  
enable?  MUST prohibit?)

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 22 Jul 2008 11:49:45 -0500

> discovered by the legitimate content owner?

Nicholas: Insofar as ALTO is a service to rank IP addresses,
it will not pay attention to content.  Insofar as ALTO is a
service to redirect peers to caches, it will pay attention to
content, but that attention is rather weak right now (i.e.,
finding the cache that may contain the hash of a resource being
searched for.)  Insofar as ALTO is a service to find the
nearest "best" resource (for some value of "best"), the content
may be null (the canonical example is that of finding a VoIP
relay.)

In the end, I don't think it is the solutions that will
constrain us as much as agreeing on the problem (i.e., the need
for this type of policing.)

> Should this be considered a requirement in any direction (MUST enable?  
> MUST prohibit?)

My read based on the workshop is that we should not consider such
policing.

Thanks,

- vijay

---

From: Reinaldo Penno <[email-masked]>
Date: Tue, 22 Jul 2008 09:51:24 -0700

Hello Nicholas,

If ALTO could be used to track illegitimate peers, then I'm not sure ALTO
will help ISPs the way they expect. And that could jeopardize the future of
ALTO.

We are here today because of illegal filesharing, let's not forget that, not
because of legal downloads, legal content viewing, etc.

Although we foresee many legitimate uses for ALTO, the amount of illegal
filesharing dwarfs any other legal use of P2P.

I would suggest ALTO to stay away from any content related activities as a
IETF WG.  

Thanks,

Reinaldo

---

From: Enrico Marocco <[email-masked]>
Date: Tue, 22 Jul 2008 19:04:05 +0200

> Should this be considered a requirement in any direction (MUST

I think this is a reasonable question. Plagiarizing the requirements
draft, my answer would be that it MUST allow users not to disclose the
content being exchange.

---

From: Nicholas Weaver <[email-masked]>
Date: Tue, 22 Jul 2008 10:50:09 -0700

> draft, my answer would be that it MUST allow users not to disclose the

There may be an inherent conflict here, however:

If the user doesn't disclose a content/group/swarm identifier, then  
other peers can't find the user who's interested in the same content  
identifier, which may reduce effectiveness considerably.

If the user does disclose a content/group/swarm identifier, then other  
peers can find the user who's interested in the same content  
identifier, which may enable a large information leakage.

If some ISP in cooperating with the MP/RIAA is running an ALTO server,  
even if the clients don't connect directly, the trackers which query  
it (or arbitrary peers, in a trackerless system) may be able to glean  
interesting information based on the incoming query stream.

Additionally, the information leakage's danger can really depend on  
the nature of the content:

For both pirated material and legal pornography, the results are at  
least embarrassing, if not legally dangerous.

For legitimate, nonpornographic content, the results of an information  
leak are far less severe, but still could be significant.  (EG, could  
ALTO be used for "Nielson for IP-TV?"  Would this be good or bad?)

---

From: Reinaldo Penno <[email-masked]>
Date: Tue, 22 Jul 2008 10:54:21 -0700

This thread made the think that we need more security wording on the
charter. 

I see ALTO maybe requiring a security document considerations of its own
besides the security section on each document.

There is clearly an expectation of privacy from the P2P client. There is
clearly an expectation from the ISP that he is not aiding (conscientiously)
illegal file sharing, amongst others.

Thanks,

Reinaldo

---

From: Nicholas Weaver <[email-masked]>
Date: Tue, 22 Jul 2008 11:02:38 -0700

> illegal file sharing, amongst others.

Actually, the expectation of privacy from a P2P client might be  
considered illusionary.

The whole point of P2P is you need to be able to discover peers, so  
any attacker who is authorized to participate in the P2P network (eg,  
able to get a Content-identifier from the tracker and therefore  
authorization to participate in the swarm) should be able to map at  
least part of the P2P network and, with sybils, generate a complete map.

Thus, for access to an ALTO server, the requirement should be "get NO  
more information than you could obtain otherwise as a participant in  
the P2P network", which is a huge amount, but generally safe.  ("If  
you know the content/network/swarm ID, you can get the peer list,  
because you need to be authorized already to know this identifier").

The interesting question, however, is can an ALTO node, which ISN'T  
necessarily authorized to participate in a swarm, gain information on  
a swarm based on both queries to it, and also use any transactional  
information it gains to contact other ALTO servers to gain information  
about the swarm.

EG, it gets a content identifier based on a request, and then queries  
other ALTO servers to find out who else is participating in this  
content identifier.

In the end, it may be necessary to write requirements on information  
leaking that specifically fall one-way or the other, eg, "There is NO  
expectation of privacy because of X, Y, Z", or "Because of the  
client's expectation of privacy, when such is enabled, ALTO can't do  
A, B, C".

---

From: Arvind Krishnamurthy <[email-masked]>
Date: Tue, 22 Jul 2008 11:20:36 -0700

This is a rather interesting discussion.  Just wanted to draw attention 
to the design proposed by p4p (another traffic localizing system in the 
spirit of ALTO).

Note that a system might have to satisfy both: a) p2p user privacy and 
b) ISP privacy.

So in this regard, p4p requires that the p2p swarm information and the 
ISP traffic/topology information are maintained by two different 
entities -- one owned by the p2p system and one owned by the ISP -- and 
that information flow between the two entities operate at somewhat 
coarse granularities.

For instance, when the p2p system needs some ISP-specific information, 
such as traffic conditions between IPs x and y, it first maps them to 
their corresponding PoPs (say New York and Los Angeles), and then 
queries the ISP's server for the current conditions between the two PoPs.

Similarly, when the ISP needs to export information on topology and ISP 
preferences, it publishes them as a full mesh, thereby somewhat limiting 
the ability to reverse-engineer ISP's objectives.

This is of course just a first and simple attempt at limiting 
information leakage, and coming up with more principled 
privacy-preserving approaches would be an interesting research direction.

---

From: Reinaldo Penno <[email-masked]>
Date: Tue, 22 Jul 2008 11:14:24 -0700

Hello Nicholas,

Inline..

I agree.

I see two modes of operation. One is where no content-identifiers are
exchanged. The client sends a list of IP address/network locations/etc and
ask the ALTO server which are the best. The query could contain IP
addresses/locations spawning many different torrents/files. All the client
is interested are best peers for whatever file.

The other is where the client sends a info-hash or something similar and ask
which peers are available/best. In this case for the ALTO server to respond
it needs to have a mapping of info-has/content id to peers.

> other ALTO servers to find out who else is participating in this

Today people can use the "scrape" method and get such info from Torrent
trackers. That's why there are private trackers and others. But when this
information is potentially moved to an ISP/Content provider the semantics
and impact of such information changes.

> client's expectation of privacy, when such is enabled, ALTO can't do

I would agree such wording is necessary.

---

From: "Damien Saucez" <[email-masked]>
Date: Tue, 22 Jul 2008 20:56:13 +0200 (CEST)

> also discussed at the IETF/MIT workshop.
Yes, it is exactly what I said: it is not a protocol design problem.
Damien Saucez

---

From: "Damien Saucez" <[email-masked]>
Date: Tue, 22 Jul 2008 21:22:22 +0200 (CEST)

Arvind,

What you propose can be done with a solution able to work with any kind of
"identifier/locator". If the protocol is not limited to a specific kind of
entity identification, you can tackle the problem by creating a ranking
hierarchy: a first engine ranks P2P IDs based on P2P-related criteria. To
refine the ranking, IDs can be translated into IP addresses sets and, a
second engine, ranks these sets based on ISP-related criteria. This can be
done with IDIPS but we'll discuss technical solutions later.

The way the engines perform the ranking is layer dependent but the
information dissemination can remain the same (using something like
address families?).

The key point of this discussion is that the ALTO protocol must be able to
deal with any kind of "identifier/locator" (at least optionally) as stated
by Stefano.

Damien Saucez

---

From: stefano previdi <[email-masked]>
Date: Wed, 23 Jul 2008 10:09:18 +0200

> From: Damien Saucez <[email-masked]>
> 
> Arvind,
> 

yes. As written in the requirements draft, IP addresses should be one option
for the ALTO ranking system.

Working with application identifiers is fine too but so far the only piece
of standardized information we can rely on are IP addresses (from a protocol
specification perspective I don't know how stable will be an application ID
encoding format).

Some concerns have been raised in terms of confidentiality that would need
to be disclosed (if client has to send into an ALTO request a list of IP
addresses) however, I believe SPs do already know how to figure out who's
doing what.

So, for me, IP addresses in ALTO req/res guarantee the transparency of the
protocol behavior when faced to a diversity of applications.

Thanks.

s.

---

From: songhaibin 64081 <[email-masked]>
Date: Mon, 28 Jul 2008 15:40:43 +0800

I agree that the legality of the contents should not be considered here.

-Song Haibin

> enable?