Subject: Re: [p2pi] Content Identifiers sent to ALTO server?

From: Laird Popkin <[email-masked]>
Date: Mon, 21 Jul 2008 20:36:13 -0400 (EDT)

There are misunderstandings of P4P (my fault - I've not explained it in much detail on this list). Perhaps we should discuss in front of a white board? Will you be at IETF Dublin, Stas?

Comments below... (until we get to a whiteboard)

- Laird Popkin, CTO, Pando Networks
  mobile: 646/465-0570

----- Original Message -----
From: "Stas Khirman" <[email-masked]>
To: "Laird Popkin" <[email-masked]>, "John Leslie" <[email-masked]>
Cc: [email-masked]
Sent: Monday, July 21, 2008 6:48:34 PM (GMT-0500) America/New_York
Subject: RE: Content Identifiers sent to ALTO server?

> -----Original Message-----
> From: Laird Popkin [mailto:[email-masked]]
> Sent: Monday, July 21, 2008 7:29 AM
> To: John Leslie
> Cc: Stas Khirman; [email-masked]
> Subject: Re: Content Identifiers sent to ALTO server?
> 
> I don't think that either of the scenarios that you list is particularly
> likely for p2p swarm file downloading. I'll explain why, they outline the
> model of ALTO that seems most valuable for that application.
> 
> 1. If the client has a list of IP addresses, the list is short enough that
> it can test throughput with each of them all fairly quickly, so the value
> of ALTO is low in terms of optimizing p2p performance. This scenario makes
> sense for some other applications (e.g. finding a good network relay, or
> optimizing small swarm downloads).

[Stas Khirman] 
Peer list is not necessary short - it is rather current BitTorrent popular
trackers configurable restriction. In Gnutella you can get a few hundreds of
peers on the first lookup. In eMule, it restricted (IMHO) by 300 IPs per
server lookup ( you can request from multiple servers simultaneously). Plus,
peers can arrive via PEX and DHT.

BitTorrent specific limitation of 50-100 peers in the announce response is
configurable and easy to change. Beside, majority of BitTorrent
implementations can work without tracker at all - via DHT .

[Laird] I mean 'short' in relative terms. All p2p system have p2p clients that know about a 'small' number of peers out of a 'large' population of potential peers, such that optimizing the selection of the 'small' number of peers out of the 'large' population can improve the p2p network's performance. It's also the case that if the 'small' number of new peers known to a given peer is sufficiently large, then there's also some value in optimizing the sequence of peer connections within the 'small list, though that's less of a win than optimizing the selection from the 'large' list.

For example, consider a peer in a swarm with 10,000 peers. If the peer can find out about all 10,000 peers instantly, it still needs to connect to and exchange data with every peer in the swarm to measure throughput to pick the best data sources. If it can test one peer every second, it'll take 10,000 seconds to test them all to determine the best data source. An ALTO server can tell me which are the closest peers instantly, giving me a much better start point. That's not to say that the ALTO server can't be helpful in other ways. For example, it's also great if a peer that knows about 300 peers can have the ALTO server tell it which are the closest of the 300. But if those 300 are the closest out of the 10,000 that's even better, right?[/Laird]

> 2) I think that it's safe to assume that p2p networks will never send a
> list of all IP addresses associated with a particular content swarm to an
> ISP's ALTO server. This raises obvious business, privacy and legal issues,
> which are at best quite concerning.

[Stas Khirman] 
If peer list is NOT associated with specific content, I see no reasons for
business, privacy and legal concerns. Please elaborate if I wrong. 

[Laird] I think that we're in agreement as far as Pando goes, as we're pretty open with ISPs, but I'm not sure that assumption is true for other p2p networks. There's a lot of sensitivity in the p2p community around exposing too much information about their networks and usage, just as there's a lot of sensitivity in the ISP community around exposing too much information about their network. So perhaps I should rephrase my claim slightly: I think that it's safe to assume that not all p2p networks will agree to send a list of all IP addresses associated with a particular content swarm to an ISP's ALTO server.[/Laird]
 
> In addition, putting the ALTO protocol in the middle of a peer request
> creates unacceptable operational problems for the p2p network that would
> be far worse than any optimization that ALTO could provide. Let's compare
> a standard BitTorrent "tracker announce", a tracker announce with ALTO IP
> lookup" and "tracker announce with ALTO guidance" for a peer in a swarm of
> 10,000 peers.
> 

[Stas Khirman] 
If I understand correctly, you are describing P4P architecture where tracker
cooperates with ISPs. It is certainly preferable model for P2P CDN where
content distributed by the same entity that operate tracker (or associated
third-party). This model also assumes that beside tracker, peer client has
no other independent ways to discover new peers (like DHT and PEX) 

[Laird] That's not how P4P works.

- P4P doesn't rely on P2P CDNs, or that the content is distributed by the same entity that operates the tracker.
- P4P does not assume a tracker, or the absence of other peer discovery methods. P4P provides "good" starting peers, so that transfers are fast/cheap, but if additional peers are discovered over time, traffic will be optimized (for most p2p networks) so that if the new peers are better than the ones provided by ISP guidance, they'll be used instead. On average, however, the peers provided by ISP guidance turn out to be much better than random peers.
[/Laird]

However, I don't see how this architecture may work in "open P2P"
applications, which generate majority of traffic today. 

The main reasons are:
- some P2Ps have no tracker at all - see Gnutella, Ares and other

[Laird] P4P doesn't rely on a tracker. It's easier to coordinate peer assignment in a tracker-based p2p network, so that's where the first P4P tests are being performed, but the same thing can (we think) be done with a trackerless p2p network. Since network intelligence is distributed in trackerless p2p networks, it's a bit more complex to figure out the proper integration, but I personally think that it'd be very interesting to distribute network guidance via DHT's, so that peers can serve as ALTO servers (of a sort) to optimize network traffic.[/Laird]

- All modern P2P clients able to discover new peers without direct tracker
involvement.

[Laird] Of course. This is assumed in P4P. Ongoing peer discovery and traffic optimization will be good for the p2p network. If P4P provides good starting peers, that's good for the p2p network. If a peer starts with good, nearby peers, then later finds a bunch of 'random' peers, that's fine - if the good, nearby peers are faster, then the p2p network will continue to use them. If the new, further away peers are faster, we want to start using them. Luckily it's turning out that nearby peers are (on average) very good data sources, so P4P tests are showing great speedups.[/Laird]

- Majority of popular open trackers operated by entities that will not
cooperate with ISPs (see ThePirateBay)

[Laird] There might be an issue here no matter how the data is exchanged. That is, there may be (IANAL) a liability for an ISP if they provide guidance to a 'bad' tracker, or to providing guidance to users who are using a 'bad' tracker.[/Laird]

- Considering BitTorrent, there are only a few major application vendors who
will probably cooperate with ALTO. Number of deployed trackers
implementations and deployments - in 100s. It will take a lot of time until
tracker owners cooperate with ALTO ( if ever)

[Laird] ALTO is not the same thing as trackerless peer guidance, and P4P is not the same thing as tracker-based peer guidance. I'm hoping that we can find a way to express P4P (or whatever P4P evolves into over time) in ALTO.[/Laird]

Probably, ALTO has to consider ( but not limit to) two major architectural
models:
- cooperative tracker (P4P architecture)
- cooperative client  ("oracle" architecture)

[Laird] I agree that ALTO needs to support both tracker-based and trackerless p2p networks (and it's in the ALTO requirements...). To make sure that there isn't any confusion on this point, P4P isn't specifically tracker based, and ALTO isn't specifically client based. My hope is that the relationship between the two is that P4P is an approach to p2p network optimization that can be expressed using the ALTO protocol.
[/Laird]

> standard: peer announces, Tracker responds from list in memory (one MTU
> each way, UDP). Fast, reliable in-memory operation.
> 
> ALTO IP list: In this model, information sent into or out of ALTO is lists
> of IP addresses. Peer announces. Tracker sends request, with the peer
> address and a complete list of the 10,000 peers in the swarm, to the ALTO
> server. ALTO server computes optimal list and returns 50 peers. Tracker
> returns list to peer. Tracker does this (send swarm peer list, receive
> ALTO peer list) 2,000 times per second. This makes the p2p tracker slow
> and unreliable (by adding external communication to what is now an in-
> memory operation), and places a huge operational burden on the ISP's ALTO
> server. I suspect that ISPs would have problems with this as well, because
> it would effectly make their ALTO server a p2p tracker, with all that
> implies.
> 
> ALTO guidance: Tracker (updated periodically, asynchronously) receives
> optimization rules from the ALTO server. Peer announces. Tracker picks
> optimal list from known swarm peers based on the ISP's rules, and returns
> 50 peers to announcing peer. This is slightly slower than a standard
> announce, but it's all in memory in the Tracker, and only imposes a low
> volume of ALTO communications. Communication is the same as the first case
> (one MTU each way, UDP), plus one ALTO message every hour (or day, etc.)
> to update the optimization rules.
> 
> - Laird Popkin, CTO, Pando Networks
>   mobile: 646/465-0570
> 
> ----- Original Message -----
> From: "John Leslie" <[email-masked]>
> To: "Laird Popkin" <[email-masked]>
> Cc: "Stas Khirman" <[email-masked]>, [email-masked]
> Sent: Saturday, July 19, 2008 10:32:40 AM (GMT-0500) America/New_York
> Subject: Content Identifiers sent to ALTO server?
> 
> Laird Popkin <[email-masked]> wrote:
> >
> > Note that content optimization can also (potentially) be
> > content-specific. For example, in P4P, the guidance to the p2p
> > networks can be 'tuned' to reflect the specific distribution of
> > content.
> 
>    This is (potentially) helpful when content is sufficiently static
> to flood the distribution information to all ALTO servers. This is
> not the general case, and might not even be the most common case in
> future scenarios.
> 
> > Perhaps the way to phrase this is that the ALTO server MUST support
> > operation with no content identifiers, but ALTO clients MAY send
> > content identifiers?
> 
>    I think Stas was saying that even receiving and discarding content
> Identifiers could cause undesirable legal exposure to large ISPs.
> In any case, I agree it could.
> 
>    I see two very different use cases for what we're discussing here.
> 
> 1. Client has a list of IP addresses, and wants to know which to
>    try first;
> 
> 2. Client doesn't have a list of IP addresses, but has a Content
>    Identifier, and wants an ordered list of IP addresses to try.
> 
>    I frankly would much prefer to develop these separately. The
> second might well have reason to call upon the first; but the first
> never has any reason to make use of the second. Trying to combine
> both into a single protocol complicates things unnecessarily (even
> before the legal complications come into play).
> 
>    Also, the two cases deal in information best known by parties
> with vastly different interests in limiting distribution of that
> information -- up to and including a business decision to obscure
> or even falsify the information passed to certain other parties.
> 
>    The technical problems of combining both into a single protocol
> look easy enough, but yield no measurable benefit. The legal and
> information-protection problem are much more serious.
> 
> --
> John Leslie <[email-masked]>

_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Thu, 24 Jul 2008 09:56:14 -0500

Laird Popkin wrote:
> There are misunderstandings of P4P (my fault - I've not explained it
> in much detail on this list). Perhaps we should discuss in front of a
> white board? Will you be at IETF Dublin, Stas?

Laird: That actually may not be a bad idea.

How about if we do this: pending AD approval, we can organize a
night session *outside* the ALTO BoF to discuss individual
solutions like P4P, IDIPS, etc.

What can we hope to accomplish from this?  Well, the biggest bang
for the buck, in your case, is that you can provide some distilled
information from the P4P experience on what ALTO should and should
not do.  What information has been helpful for P4P that ALTO
should try to incorporate and what information should be
abstracted out.  The other benefit is that all of us will get up
to speed on the existing state of the art in these systems.

So ... Enrico and I will talk to the ADs in Dublin and if they
approve, we'll arrange for a night session on Wednesday.  Further
logistical details will be provided during the normal BoF
session on Tuesday.

Those besides Laird who are interested in presenting a specific
solution should contact Enrico and me.  Please remember that the
intent of these talks will be what can we learn from them that
may prove beneficial while designing ALTO.

Thank you.

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
2701 Lucent Lane, Rm. 9F-546, Lisle, Illinois 60532 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
WWW:   http://www.alcatel-lucent.com/bell-labs
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: Stanislav Shalunov <[email-masked]>
Date: Thu, 24 Jul 2008 15:17:08 -0700

If this is an informal meeting outside of the ALTO BoF, it should not  
require anyone's approval.  I don't know if freedom of assembly is  
constitutionally guaranteed in Ireland, but I'm willing to risk it.

If it were the first ALTO WG session, then it'd up to the ADs and the  
WG chairs, but it's obviously too early for that.

Why don't we set up a time and get together in the bar or an otherwise  
empty room?  Thursday night is best for me, followed by Tuesday and  
Wednesday.  Monday doesn't work (and is before the ALTO BoF anyway).

-- Stas

--
Stanislav Shalunov

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Thu, 24 Jul 2008 22:34:04 -0500

Stanislav Shalunov wrote:
> require anyone's approval.  

Stanislav: You're probably right, but it does not hurt to socialize
it with the ADs as I am not sure what the exact expectations are.

> Wednesday.  Monday doesn't work (and is before the ALTO BoF anyway).

I can't Thu night.  Tue and Wed is okay.  Probably an empty
room is best in case someone wants to show slides or use a white-
board.

Thanks,

- vijay