Subject: Re: [p2pi] why I didn't hum yes (yet)

From: Bob Briscoe <[email-masked]>
Date: Sat, 02 Aug 2008 17:18:00 +0100

Phil,

See end...

At 02:14 01/08/2008, Philip Levis wrote:

>On Jul 31, 2008, at 6:54 PM, Bob Briscoe wrote:
>
>>Laird,
>>
>>At 13:45 31/07/2008, Laird Popkin wrote:
>>>Addressing remark 1: localizing network traffic (based on the p4p
>>>field tests, which used ISP network maps and an "oracle") shows a
>>>strong benefit in transit reduction and in performance improvement,
>>>for networks that are DSL, Cable or FTTH.
>>
>>I think my rather important previous point got missed (sorry my mail
>>had a lot of points)...
>>
>>Neither of these metrics for 'better' impresses me much at all.
>>
>>* If performance of p2p is better, and if there is shared congestion
>>with other non-p2p users on the same local network, this means the
>>performance of others will most likely be worse. Overall, is this
>>better or worse?
>>
>>* If there's transit reduction, equivalent traffic will instead be
>>being uploaded from the local network. Again, if the local network
>>already has shared congestion upstream (a cable network does),
>>that's making it _worse_ both for local p2p & non-p2p users.
>>
>>
>>It costs more (bit for bit) to upgrade access and backhaul than it
>>does to upgrade interconnect (there's a considerable economy of
>>scale). If someone from a cablecco has told you it's most important
>>to reduce transit costs, you'll find they did NOT mean "and at the
>>same time please introduce higher backhaul and access costs instead".
>>
>>See the picture I posted yesterday to this list: 
>><http://www.cs.ucl.ac.uk/staff/B.Briscoe/presents/0807ietf/0807alto-briscoe.pdf  
>> >
>>
>>The Internetwork is provided to be used. Let's not assume a
>>DISINTERNET is 'better' without being clear what 'better' means.
>>
>>I've seen no research that uses a full cost-benefit analysis. That's
>>hard (e.g. what's the relative value of p2p bits versus non-p2p
>>bits). But without such a study, unfortunately, what everyone is
>>calling 'better' looks like 'worse' to me on second looks - at least
>>for networks with a congested upstream.
>>
>>On third looks (ie a full study) it might not be worse. But can
>>people please enter this debate on the assumption that network
>>economics is dangerous in the hands of amateurs (which includes the
>>large majority of folks even in telcos/cablecos).
>
>
>Bob, I think you're getting lost in the weeds here.
>
>The goal is to define a mechanism by which entities with a greater
>knowledge of the network can provide guidance to clients on whom those
>clients might want to communicate with.

I'm well aware of this and not getting lost in the weeds. I'm trying 
to me very clear: I'm saying: If it were just a localisation problem 
it would be easy to standardise. As it's not proven that it's a 
simple localisation problem, this is probably still research.

>This guidance could be based on localization, or it could be based on
>other properties which turn out to be better but we have not
>discovered yet. The interface proposed, (X,Y,Z) -> (Z), does not
>assume localization. Doing so would be folly, as it would preclude
>using the mechanism for applications which are better served by other
>selection policies.

As I said in my mail, many P4P folks know there's more to it than 
localisation and I was happy they emphasised this at the p2pi 
workshop. But in ALTO, a different bunch of people seemed to have 
taken over the asylum, all of whom said "localisation is the goal". 
To the uninitiated, this makes it look like it's a simple task, and 
ready for standardisation.

So far, most experimentation with oracles has made localisation the 
goal then proved (unsurprisingly) they've achieved more localisation. 
They often say that in practice policies and congestion might need to 
be taken into account, and that their framework allows for these 
extensions. But unsurprisingly no-one's fully taken that step. 
Because that's the really hard bit - particularly congestion, because 
the endpoints know more than the network.

To be clear, I'm not saying do nothing until it's perfect. I'm saying 
we can't judge whether it's even 'better' while the research makes 
claims of 'better' against the wrong metric.

>That being said, the points in your slides are interesting. Do you
>have any real and repeatable experimental evidence that it's "worse?"

No; I'm not doing research on oracles. I was just acting as a 
critical research reviewer. The experimental evidence being quoted 
from trials and simulations seemed to be using bogus metrics for 
claims of 'better'.
         less transit            != better (not unconditionally)
         better p2p performance  != better (not necessarily)

>I'll admit that none of the experimental studies cover every single
>edge case -- they would then be so voluminous as to be unreadable --
>but I'm tempted to trust experimental data collected from real
>networks over back-of-the-envelope calculations.

Just because I gave a very common scenario where localisation is 
unlikely to work, doesn't mean I'm asking researchers to exhaustively 
cover all scenarios. I'm asking them to use the right metric for 
'better', whatever representative scenarios they choose.

Bob

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196  

_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: Bob Briscoe <[email-masked]>
Date: Sat, 02 Aug 2008 16:51:25 +0100

Rich,

Thanks - this data is getting more useful. More inline...

At 09:25 01/08/2008, Woundy, Richard wrote:
>Bob,
>
>I feel compelled to respond to some of your points.
>
>At Comcast, we studied the P4P results from Verizon earlier this 
>year, before we committed to a trial this summer. There were three 
>items of interest from those results:
>
>1.
>         Faster downloads for P2P users
>2.
>         Reduced backbone transit traffic for the ISP
>3.
>         * The same aggregate amount of uploading by ISP customers

Do you mean uploading by p2p apps that P4P has helped shift, or 
uploading by all apps including other p2p systems?

If you mean not just P4P apps but all apps, getting no increase in 
uploading could be an effect of the performance probing of non-P4P 
apps finding that Comcast's upstream is getting congested so they 
naturally choose peers in other networks to download from instead of 
yours (as you say below).

In that case, P4P can indeed be judged as 'better'. Essentially it 
would be initiating a shuffle around of /all/ the transit flows so 
more were local, not just it's own.

I hope I've made clear I'm not saying I believe P4P is wrong, I'm 
just asking for rigor in the claims of 'better'.

>We're not ready to reveal the results of our own recently-concluded 
>P4P trial, but suffice it to say that we are seeing similar results. 
>I suspect the reason behind point #3 is that P2P uploads within the 
>ISP displace the P2P uploads outside the ISP. And the reason behind 
>point #2 may be that there are fewer outside factors (e.g. multiple 
>transit backbones) that could cause congestion.
>
>We need to perform more analysis before we can conclude that ISP 
>localization doesn't increase upstream traffic, but at least on 
>average it does not increase upstream traffic. If it turns out that 
>P4P localization is cost-neutral on an access basis, then that would 
>be a critical item to retain for ALTO.

Certainly.

And I'm glad you're not jumping to conclusions.

>BTW, I agree with many of your other comments. For example, I agree 
>that minimizing RTT for P2P connections isn't all that critical. For 
>example, I've heard that clever P2P users are often motivated (where 
>possible) to select peers in Japan and Korea, because higher upload 
>throughput trumps RTT every time.

Just to be clear, I was saying RTT /will/ become much less critical 
over the next year or so as these new congestion controls become 
widespread. That drastically cuts the incentive for p2p users to 
voluntarily ask an oracle (losing your point 1: 'faster downloads for 
p2p users').

>I've been talking to Laird, and I seem to be from a 'cableco', so 
>maybe you're referring to me. (And maybe I'm one of the amateurs as well.) :^/

:)

>
>But I am pretty sure I have been consistent on this point, from the 
>P2PI Workshop to the ALTO BOF to other private conversations -- our 
>broadband access costs are much higher than our backbone / transit 
>costs. As my boss would say, "backhoe costs don't follow Moore's 
>Law". For Comcast, minimizing transit costs is nice but not the 
>fundamental concern.

Everyone on the list, please take this to heart.

>
>BTW, I have heard that for other ISPs, transit costs *are* a 
>fundamental issue. And not just in the so-called "third world".

Certainly, Telecom New Zealand say this and our friends in Oz.

Bob

>-- Rich
>already has shared congestion upstream (a cable network does), that's
>making it _worse_ both for local p2p & non-p2p users.
>
>
><http://www.cs.ucl.ac.uk/staff/B.Briscoe/presents/0807ietf/0807alto-briscoe.pdf>
>

---

From: Bob Briscoe <[email-masked]>
Date: Sat, 02 Aug 2008 17:05:10 +0100

Laird,

This sounds good.

Bob

At 09:31 01/08/2008, Laird Popkin wrote:
>To add a little field test data, so far we've not seen any cases 
>where traffic localization increased "last mile" uplink consumption. 
>When traffic isn't localized, an ISP's customers are uploading to 
>the general internet. When traffic is localized, an ISP's customers 
>upload about the same volume, but the destination of the upload 
>shifts from external to internal destinations. This happens in part 
>because the ISP's customers connect to each other, and in part 
>because the external users connect to each other instead of 
>connecting into the ISP. The result is that the consumption of the 
>"last mile" doesn't change, but the consumption of internal and 
>external links drops.
>
>You're right that a cost analysis is complex, since it varies 
>dramatically between ISPs (and is subject to debate within ISPs). 
>That's why I'd rather talk about resource consumption (e.g. data 
>volumes, link utilization).
>
>- Laird Popkin, CTO, Pando Networks
>   mobile: 646/465-0570
>
>----- Original Message -----
>From: "Philip Levis" <[email-masked]>
>To: "Bob Briscoe" <[email-masked]>
>Cc: "Laird Popkin" <[email-masked]>, [email-masked]
>Sent: 01 August 2008 02:14:25 o'clock (GMT) Europe/London
>Subject: Re: [p2pi] why I didn't hum yes (yet)
>
>
>Phil

---

From: "Woundy, Richard" <[email-masked]>
Date: Sat, 2 Aug 2008 18:04:03 -0400

>Do you mean uploading by p2p apps that P4P has helped shift, or uploading by all apps including other p2p systems?
 
I meant only P2P uploading that has been influenced by P4P, which is a very small proportion of our current P2P traffic. That's because we have only a handful of cooperating P2P trackers (from one source) in the current P4P trial, in a world with tens of thousands of trackers (from many different sources).
 
>I hope I've made clear I'm not saying I believe P4P is wrong, I'm just asking for rigor in the claims of 'better'.
 
That's reasonable.
 
Incidentally, I was just reading some comments from journalist Thomas Friedman (New York Times) that seem appropriate to quote here: "A lot of journalists have lost the distinction between skepticism and cynicism. A skeptic says, 'I don't know. I'm going to find out.' A cynic says, 'I already know. I don't need to find out.'"
 
So I appreciate your skepticism on this topic. :)
 
-- Rich

________________________________

From: [email-masked] on behalf of Bob Briscoe
Sent: Sat 8/2/2008 11:51 AM
To: Woundy, Richard
Cc: [email-masked]
Subject: Re: [p2pi] why I didn't hum yes (yet)

Rich,

Certainly.

:)

Bob

---

From: Henning Schulzrinne <[email-masked]>
Date: Sun, 3 Aug 2008 12:16:31 -0400

I'm changing the subject line, as the previous one has become somewhat  
uninformative.

> fundamental issue. And not just in the so-called "third world".

There are many types of "ISPs", to it's hard to generalize. One class  
of ISPs that definitely falls into this category are large  
universities, a favorite location of P2P activity, both upload and  
download. Universities can often relatively easily upgrade their own  
internal networks, but we pay real $/Mb for commercial Internet  
access. As I mentioned, costs are essentially four-tiered: "free/ 
unbounded" for the same large Ethernet switch (say, a dorm), "free/but  
bounded capacity" inside campus, "cheap/plenty of capacity" to  
Internet2 universities and "expensive/bottlenecked" to the commercial  
Internet. I suspect that many large, multi-site corporations fall into  
roughly similar divisions, although they typically don't serve upload  
traffic outside their firewalls.

There are also lots of issues of capital costs (upgrading access and  
backhaul) vs. operational expenses (such as peering/bandwidth  
charges). Some costs are essentially linear with (monthly) usage,  
others only occur when a capacity upgrade is needed.

As long as we understand the distance in the term "localization" to be  
measured in dollars (or your local more-valuable, currency) and/or  
download bandwidth, not miles, we don't really have to play the "my  
ISP model is better than your ISP model" game.

In some cases, these trade-offs are only known to the ISP, so if a  
third party runs an information service, it will have to extrapolate  
from typical conditions.

Henning