Subject: Re: [gaia] Heterogeneity in network capacity: growing?

From: Jon Crowcroft <[email-masked]>
Date: Tue, 01 Mar 2016 09:17:12 +0000

so there's all sorts of weird new access tech, like there's two zigbee
nets in my house alone, plus community mesh - what do people measure
on those? Most the hard work in last few years has been on
pay-per-use nets or broadband tech of the legacy (adsl, cable modem,
and cellular data)  - but i guess the question here is what are the
weird and wonderful lashups occuring out of cheap & cheerful, as well
as the ever onwards march to fiber to the eye+ear, no?

> > On 29. feb. 2016, at 10.11, Manner Jukka <[email-masked]> wrote:
> >
> > Hi,
> >
> > I don’t have a published analysis to cite, have done similar 
> comparisons many times in various presentations. I could run that 
> distribution from the Netradar database for 2014 and 2015, just need to 
> give me the X buckets where to allocate the samples.
> 
> NetRadar data would definitely be nice as a data point!  About buckets, 
> not sure I get it: I mean e.g. a CDF of bandwidth values that were seen 
> overall in 2014 and 2015; that would show if there is a shift not only 
> towards higher bandwidths but if the diversity als increases, I guess.
> 
> If you need to have buckets to query, I guess it can be any reasonable 
> low / high bandwidth values… the question is, do the low bandwidths 
> disappear as quickly as the higher ones pop up?
> 
> 
> > Market analysis is a bit trickier, since measurements need to be mapped 
> to a region somehow. IP address is one option if it can be trusted enough.
> 
> *I* don’t actually care about markets much. I just wonder what range of 
> bandwidths network protocols need to be able to operate over… is it 
> always the same range or is the range growing (as I suspect)?
> 
> Cheers,
> Michael
> 
> _______________________________________________
> gaia mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/gaia
>

---

From: Mirjam Kuehne <[email-masked]>
Date: Tue, 1 Mar 2016 14:48:04 +0100

Hi,

Thanks to Jane Coffin for pointing me to this discussion.

You might also want to have a look at the RIPE Atlas system, a worldwide
active measurements network: https://atlas.ripe.net/

While we don't actually measure bandwidth, the thousands of RIPE Atlas
measurement probes regularly send pings and traceroutes (and also SSL,
DNS queries) to certain pre-defined destinations on the Internet.
Especially the traceroute results could be interesting for the questions
below.

The measurement results are public and are used by many researchers and
operators for various use cases. Have a look at this blog for some
inspiration: https://labs.ripe.net/atlas

If you have any questions, please don't hesitate to contact me directly.

Kind regards,
Mirjam Kuehne
RIPE NCC

> 
> On 2/29/16, 2:59 PM, "gaia on behalf of Michael Welzl" <[email-masked] on behalf of [email-masked]> wrote:
> 
>> Great, thanks!
>>
>> Good thinking about Akamai’s report - of course! I should have looked at that first
>>
>>
>>> On 29. feb. 2016, at 19.22, [email-masked] wrote:
>>>
>>> Hi Michael
>>>
>>> A good starting point is Ofcom's infrastructure annual report; here is the 2015 one:
>>> http://stakeholders.ofcom.org.uk/binaries/research/infrastructure/2015/downloads/connected_nations2015.pdf
>>>
>>> You will find several distribution graphs like the one in Figure 4 that compares the monthly usage to the access speed range for 2014 and 2015.
>>>
>>> You could also find the reports from previous years to see how things have changed over time:
>>> http://stakeholders.ofcom.org.uk/market-data-research/market-data/infrastructure/?a=0
>>>
>>> For a more global point of view, you could check Akamai's State of the Internet Report:
>>> https://www.stateoftheinternet.com/resources-connectivity-2015-q3-state-of-the-internet-report.html
>>>
>>> or play with their visualisations:
>>> https://www.stateoftheinternet.com/trends-visualizations-connectivity-global-heat-map-internet-speeds-broadband-adoption.html
>>>
>>> Hope this helps
>>> Achilles
>>>
>>>
>>> -----Original Message-----
>>> From: gaia [mailto:[email-masked]] On Behalf Of Michael Welzl
>>> Sent: 29 February 2016 08:16
>>> To: Manner Jukka <[email-masked]>
>>> Cc: [email-masked]
>>> Subject: Re: [gaia] Heterogeneity in network capacity: growing?
>>>
>>> First,
>>>
>>> Apologies for asking first formulating a question in two ways with contradictory outcomes (yes/no) and then saying "I guess the answer is no" - that was confusing!  :)   I was tired when I wrote it...
>>> so I meant the second question, i.e. my guess was also that heterogeneity *is* growing.
>>>
>>> How would you calculate if there was data: I guess what I'm interested in is bandwidth statistics: how often do we see a bottleneck of X bit/s? The Internet-wide distribution of X is what I'm looking for...
>>>
>>> Depending on area/market: I'd also be happy to have X for India, X for Europe, X for ... whatever. It's a start!
>>> Ideally citable - some publicly available study I could refer to.
>>>
>>> Cheers,
>>> Michael
>>>
>>>
>>>
>>>> On 29 Feb 2016, at 08:33, Manner Jukka <[email-masked]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I would say yes, they are globally, but it also depends on what area/market you are talking about. How would you calculate it if there was data? I have data, but not sure how to calculate this.
>>>>
>>>> cheers,
>>>> Jukka
>>>>
>>>>> On 29 Feb 2016, at 01:04, Michael Welzl <[email-masked]> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I guess this is probably the right group to ask: is the heterogeneity of link bandwidths growing?
>>>>> Another way of asking this is: across the whole Internet, do old links roughly get removed as quickly as capacities grow?
>>>>>
>>>>> I guess the answer is no - but are there any good citable references for this? Data?
>>>>>
>>>>> Cheers,
>>>>> Michael
>>>>>

---

From: Michael Welzl <[email-masked]>
Date: Tue, 1 Mar 2016 15:04:17 +0100

Thanks a lot for that pointer too!
I know about the RIPE atlas and do find it very interesting.

However, probably I'm missing something - how is the traceroute dataset relevant for what I'm asking?
My question is about link bandwidth...

> On 01 Mar 2016, at 14:48, Mirjam Kuehne <[email-masked]> wrote:
> 
> Hi,
>

---

From: Mirjam Kuehne <[email-masked]>
Date: Tue, 1 Mar 2016 15:17:41 +0100

Hi Michael,

You are right: RIPE Atlas doesn't measure bandwidth but I thought
measuring latency could add another aspect to the question.

Kind regards,
Mirjam

---

From: Nick Feamster <[email-masked]>
Date: Tue, 1 Mar 2016 10:10:26 -0500

Folks may also be interested in the BISmark project:
http://projectbismark.net/

We measure throughput from hundreds of home networks around the world in about 30 countries.  All of that data is available at http://uploads.projectbismark.net/

We also have a mobile throughput application:
http://play.google.com/store/apps/details?id=com.num

We’ve used these to study throughput in the US, South Africa, and Pakistan. In South Africa and Pakistan, we looked at both fixed line and mobile throughput.  Here are the relevant papers:

* US: http://goo.gl/p48tDr
* South Africa: http://goo.gl/kONyHu
* Pakistan: http://goo.gl/9RdzXK

-Nick

> On Mar 1, 2016, at 9:17 AM, Mirjam Kuehne <[email-masked]> wrote:
> 
> Hi Michael,
>

---

From: Jane Coffin <[email-masked]>
Date: Tue, 1 Mar 2016 15:36:25 +0000

This all fits together as Mirjam and Nick’s info points out.

From terrestrial fibre related issues to submarine cables we have many moving parts that relate to bottlenecks and throughput and bandwidth.

We would need to define Internet-wide distribution of X as the variables within some regions is enormous.

---

From: "Steven G. Huter" <[email-masked]>
Date: Tue, 1 Mar 2016 08:03:18 -0800 (PST)

Thank you, Mirjam.

Steve Huter

---

From: Richard Dent <[email-masked]>
Date: Fri, 4 Mar 2016 14:15:38 +0000

Facebook's 2015 State Of Connectivity report is out:
http://newsroom.fb.com/news/2016/02/state-of-connectivity-2015-a-report-on-global-internet-access/

"In order to address the barriers to connectivity, corporations, 
governments, NGOs and non-profits need to work together to continue 
gathering more accurate data on the state of global connectivity, and 
develop global standards for collecting, reporting, and distributing 
this data.

As one example, Facebook is collaborating with the Center for 
International Earth Science Information Network at Columbia University 
to produce detailed maps showing the population distribution of 20 
countries. These maps were created using new machine learning techniques 
and show the most accurate estimates of population distribution and 
settlements available to date."