Subject: [p2p-sip] First Rule of Distributed Computing:

From: adamfisk at gmail.com (Adam Fisk)
Date: Sun, 10 Dec 2006 16:50:46 -0500

Don't distribute your objects.

No specific point to be made here, but I find that rule useful in my own
coding and thinking, and it's surprisingly easy to forget.  If you can avoid
distributing your objects, you're usually left with a much easier system to
implement and maintain.  Is that Radia Perlman?  Not sure.

-Adam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.cs.columbia.edu/pipermail/p2p-sip/attachments/20061210/1d1ddee5/attachment.html

---

From: dbarrett at quinthar.com (David Barrett)
Date: Sun, 10 Dec 2006 20:44:51 -0800

Heh, this seems like very good advice, but I'm not sure I follow your
meaning.  Can you give us an example?

-david

  _____  

From: p2p-sip-bounces at cs.columbia.edu
[mailto:p2p-sip-bounces at cs.columbia.edu] On Behalf Of Adam Fisk
Sent: Sunday, December 10, 2006 1:51 PM
To: p2p-sip at cs.columbia.edu
Subject: [p2p-sip] First Rule of Distributed Computing:

-Adam

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.cs.columbia.edu/pipermail/p2p-sip/attachments/20061210/55905ede/attachment.html

---

From: adamfisk at gmail.com (Adam Fisk)
Date: Mon, 11 Dec 2006 10:11:22 -0500

The example I encounter most frequently is search.  Distributed search is
hard.  It's hard to search a lot of computers without taking up a lot of
bandwidth, not to mention the algorithms are just tough and require massive
coordination amongst many machines.  The bandwidth constraints lead to
architectures that increase latency even if they're scalable, like DHTs or
distributed indexing with random walk-type approaches on top of them.  The
bandwidth constraints and latency also put a constraint on message sizes,
significantly limiting your options for whatever protocol you're working
with.  Updates are also hard.  Instead of updating a server cluster, you
have to update sometimes millions of machines.

With centralized search, these constraints largely dissappear.  You get low
latency, global search, all with much less work.  Things like collaborative
filtering and tagging also become possible, whereas they are extremely
complicated at best with distributed systems.  Sure there's a single point
of failure, but that's rarely an issue in practice if you know what you're
doing.

There have to be really, really good reasons to choose the first,
distributed approach.  In the case of the file sharing networks, that reason
was legal liability.  In other cases, it might be cost.

That's just a rule I frequently have to remind myself of, and it's saved me
a great deal of time in the past.

-Adam

On 12/10/06, David Barrett <dbarrett at quinthar.com> wrote:
>   ------------------------------
>
> *From:* p2p-sip-bounces at cs.columbia.edu [mailto:
> p2p-sip-bounces at cs.columbia.edu] *On Behalf Of *Adam Fisk
> *Sent:* Sunday, December 10, 2006 1:51 PM
> *To:* p2p-sip at cs.columbia.edu
> *Subject:* [p2p-sip] First Rule of Distributed Computing:
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.cs.columbia.edu/pipermail/p2p-sip/attachments/20061211/fcb6696b/attachment.html

---

From: hsinnrei at adobe.com (Henry Sinnreich)
Date: Mon, 11 Dec 2006 19:50:57 -0800

Adam Fisk wrote:

>There have to be really, really good reasons to choose the first, distributed approach.

Right. Here are some reasons in the case of Client-Server SIP VoIP. There are about 10-100 VoIP network elements in larger VoIP networks, such as:

* DNS, ENUM servers with VoIP data entries
* SIP registrar and proxy servers, fronting SIP proxies
* Presence servers
* Component servers: 
  - Announcements, voice mail, media servers, conference servers
* Session border controllers (at several points in the network)
* Softswitches: Class 5 switch emulation and/or hosted IP PBXs (Centrex)
* Media servers
* Network elements for QoS
* Policy servers
* Network management
* Voice quality monitoring probes
* Network engineering
* System integration
* VoIP network IT (OSS) systems for all the above
* IT systems provisioned with churning user data for most of the above - this is the most onerous part.
Note that I did not count TDM gateways since gateways would be needed for P2P as well.

This is a nightmare of an infrastructure, since PCs at the endpoints are more than powerful enough to provide these functions and also to move the cost of hosting and bandwidth to the endpoints. There is no P2P VoIP infrastructure and P2P systems are self organizing.

Skype with its >100M users has proven in the market that P2P avoids this complexity and cost (staffed operations 7x24, staffed custom system integration, huge IT systems development, etc.) and most of all, lack of flexibility can be mostly avoided and still provide rich consumer presence, IM, VoIP, video, file transfer and now also conferencing and various enterprise applications. 

I don't have profit numbers for Skype (they had a good business model based on prepaid float I believe before becoming eBay), but I have not seen any profit statements from any VoIP company that I know of, except when VoIP is bundled with broadband Internet or with something else. Does the bundling hide the losses for VoIP service?

VoIP is just an application sitting in the endpoints and that's how it should be on "the dumb network". 
Since when do we need the "Intelligent Network" on the Internet (the IETF's task)? 
This is the rationale for developing standards for P2P SIP.

Leaving VoIP aside, there are roughly 5,000 papers that deal with P2P and for what it is worth, p2p traffic dominates the Internet. Obviously for good reasons.

Thanks, Henry
________________________________________
From: p2p-sip-bounces at cs.columbia.edu [mailto:p2p-sip-bounces at cs.columbia.edu] On Behalf Of Adam Fisk
Sent: Monday, December 11, 2006 9:11 AM
To: David Barrett
Cc: p2p-sip at cs.columbia.edu
Subject: Re: [p2p-sip] First Rule of Distributed Computing:

The example I encounter most frequently is search.? Distributed search is hard.? It's hard to search a lot of computers without taking up a lot of bandwidth, not to mention the algorithms are just tough and require massive coordination amongst many machines.? The bandwidth constraints lead to architectures that increase latency even if they're scalable, like DHTs or distributed indexing with random walk-type approaches on top of them.? The bandwidth constraints and latency also put a constraint on message sizes, significantly limiting your options for whatever protocol you're working with.? Updates are also hard.? Instead of updating a server cluster, you have to update sometimes millions of machines. 

With centralized search, these constraints largely dissappear.? You get low latency, global search, all with much less work.? Things like collaborative filtering and tagging also become possible, whereas they are extremely complicated at best with distributed systems.? Sure there's a single point of failure, but that's rarely an issue in practice if you know what you're doing. 

There have to be really, really good reasons to choose the first, distributed approach.? In the case of the file sharing networks, that reason was legal liability.? In other cases, it might be cost.

That's just a rule I frequently have to remind myself of, and it's saved me a great deal of time in the past. 

-Adam

On 12/10/06, David Barrett <dbarrett at quinthar.com> wrote:
Heh, this seems like very good advice, but I'm not sure I follow your meaning.? Can you give us an example?
?
-david
?
________________________________________
From: p2p-sip-bounces at cs.columbia.edu [mailto: p2p-sip-bounces at cs.columbia.edu] On Behalf Of Adam Fisk
Sent: Sunday, December 10, 2006 1:51 PM
To: p2p-sip at cs.columbia.edu
Subject: [p2p-sip] First Rule of Distributed Computing:
?
Don't distribute your objects.? 

No specific point to be made here, but I find that rule useful in my own coding and thinking, and it's surprisingly easy to forget.? If you can avoid distributing your objects, you're usually left with a much easier system to implement and maintain.? Is that Radia Perlman?? Not sure. 

-Adam

---

From: michael.slavitch at e2exchange.com (Michael Slavitch)
Date: Tue, 12 Dec 2006 11:10:02 -0500

There has to be really good reasons to choose the centralized approach.

Historically, strict centralization works best only when all resources are
scarce, for everything from food distribution to decision making to
economics.  Centralized food distribution goes back to Old Testament times.
The old phrase 'seven years of bumper crop, seven years of famine' might as
well apply to the technology industry with 2001 being the start of famine,
although it may be the telecoms that end up suffering Pharaoh's fate. They
should well consider the consumer's Technicolor Dream-Coat.

Central planning always fails when there is abundance of resource supply,
broad education  and free access to data.  Central planning  fails because
with abundance the central planners can be ignored.  Given that, the first
act of central planners when faced with a revolt of abundance is and always
has been repression and more centralization.

This has been demonstrated over the centuries.

Some successful revolts include the signing of the Magna Carta, the
Reformation,  the Renaissance, and the American Revolution.

In communications terms that analogy relates to long term store, CPU and
bandwidth.

In political terms it looks like a counter-revolution is underway in
Russia.  The telecoms industry is mounting one in the United States.

I suspect the latter will fail.

Classical distributed systems that are used as a solution to scarcity are
inappropriate for comparison to P2P systems that designed to accommodate and
self organize based on resource feast and resource famine. It may be a
mistake to compare the problems seen in distributed systems with the
opportunity in peer to peer systems, especially  if those distributed system
are designed only for scarcity and not abundance.

Scarcity is not a constant. It is now an occasional condition that must be
accommodated and respected. It is no longer the norm.

This means that distributed solutions designed primarily to defeat scarcity
without taking into consideration abundance are inappropriate.
A global calling database for every human being in the world could easily
fit on a PC hard disk, the updates being merely the deltas over time.   A
contact list containing every person in the US can easily fit on a flash
card.   That alone would be an interesting product for smart-phones. Every
sales rep would want one.  A nightly update over WiLAN can update its
database to five-sigma accuracy without cost to the user.  The sixth sigma,
done over an expensive 3G network, will cost a few pennies once in a while
over a dumb network. In economic terms that is a rational system, a rational
consumer will always gravitate towards such a system given a choice.

The search problem is eliminated when distributed systems are not completely
distributed but instead hyper-redundant.

Every hand held smart phone can now contain a class-5 switch database. There
is enough store. There certainly is enough CPU for the call processing. If
every smart phone can be a class-5 switch why do you need centralized
class-5 switches?

I agree that the search problem is a problem for older P2P systems, like
those used for file sharing. By definition a database of stolen goods is
always unstable. That is not true for identities or other legal attributes.

If search is a problem, eliminate search as a problem, and limit the churn
to updates.  The research says that you'll reduce lookups by many orders of
magnitude.

Regards

Michael

Please note the new email and domain, www.e2exchange.com

On 12/12/06, Michael Slavitch <slavitch at gmail.com> wrote:
>
> ---------- Forwarded message ----------
> From: Henry Sinnreich <hsinnrei at adobe.com>
> Date: Dec 11, 2006 10:50 PM
> To: Adam Fisk < adamfisk at gmail.com>
> distributed approach.
> Right. Here are some reasons in the case of Client-Server SIP VoIP.
> There are about 10-100 VoIP network elements in larger VoIP networks,
> such as:
>
> This is a nightmare of an infrastructure, since PCs at the endpoints
> are more than powerful enough to provide these functions and also to
> move the cost of hosting and bandwidth to the endpoints. There is no
> P2P VoIP infrastructure and P2P systems are self organizing.
>
> Skype with its >100M users has proven in the market that P2P avoids
> this complexity and cost (staffed operations 7x24, staffed custom
> system integration, huge IT systems development, etc.) and most of
> all, lack of flexibility can be mostly avoided and still provide rich
> consumer presence, IM, VoIP, video, file transfer and now also
> conferencing and various enterprise applications.
>
> I don't have profit numbers for Skype (they had a good business model
> based on prepaid float I believe before becoming eBay), but I have not
> seen any profit statements from any VoIP company that I know of,
> except when VoIP is bundled with broadband Internet or with something
> else. Does the bundling hide the losses for VoIP service?
>
> Leaving VoIP aside, there are roughly 5,000 papers that deal with P2P
> and for what it is worth, p2p traffic dominates the Internet.
> Obviously for good reasons.
>
> Thanks, Henry
> The example I encounter most frequently is search. Distributed search
> is hard. It's hard to search a lot of computers without taking up a
> lot of bandwidth, not to mention the algorithms are just tough and
> require massive coordination amongst many machines. The bandwidth
> constraints lead to architectures that increase latency even if
> they're scalable, like DHTs or distributed indexing with random
> walk-type approaches on top of them. The bandwidth constraints and
> latency also put a constraint on message sizes, significantly limiting
> your options for whatever protocol you're working with. Updates are
> also hard. Instead of updating a server cluster, you have to update
> sometimes millions of machines.
>
> With centralized search, these constraints largely dissappear. You get
> low latency, global search, all with much less work. Things like
> collaborative filtering and tagging also become possible, whereas they
> are extremely complicated at best with distributed systems. Sure
> there's a single point of failure, but that's rarely an issue in
> practice if you know what you're doing.
>
> distributed approach. In the case of the file sharing networks, that
> reason was legal liability. In other cases, it might be cost.
>
> That's just a rule I frequently have to remind myself of, and it's
> saved me a great deal of time in the past.
>
> -Adam
>
> From: p2p-sip-bounces at cs.columbia.edu [mailto:
> p2p-sip-bounces at cs.columbia.edu ] On Behalf Of Adam Fisk
> No specific point to be made here, but I find that rule useful in my
> own coding and thinking, and it's surprisingly easy to forget. If you
> can avoid distributing your objects, you're usually left with a much
> easier system to implement and maintain. Is that Radia Perlman? Not
> sure.
>
> -Adam
>
>
> _______________________________________________
> p2p-sip mailing list
> p2p-sip at cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/p2p-sip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.cs.columbia.edu/pipermail/p2p-sip/attachments/20061212/a962fca8/attachment-0001.html

---

From: RADHIKA.R.ROY at saic.com (Roy, Radhika R.)
Date: Tue, 12 Dec 2006 11:25:42 -0500

It depends what the basic assumptions are. It is true that the
centralized systems will have their uses as it has been pointed by you.
There is no debate on this point.

For example, all telcos have their existing centralized systems and
there is no need to change those for them as those business models will
be working for them for many foreseeable years.

However, companies like Skype/eBay did not have those luxuries:
P2P-system is a good business model for them.

The lesson is this: P2P systems are opening the businesses to millions
of people who can start with nothing and still can have most of it
(i.e., as if, opening the business models for creation of wealth by a
community of large-scale masses).

Best regards,

Radhika

________________________________

From: p2p-sip-bounces at cs.columbia.edu
[mailto:p2p-sip-bounces at cs.columbia.edu] On Behalf Of Michael Slavitch
Sent: Tuesday, December 12, 2006 11:10 AM
To: p2p-sip at cs.columbia.edu
Subject: Re: [p2p-sip] First Rule of Distributed Computing:

Historically, strict centralization works best only when all resources
are scarce, for everything from food distribution to decision making to
economics.  Centralized food distribution goes back to Old Testament
times.  The old phrase 'seven years of bumper crop, seven years of
famine' might as well apply to the technology industry with 2001 being
the start of famine, although it may be the telecoms that end up
suffering Pharaoh's fate. They should well consider the consumer's
Technicolor Dream-Coat.  

Central planning always fails when there is abundance of resource
supply,  broad education  and free access to data.  Central planning
fails because with abundance the central planners can be ignored.  Given
that, the first act of central planners when faced with a revolt of
abundance is and always has been repression and more centralization. 

Classical distributed systems that are used as a solution to scarcity
are inappropriate for comparison to P2P systems that designed to
accommodate and self organize based on resource feast and resource
famine. It may be a mistake to compare the problems seen in distributed
systems with the opportunity in peer to peer systems, especially  if
those distributed system are designed only for scarcity and not
abundance.  

Scarcity is not a constant. It is now an occasional condition that must
be accommodated and respected. It is no longer the norm.

This means that distributed solutions designed primarily to defeat
scarcity without taking into consideration abundance are inappropriate.

A global calling database for every human being in the world could
easily fit on a PC hard disk, the updates being merely the deltas over
time.   A contact list containing every person in the US can easily fit
on a flash card.   That alone would be an interesting product for
smart-phones. Every sales rep would want one.  A nightly update over
WiLAN can update its database to five-sigma accuracy without cost to the
user.  The sixth sigma, done over an expensive 3G network, will cost a
few pennies once in a while over a dumb network. In economic terms that
is a rational system, a rational consumer will always gravitate towards
such a system given a choice. 

The search problem is eliminated when distributed systems are not
completely distributed but instead hyper-redundant.  

Every hand held smart phone can now contain a class-5 switch database.
There is enough store. There certainly is enough CPU for the call
processing. If every smart phone can be a class-5 switch why do you need
centralized class-5 switches?

I agree that the search problem is a problem for older P2P systems, like
those used for file sharing. By definition a database of stolen goods is
always unstable. That is not true for identities or other legal
attributes. 

If search is a problem, eliminate search as a problem, and limit the
churn to updates.  The research says that you'll reduce lookups by many
orders of magnitude. 

Regards

Michael

Please note the new email and domain, www.e2exchange.com
<http://www.e2exchange.com/> 

---------- Forwarded message ---------- 
From: Henry Sinnreich <hsinnrei at adobe.com>
Date: Dec 11, 2006 10:50 PM
Subject: Re: [p2p-sip] First Rule of Distributed Computing: 
To: Adam Fisk < adamfisk at gmail.com <mailto:adamfisk at gmail.com> >
Cc: p2p-sip at cs.columbia.edu

distributed approach. 

* Media servers
* Network elements for QoS
* Policy servers
* Network management
* Voice quality monitoring probes
* Network engineering
* System integration
* VoIP network IT (OSS) systems for all the above 
* IT systems provisioned with churning user data for most of the above
- this is the most onerous part.
Note that I did not count TDM gateways since gateways would be needed
for P2P as well.

VoIP is just an application sitting in the endpoints and that's how it
should be on "the dumb network". 
Since when do we need the "Intelligent Network" on the Internet (the
IETF's task)?
This is the rationale for developing standards for P2P SIP.

Thanks, Henry

-Adam

-david

-Adam

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.cs.columbia.edu/pipermail/p2p-sip/attachments/20061212/e3030698/attachment.html

---

From: warlock at media.mit.edu (Jacky Mallett)
Date: Fri, 15 Dec 2006 00:50:06 -0500

On Dec 11, 2006, at 10:50 PM, Henry Sinnreich wrote:

> all, lack of flexibility can be mostly avoided and still provide  
> rich consumer presence, IM, VoIP, video, file transfer and now also  

I've seen you say this a few times, so i'd like to challenge a couple  
of things about it. First of all - Skype may have 100M users in their  
(centralised) databases. The most i've ever seen online is about 8.5  
million.  When talking about scaling issues within peer-to-peer  
applications, it's the 8.5 million figure that's significant. And the  
proper comparison would be the number of concurrent users of the  
global phone system.

In addition to the centralised login database, it appears there are  
also at least 7 bootstrap nodes with statically assigned IP addresses.

http://skype.nexxen.cz/dokumentace/public/SkypeProtocol/html/ 
SkypeProtocol_4.htm

It is correct to say that the SIP protocol as implemented is the  
endpoint application. The distributed _system_ that uses it and other  
protocols to interconnect users, is not. It has network like  
properties all of its own.  Network in this sense is used to mean  
systems with organised communications between large numbers of nodes.

....jacky

>
> I don't have profit numbers for Skype (they had a good business  
> model based on prepaid float I believe before becoming eBay), but I  
> have not seen any profit statements from any VoIP company that I  
> know of, except when VoIP is bundled with broadband Internet or  
> with something else. Does the bundling hide the losses for VoIP  
> service?
>
> Leaving VoIP aside, there are roughly 5,000 papers that deal with  
> P2P and for what it is worth, p2p traffic dominates the Internet.  
> From: p2p-sip-bounces at cs.columbia.edu [mailto:p2p-sip- 
> bounces at cs.columbia.edu] On Behalf Of Adam Fisk
> The example I encounter most frequently is search.  Distributed  
> search is hard.  It's hard to search a lot of computers without  
> taking up a lot of bandwidth, not to mention the algorithms are  
> just tough and require massive coordination amongst many machines.   
> The bandwidth constraints lead to architectures that increase  
> latency even if they're scalable, like DHTs or distributed indexing  
> with random walk-type approaches on top of them.  The bandwidth  
> constraints and latency also put a constraint on message sizes,  
> significantly limiting your options for whatever protocol you're  
> working with.  Updates are also hard.  Instead of updating a server  
> cluster, you have to update sometimes millions of machines.
>
> With centralized search, these constraints largely dissappear.  You  
> get low latency, global search, all with much less work.  Things  
> like collaborative filtering and tagging also become possible,  
> whereas they are extremely complicated at best with distributed  
> systems.  Sure there's a single point of failure, but that's rarely  
> an issue in practice if you know what you're doing.
>
> distributed approach.  In the case of the file sharing networks,  
> that reason was legal liability.  In other cases, it might be cost.
>
> Heh, this seems like very good advice, but I'm not sure I follow  
> your meaning.  Can you give us an example?
>
> -david
>
> From: p2p-sip-bounces at cs.columbia.edu [mailto: p2p-sip- 
> bounces at cs.columbia.edu] On Behalf Of Adam Fisk
> No specific point to be made here, but I find that rule useful in  
> my own coding and thinking, and it's surprisingly easy to forget.   
> If you can avoid distributing your objects, you're usually left  
> with a much easier system to implement and maintain.  Is that Radia  
> Perlman?  Not sure.
>
> -Adam
>
>

---

From: br at brianrosen.net (Brian Rosen)
Date: Fri, 15 Dec 2006 10:37:19 -0500

Jacky

> proper comparison would be the number of concurrent users of the
I'm not sure what "concurrent users" means.  The 8.5M number is the number
of subscribers on-line, not the number talking.  It's equivalent to a
wireless phone connected to a network and able to, or actively placing or
receiving a call.  I haven't seen numbers like that for wireless systems.
What you see is "subscribers", which is the big number for Skype.  It's
clearly true that the ratio of subscribers to devices on line and available
for or in a call is much higher for wireless systems than it is for Skype.

Also, the numbers are for a single carrier, not the entire PSTN.  Carriers
have interconnects, as will P2P-SIP domains.

> http://skype.nexxen.cz/dokumentace/public/SkypeProtocol/html/
Yes, the bootstrap uses fixed addresses.  Once you have contacted a peer,
you don't need them any more.  We will need some kind of a bootstrap
mechanism, and I think while we won't prohibit a fixed set of IP addresses,
I think we're probably going to have other mechanisms as well.

> systems with organised communications between large numbers of nodes.
Well, at least most of the proposals for the client protocol have that being
SIP.  There are some people who think that the peer protocol is some kind of
extension to SIP.  We'll see.

Brian

---

From: dan at ag-projects.com (Dan Pascu)
Date: Fri, 15 Dec 2006 17:58:57 +0200

On Friday 15 December 2006 17:37, Brian Rosen wrote:
> Yes, the bootstrap uses fixed addresses.  Once you have contacted a
> peer, you don't need them any more.  We will need some kind of a
> bootstrap mechanism, and I think while we won't prohibit a fixed set of
> IP addresses, I think we're probably going to have other mechanisms as
> well.

I think that a more effective and also simpler mechanism for bootstrapping 
(than the list of fixed IP addresses) would be to have a domain name as 
the bootstrap point. This domain name would have a specific DNS SRV 
record which would finally point to a set of A records holding the IP 
addresses that will be used for bootstrapping.

While it has the same result, it is simpler to operate because if the 
fixed IP addresses change, you do not have to update the IP address list 
in each peer. All you have to do is update the list of A records to which 
the SRV record points and any peer will find the new IP's immediately 
using a simple DNS lookup.

-- 
Dan

---

From: warlock at media.mit.edu (Jacky Mallett)
Date: Sat, 16 Dec 2006 15:31:18 -0500

> for or in a call is much higher for wireless systems than it is for  

that's what i meant. Skype essentially provides backbone network  
support to locate its users and allow them to circumvent NAT  
restrictions when necessary, and then the call traffic itself is  
mostly peer-to-peer. It doesn't seem to be widely appreciated that  
there are also limits on the size of the super node layer that is  
used to achieve this,  which implies that there are also limits on  
the number of online users that can be supported with their current  
architecture.

There is a similar disconnect in the phone network between the number  
of people with phone lines, and the number of people who could  
simultaneously make a phone call - i don't know the wireless figures  
either, but iirc the old PSTN engineering guideline was something  
like 400:1.

> have interconnects, as will P2P-SIP domains.

Yes, in fact that's the only way we'll be able to scale the solution.  
Skype, of course, is notoriously a closed and encrypted system with  
no interconnects. Since some people keep using it as the reference  
case, i felt it might be useful to be a little more precise about how  
it actually works.

> extension to SIP.  We'll see.

Indeed.

...Jacky

---

From: br at brianrosen.net (Brian Rosen)
Date: Sat, 16 Dec 2006 16:25:25 -0500

> the number of online users that can be supported with their current
I didn't think there was any effective limit on the size of the supernode
layer.  What kind of limit would there be?  It seems to be a pretty
classical P2P system which normally doesn't have such limits, at least until
the systems get larger than the kinds of systems we're talking about.

I've often wondered if there is an effective limit on the number of clients
behind a NAT related to the number of supernodes not behind the NAT.  I
suspect that if there are too many clients per supernode, it won't work
well.

> either, but iirc the old PSTN engineering guideline was something

Here the Internet has a significant advantage.  Because at least voice
communications is so much smaller than pipe size, I think it would actually
be possible for every logged in Skype user to be talking at the same time.
Unlike most TDM switches, with respect to the kinds of bandwidth needed for
voice and/or text, Skype should be non-blocking.

Brian

---

From: warlock at media.mit.edu (Jacky Mallett)
Date: Sat, 16 Dec 2006 17:16:48 -0500

> the systems get larger than the kinds of systems we're talking about.

the limit on the size of the supernode layer is the limit on the  
number of connections each supernode can support with every other  
supernode (assuming a completely meshed network.) Otherwise, for a  
partially meshed network, it's a combination of that and the traffic  
within that level, bearing in mind that the partial mesh reduces the  
amount of connections each super node has to support, but increases  
the traffic within the supernode layer because not all traffic can  
then be sent directly supernode to supernode any more. Although most  
descriptions of Skype present the supernode layer as a complete mesh,  
it has to be partial,  otherwise the number of connections per  
supernode would be ridiculous.

Similarly there's a limit on the number of clients each supernode can  
support, since they're effectively in a client-server relationship  
with each other.

When the first p2p file sharing networks came in, they essentially  
functioned as a partial mesh network themselves, so they were in  
essence the super-node layer, and they eventually bogged down under  
the limits of scale under this model. Introducing the super-node  
layer doesn't solve that problem so much as postpone it, the same  
problems of excessive hop related administrative traffic eventually  
re-appears at that level as it grows.

> suspect that if there are too many clients per supernode, it won't  

that's the other problem. Which is what will make the upcoming surge  
in wifi-pda users quite interesting to watch.

> voice and/or text, Skype should be non-blocking.

And this is where it will get very interesting. Yes, the fibre optic  
lines have dramatically increased the size of the pipe. And yes, the  
amount of traffic that a mesh/partial network can support is  
dramatically larger than that of any hierarchically organised  
network. O(Lsqrt(N)), vs O(L), where L is the connectivity supported  
by respectively the nodes in the mesh, and the hierarchical servers  
in the respective network topologies. However, the amount of traffic  
generated by all Skype users talking to each other simultaneously is O 
(N/2) which is eventually greater than O(Lsqrt(N)) for large numbers  
of N and reasonable assumptions for L.

The less theoretical observation that should also be made, is that  
today's internet is not engineered for continuous two way traffic, it  
is, by and large engineered with the assumption that traffic is  
highly asymmetric at the end points, which is already presenting its  
own problems. And let's not even think about what any significant  
amount of real time video traffic could do to this.

Some of these problems can be solved by technology as the size of the  
pipes increases, some of them are limits on any distributed system  
and will have to be engineered.

...Jacky