Subject: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion

From: Jonathan Rosenberg <[email-masked]>
Date: Fri, 13 Jul 2007 17:35:39 -0400

I've read draft-bryan-p2psip-requirements-00, and have a bunch of 
comments. But a few items came to mind I wanted to discuss which merit 
their own threads.

First, is the concept of mandatory-to-implement DHT. The requirements 
document talks a lot about requirements around selecting a mandatory to 
implement DHT. Its something we've discussed in the meeting and on the 
list. I think everyone agrees also that we need to allow for multiple 
DHT and all of the recent protocol proposals support it.

However, I do not think we should have a mandatory-to-implement DHT at all.

There are several reasons for this. First of all, its important to 
understand the reason why IETF has mandatory-to-implement functionality 
in its protocols. The one and only reason is that it ensures 
interoperability. It ensures that, independent of the optional features 
selected by a pair of entities, when you connect them together, they can 
still interoperate.

This property will NOT be retained with a mandatory-to-implement DHT. 
Once the ring forms, the DHT protocol is chosen and locked. Thus, if any 
  DHT besides the 'mandatory-to-implement' one has been selected for the 
DHT, a new node not supporting that DHT will not be able to join the 
ring even if it implements the mandatory-to-implement one. Thus, the 
entire purpose of mandatory-to-implement is eliminated - we don't 
actually get interoperability.

Indeed, to get this kind of interoperability, we'd need to pick one and 
only one DHT that ever gets used with p2psip. I think that is a mistake. 
This is an evolving area and is one where agility is important.

Instead, what I think happens is that a 'provider' that deploys a p2p 
network will need to pick a DHT, and make sure that the clients all 
support that DHT. This is something the market should drive, not us.

Secondly, I think its important to realize that we are picking up work 
in an area that is well trod. There are lots and lots of papers and 
protocols and software written around P2P networks. I don't think anyone 
looks at an IETF protocol and says, "IETF are the guys who know about 
DHTs, lets go with their recommendation". What IETF is really good at, 
and what this group has expertise in, is SIP, and on designing good, 
scalable protocols in general - things that have security, 
extensibility, good performance, and so on. So I think the greatest 
value we can bring to the table is to create a protocol that allows 
others to take DHTs and easily turn them into a real wire protocol that 
you can actually deploy, on the Internet, to support SIP and ideally 
other things too. Interestingly, this is exactly what all of the 
protocol proposals do. That kind of focus goes hand-in-hand with saying, 
let someone else figure out which DHT to use. I think its perfectly 
reasonable for us, and for others, to write documents on how to use our 
protocol with various DHTs. But I think we should be out of the business 
of picking one or recommending one (or ones) to be used.

-Jonathan R.
-- 
Jonathan D. Rosenberg, Ph.D.                   600 Lanidex Plaza
Cisco Fellow                                   Parsippany, NJ 07054-2711
Cisco Systems
[email-masked]                              FAX:   (973) 952-5050
http://www.jdrosen.net                         PHONE: (973) 952-5000
http://www.cisco.com

_______________________________________________
P2PSIP mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/p2psip

---

From: "Henry Sinnreich" <[email-masked]>
Date: Fri, 13 Jul 2007 14:48:11 -0700

Jonathan,

Would it make sense to replace "mandatory" with "default"?

After all, there are just as many advantages in having a good chance of
interoperability as in flexibility for the future.

It's also useful to have a smallest common denominator to measure and
compare various implementations.

Henry

-----Original Message-----
From: Jonathan Rosenberg [mailto:[email-masked]] 
Sent: Friday, July 13, 2007 2:36 PM
To: P2PSIP WG
Subject: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion

However, I do not think we should have a mandatory-to-implement DHT at
all.

still interoperate.

-Jonathan R.

---

From: "David A. Bryan" <[email-masked]>
Date: Fri, 13 Jul 2007 15:16:28 -0700

I think this likely to be a source of serious discussion. At least for
now (maybe Jonathan can convince me), I'm inclined to disagree.

I think the problem is that if we don't specify a mandatory to
implement, there may be many devices from different vendors that don't
even implement a common DHT. Each will pick and choose which DHTs to
implement, and in the end, there won't be a common DHT all can even
run if they want to.

I think looking at it from the perspective of "once the DHT chosen it
is locked" is the wrong way to look at it. The idea isn't that any
device can join any already established overlay, but that if I deploy
a system of heterogeneous devices, there is at least one that I can
select when I start and know that all "P2PSIP endpoints" (assuming
that term means anything) will at least support that and I can form an
overlay with them. Maybe not the ideal DHT for my overlay, but
something I know all devices can support, and if I run it in that
mode, it works.

Personally, I think if we don't do that, many DHTs will be
standardized for different environments (good) but each vendor will
implement a (potentially disjoint) subset, and there may be no way to
use two endpoints from different vendors together at all.

I agree that if a user selects a non-mandatory DHT, they may risk not
being fully interoperable, but I think if we don't pick one at all, we
almost guarantee things won't be interoperable.

And I (and I think nearly everyone) is 100% in the camp of "we should
allow pluggable DHTs". I think it is safe to say that one is a done
deal, but I would personally have thought a mandatory was as well. In
any case, my vote is definitely for pluggable DHTs.

David

-- 
David A. Bryan
[email-masked]
+1.757.565.0101 x101
+1.757.565.0088 (fax)
www.SIPeerior.com

---

From: Ted Hardie <[email-masked]>
Date: Fri, 13 Jul 2007 15:30:43 -0700

>use two endpoints from different vendors together at all.

I had understood a design goal of this work to be that endpoints
could speak SIP from an overlay to a SIP user attached to a
proxy. I also understood communication between overlays
to be in scope.  Did I get that wrong?

>any case, my vote is definitely for pluggable DHTs.

I'm strongly in favor of a framework-based approach.  I think Jonathan
makes a good case that a mandatory-to-implement  mechanism for
forming overlays has fewer benefits than it might appear.  There seem
to be a couple of ways past that.  I tend to think extending the design
goals for the interchange mechanism to make inter-overlay communication
a very explicit goal seems like that way to go; *that* becomes mandatory
to implement, where the overlay formation remains agile.

Two cents from,
				Ted Hardie

---

From: Emil Ivov <[email-masked]>
Date: Sat, 14 Jul 2007 01:11:07 +0200

Ted Hardie wrote:
> a very explicit goal seems like that way to go.

I find that David has made quite a good point here. What happens when
you have a Grandstream, a Linksys and some softphone all running in your
office? Are they supposed to establish 3 different overlays and then try
to maintain inter-overlay communication with each other?

I'd personally expect to be able to configure every one of them and find
at least one entry in a DHT combo box, that all three are bound have.

Emil

; *that* becomes mandatory

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 13 Jul 2007 20:02:12 -0400

Lack of a common DHT does not necessarily interfere with  
interoperability. The client protocol, at least in our thinking, is  
independent of the DHT, so a node that doesn't implement the DHT  
algorithm of the overlay it wants to talk to can still reach others,  
it just can't be a full peer. Since typically only a small fraction  
of nodes need to act as peers at any given time, this may not be much  
of a problem.

You could design a system that essentially replicated the key-data  
mappings across two DHT algorithm, a baseline universal one and the  
"fancy"/"better" one less universally spoken. This incurs  
significantly more overhead and probably yields some strange partial  
reachability cases particularly as new nodes are added, but it could  
be done.

Henning

---

From: Ted Hardie <[email-masked]>
Date: Fri, 13 Jul 2007 17:04:56 -0700

>at least one entry in a DHT combo box, that all three are bound have.

Given a plug-in architecture, you would presumably plug in the same one to
each of these when you created the overlay.  For static devices, yes, you
might have to create more than one overly and do inter-overlay devices
(for three, I'd buy new, but I'm reckless that way).   That would be bad,
especially if static-capability devices were ubiquitous and had chosen
competing methods for setting up overlays.

But should we really be aiming at either static devices or a long-term
least common denominator?  The Security area is currently talking up
the value of algorithm agility, because they are discovering that things
that don't have that built in are wicked hard to change when they
need to.  Having agility here seems something that needs to be built
in now and a built-in default seems to me to push against that in some
bad ways.

(I'm out of cents, as it is the weekend, so accept my)

				regards,
					Ted

---

From: "Medhavi Bhatia" <[email-masked]>
Date: Fri, 13 Jul 2007 20:20:58 -0400

I don't think the DHT is something which gets negotiated between two nodes.
So I don't think we need to look at the mandatory DHT in the same way as we
look as at a mandatory codec. The requirement here is to prevent a situation
where all vendors implement their own DHT to gain commercial advantage. If
we don't pick a mandatory DHT, someone else outside this group will end up
doing that anyway - maybe Skype ;)

-- 
Medhavi Bhatia

CTO, 3CLogic
www.3clogic.com

---

From: "Krishna Sankar \(ksankar\)" <[email-masked]>
Date: Fri, 13 Jul 2007 18:57:54 -0700

David,
	I assume by "pluggable DHTs" you mean common interface and
protocols. 

	Supporting your line of thought, as I understand them, basically
if we have a model which says a DHT should do this thing, that thing and
all those (well defined) things; plus a way of interacting with DHT
instances (via well established and crisp protocols), do we care how the
DHTs are implemented ? In fact, isn't this how routing works ?

> to implement, there may be many devices from different 
> vendors that don't even implement a common DHT. Each will 
> pick and choose which DHTs to implement, and in the end, 
> there won't be a common DHT all can even run if they want to.
<KS>
	Wouldn't a device implement only the protocols and interfaces ?
In fact, depending on the domain, we might need different types of DHTs
anyway. I am not sure one size will fit all domains for various reasons
- for example there might be a need for power saving; in which case that
domain might choose a lightweight DHT implementation; Apple definitely
will want to have an iDHT for no reason ;o),... 
</KS>

Also, I keep on seeing interoperability thrown around; interoperability
has different forms in different domains - as Medhavi mentions, in the
codec world what it means is different than in the routing world which
is different than in the SAML world, for example. In short what exactly
is interoperability in the world of P2PSIP ? I think if we have a clear
& common understanding of the definition of interoperability, that would
be a good start; we all have it in our heads, may be it is written down
somewhere.  

Cheers
<k/>
> From: David A. Bryan [mailto:[email-masked]] 
> Sent: Friday, July 13, 2007 3:16 PM
> To: Jonathan Rosenberg (jdrosen)
> Cc: P2PSIP WG
> Subject: Re: [P2PSIP] Mandatory-to-implement DHTs - a 
> dissenting opinion
> 
> I think this likely to be a source of serious discussion. At 
> least for now (maybe Jonathan can convince me), I'm inclined 
> to disagree.
> 
> I think the problem is that if we don't specify a mandatory 
> to implement, there may be many devices from different 
> vendors that don't even implement a common DHT. Each will 
> pick and choose which DHTs to implement, and in the end, 
> there won't be a common DHT all can even run if they want to.
> 
> I think looking at it from the perspective of "once the DHT 
> chosen it is locked" is the wrong way to look at it. The idea 
> isn't that any device can join any already established 
> overlay, but that if I deploy a system of heterogeneous 
> devices, there is at least one that I can select when I start 
> and know that all "P2PSIP endpoints" (assuming that term 
> means anything) will at least support that and I can form an 
> overlay with them. Maybe not the ideal DHT for my overlay, 
> but something I know all devices can support, and if I run it 
> in that mode, it works.
> 
> standardized for different environments (good) but each 
> vendor will implement a (potentially disjoint) subset, and 
> there may be no way to use two endpoints from different 
> vendors together at all.
> 
> I agree that if a user selects a non-mandatory DHT, they may 
> risk not being fully interoperable, but I think if we don't 
> pick one at all, we almost guarantee things won't be interoperable.
> 
> And I (and I think nearly everyone) is 100% in the camp of 
> "we should allow pluggable DHTs". I think it is safe to say 
> that one is a done deal, but I would personally have thought 
> a mandatory was as well. In any case, my vote is definitely 
> for pluggable DHTs.
> 
> David
> 
> > document talks a lot about requirements around selecting a 
> mandatory 
> > to implement DHT. Its something we've discussed in the 
> meeting and on 
> > the list. I think everyone agrees also that we need to allow for 
> > multiple DHT and all of the recent protocol proposals support it.
> >
> > understand the reason why IETF has mandatory-to-implement 
> > functionality in its protocols. The one and only reason is that it 
> > ensures interoperability. It ensures that, independent of 
> the optional 
> > features selected by a pair of entities, when you connect them 
> > together, they can still interoperate.
> >
> >   DHT besides the 'mandatory-to-implement' one has been 
> selected for 
> > the DHT, a new node not supporting that DHT will not be 
> able to join 
> > the ring even if it implements the mandatory-to-implement 
> one. Thus, 
> > the entire purpose of mandatory-to-implement is eliminated 
> - we don't 
> > Indeed, to get this kind of interoperability, we'd need to pick one 
> > and only one DHT that ever gets used with p2psip. I think 
> that is a mistake.
> > protocols and software written around P2P networks. I don't think 
> > anyone looks at an IETF protocol and says, "IETF are the 
> guys who know 
> > about DHTs, lets go with their recommendation". What IETF is really 
> > good at, and what this group has expertise in, is SIP, and on 
> > designing good, scalable protocols in general - things that have 
> > security, extensibility, good performance, and so on. So I 
> think the 
> > greatest value we can bring to the table is to create a 
> protocol that 
> > allows others to take DHTs and easily turn them into a real wire 
> > protocol that you can actually deploy, on the Internet, to 
> support SIP 
> > and ideally other things too. Interestingly, this is 
> exactly what all 
> > of the protocol proposals do. That kind of focus goes hand-in-hand 
> > with saying, let someone else figure out which DHT to use. 
> I think its 
> > perfectly reasonable for us, and for others, to write 
> documents on how 
> > to use our protocol with various DHTs. But I think we 
> should be out of 
> > the business of picking one or recommending one (or ones) 
> to be used.
> >
> > -Jonathan R.
> > --

---

From: "David A. Bryan" <[email-masked]>
Date: Sat, 14 Jul 2007 10:32:10 -0400

I think this is a bit short sighted. Skype works this way (only a
small fraction act as peers at a given time), but many proposals
beyond a replica for Skype have been proposed. The biggest gains in
peer to peer come when there are many peers to distribute the work, so
I can imagine that there will be cases where most entities will be
peers.

The replicate idea is interesting...seems somewhat analagous to the
migration between DHTs mechanism in ASP. I agree this sounds like it
could add significant overhead.

David

On 7/13/07, Henning Schulzrinne <[email-masked]> wrote:
> >> mandatory to
> >> functionality
> >> anyone

-- 
David A. Bryan
[email-masked]
+1.757.565.0101 x101
+1.757.565.0088 (fax)
www.SIPeerior.com

---

From: "David A. Bryan" <[email-masked]>
Date: Sat, 14 Jul 2007 11:03:23 -0400

I think there is some confusion here. See inline.

> DHTs are implemented ? In fact, isn't this how routing works ?

You won't be able to have various different DHTs share the same
overlay. I don't think anyone believes that will work. The protocol
will define how messages are transported, and the syntax to get/put
etc., but the particular DHT will decide specifics, such as where
something is stored, which areas a particular peer is responsible for,
and possibly some details of the routing. (For example the DHT
EpiChord improves chord performance by routing on request forward
around the ring and the other backward)

This flexability is great -- and why I think we want pluggable DHTs.
The field is still evolving, and new, better DHTs will be developed.
Once someone who is deploying a system chooses the one to use, only
those that can support it will be able to join, and I think that is
fine for the reasons you mention below (they may have picked a good
one for power saving, or optimized for an enterprise, or a global
deployment with many NATs...), but if their goal is "I want to be able
to use endpoints from any vendor", and we have no standard, it may not
be possible. Say vendor A implements DHTs 1 and 2, Vendor B 2 and 3,
and vendor C only 4. If I want to build an overlay and have devices
from all vendors, my choices are:

* Run a network on 2, A and B devices participate, but C devices run
only in client mode, which less efficient (especially if I have about
even numbers of A/B and C devices) It would be even worse if A, B and
C had no common DHT and perhaps I only get to spread the work among
1/3 of the devices.

* Run multiple overlays and connect via inter-overlay operations, but
this is also ugly, and likely to be less efficient since multiple
calls are made. To me the proposal that this is satisfactory would be
akin to saying when the IETF was looking at SIP that we shouldn't
bother, and that different vendors could just build VoIP systems with
their own protocols and we can define how to use the PSTN to
interconnect them. This seems destined to lead to a network where the
large vendors get to define an industry standard of "their" DHT,
patent it, license it, and control the market.

I'd prefer another option, where all could run 0. It might not be the
best DHT, but I can leverage the fact that all peers are sharing the
load now, which is the whole point of P2P. In my experience with DHTs
(and much of my Ph.D. thesis revolves around details of them, so I've
studied them rather intently), The improvement between DHT A from the
90s and new super DHT Z are relatively minor in comparison the the
performance gain of spreading the work among the various peers. That
is really what P2P is trying to do -- spread the work over as many
devices as possible to ensure each does little work, while
collectively they get the work done.

If we spread it over "just a few peers" and use client protocol
between the rest, this looks kind of like a degenerate case of a proxy
farm (or perhaps redirect farm) to me, where we aren't really
spreading the load. Spreading work among 1000 peers with a fairly
simply DHT seems preferable to having it spread across 400 marginally
more efficient ones that must serve another 600.

As Johnathan pointed out, there are a couple of models in the
requirements document, because it is the result of many authors. The
proxy model (which I personally don't like) and a true end to end
where as many devices are in the overlay as possible (which I have
always preferred and proposed) I want this to be as end to end as
possible, and want to ensure that someone setting up a network of
these can choose from any vendor. I'm also not sure there is a ton of
value in each vendor sending messages that are the same format over
the wire, but not being able to combine those devices together anyway
except as islands interconnected by SIP or as dumb clients where only
one vendor's device provide the smarts and stores the data.

So far, at least, I'm not convinced we can do that without a mandatory
DHT, although maybe in this discussion we'll figure it out.

David

-- 
David A. Bryan
[email-masked]
+1.757.565.0101 x101
+1.757.565.0088 (fax)
www.SIPeerior.com

---

From: "Wei Gengyu" <[email-masked]>
Date: Sun, 15 Jul 2007 00:53:49 +0800

Hi, 

I agree that there is a default DHT algorithm. 
And for flexibility, there may be an option for selecting other DHTs in Peer protocols.  

Some problem will occur:
1. How can a joining peer get what is the default DHT?
,It probably could get the configuation data by exchange messages during joining phase.

2. And whether the default DHT can be changed or reconfigured?
If yes, it needs to define mechanisms in Peer Protocol.

In practice, the default is required. 
Even we have the fault configurations, there are often interoperability problems due to different understanding or development choices.
And I can not image the result if there is no default DHT.

Regards, 

Gengyu

---

From: "Kundan Singh" <[email-masked]>
Date: Sat, 14 Jul 2007 12:40:26 -0700

Suppose we define a mandatory-to-implement DHT, say D0.

Now node N1 implements DHTs D0 and D1, and node N2 implements D0 and D2.
When N1 started, it decided to use DHT D1 and joined the existing DHT
with D1 (hence locked to D1).
When N2 started, it decided on D2 (hence locked to D2).

I suppose in this case we all agree that N1 and N2 can talk only via
client protocol. --- (A)

Essentially N1 and N2 are not interoperating using the peer protocol of
mandatory-to-implement D0.

Now in the example, to effectively utilize mandatory-to-implement DHT,
both N1 and N2 _MUST_ also join D0 using the peer protocol --- (B)

(Does anyone disagree with statement (B)?)

This means, for N1, the DHT maintenance operations must be done in D0 as
well as in D1, and any lookup operation must fallback to D0 if failed in
D1. Essentially, the performance gain of D1 is nullified and the
performance is limited to that of the algorithm in D0. This defeats the
purpose of let-the-best-DHT-win motto, and in practice *discourages*
implementing other DHTs besides default D0 since you don't gain
performance. (Which encourages implementations to become non-standard in
favor of better performance.)

I agree with Jonathan. I don't think we should limit innovation by
specifying a mandatory-to-implement DHT when we clearly don't know which
is the best DHT. Like Henning said, the client protocol can be used for
interconnecting.

(Talking about 'the best DHT', I think depending on the deployment
scenario different DHTs will perform differently. So one-size-fits-all
model may not be best for all the use cases we have. For example Bamboo
is proven to be good for DHT service hence probably good for low-churn
high-storage P2PSIP-proxies use case, but too many connections and low
keep-alives and data sync timers of Bamboo may not be good for
high-churn behind-the-NAT residential peers running on user laptops.)

---

From: "David Barrett" <[email-masked]>
Date: Sat, 14 Jul 2007 13:27:44 -0700

> From: Kundan Singh [mailto:[email-masked]]
> Subject: RE: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion
> 

If we can't standardize on a set of DHT messages, what *can* we standardize
on?

For something like TCP, so long as the packets over the wire are
standardized, the internal state machines and congestion-control algorithms
can vary on a per-endpoint basis (to some degree). 

Ideally we could find a similar approach: standardize the packet format such
that any two endpoints can talk in some form, but leave the algorithm
flexible such that even if each side uses a different DHT algorithm it still
somehow works (just like with TCP, we might be using different congestion
control algorithms, but we can still talk fine).

However, I don't know if such a thing is realistically possible because DHTs
require much greater cooperation: not only at the protocol level, but
between endpoints, and across the overlay as a whole.

Given this, I don't know if it's realistically possible to *not* standardize
on a given DHT implementation.  And as Kundan points out, merely saying "you
need to support at least X and optionally Y or Z" doesn't work unless
everybody always connects with at least X -- if you and I support X but only
connect with Y or Z, then the X DHT suffers.  (And if everybody connects
with at least X, then we've effectively standardized on that and why bother
with Y and Z?)

So I say we pick one and be done with it.  Yes, a better one will come
along, that's a certainty.  But let's cross that bridge when we get there,
and cross the bridge before us now.

-david

---

From: "Kundan Singh" <[email-masked]>
Date: Sat, 14 Jul 2007 14:28:47 -0700

> If we can't standardize on a set of DHT messages, what *can* 
> we standardize on?

Agreed. That's why the pluggable DHT approach is good. In particular,
the specification can define the common framework (similar to ASP
including on-the-wire message) but does not require that a particular
DHT algorithm must be implemented. Let the individual DHT algorithms
that confirm to the framework be defined separately. The implementators
can say 'this is a P2P SIP implementation that uses so-and-so DHT
algorithm'.

Now one can argue that if the DHT algorithms are different what is the
point of having a common peer protocol's message format because in
practice peers with different DHT algorithms may not be communicating
with each other using the peer protocol. One reason is that most of
these DHT protocols share common parts and the framework unifies the
common parts. Hence a new DHT algorithm specification does not have to
re-specify the common parts, if the common parts are abstracted in the
framework. Also, it allows migrating from one algorithm to another.

> So I say we pick one and be done with it.  Yes, a better one 
> will come along, that's a certainty.  But let's cross that 
> bridge when we get there, and cross the bridge before us now.

Yes. 
But this doesn't require a mandatory DHT in the specification. The
roadmap could be
(1) a P2P-SIP framework specification for common parts
(2) individual DHT algorithms that confirm to the interface in (1).

Let the implementors decide which DHT algorithm(s) they want to confirm
to. If two products implement different DHT algorithm, they are in
entirely different P2P overlay. Thus, an implementation claiming to be
P2P-SIP compliant must also specify the DHT algorithm(s) it implements.
The ASP draft also describes the migration strategy when a vendor wants
to change its DHT algorithm. 

I think it would be nice if P2P-SIP can handle DHT evolution without
enforcing backward compatibility with that one mandatory-to-implement
DHT.

---

From: David Barrett <[email-masked]>
Date: Sat, 14 Jul 2007 15:06:33 -0700

Ah, I agree with you here.  I like the goal of separating DHT-specific 
code from general shared code.

The one change I'd make to your proposal (as I understand it) is to 
first just pick a single DHT as if it were the only choice, build 
everything exactly around that DHT, and call that P2PSIP 1.0.  Then, 
after we've done it once, we'll have a very detailed understanding of 
its real-word issues and can thus figure out the right abstraction layer 
to enable pluggable DHTs.

But my fear is before doing the specific case once from top to bottom, 
it's premature to try to generalize for other types of DHTs.

-david

---

From: Henning Schulzrinne <[email-masked]>
Date: Sat, 14 Jul 2007 18:15:49 -0400

A number of proposals have moved beyond this stage and demonstrated,  
in running code or plausible description, that using a single  
maintenance and look-up protocol is quite feasible across multiple  
different DHTs. Naturally, there is no way to prove that every  
conceivable DHT that somebody may invent in the future can be handled  
by these protocols, but I see no reason to regress below working code  
and good reason to suspect that at least structured overlays can be  
maintained by these protocols if they are designed correctly, i.e.,  
without making unwarranted assumptions.

DHTs aren't magic. If we ignore the 'distributed' part, DHTs are  
essentially graphs, where vertices are network nodes. Just like a  
'graph maintenance protocol' within a host can build new data  
structures beyond, say, trees, it is very likely that the protocol  
proposals can maintain new distributed structures, as long as they  
follow that basic model. The protocol is really not much more than a  
pointer establishment mechanism, with lots of machinery to take into  
account that network 'pointers' aren't as nicely behaved as pointers  
in C/C++.

Henning

On Jul 14, 2007, at 6:06 PM, David Barrett wrote:

> Ah, I agree with you here.  I like the goal of separating DHT- 
> specific code from general shared code.
>
> everything exactly around that DHT, and call that P2PSIP 1.0.   
> Then, after we've done it once, we'll have a very detailed  
> understanding of its real-word issues and can thus figure out the  
> right abstraction layer to enable pluggable DHTs.
>
> But my fear is before doing the specific case once from top to  
> bottom, it's premature to try to generalize for other types of DHTs.
>
> -david
>

---

From: "Bruce Lowekamp" <[email-masked]>
Date: Sat, 14 Jul 2007 18:22:29 -0400

> support that DHT. This is something the market should drive, not us.

I think it's important that we carefully consider what use-cases we
care about and evaluate our decisions in those terms.  While I
understand your point, the "provider" use case is only one of many
(and we really need more work on use-cases).

The problem, as you point out, is that a default only helps if all of
the relevant peers are present at the time the overlay is formed (and
I haven't seen any proposals for a distributed consensus algorithm in
this group).  While I disagree with Henning's position that there will
be relatively few peers in the overlay (for many use cases), the
general concept that those nodes that don't speak the chosen DHT can
become clients is probably the right approach.

If we don't specify a mandatory DHT, then we run the risk that a large
percentage of the use-cases we have will only work with devices from
the same vendor.  I do hope that the market will drive solutions (and
that a larger percentage of these dht algorithm papers will move
beyond the paper stage) that lead to better DHTs being commonly
available, but I would hate for the first generation of devices to
fail to provide any level of interoperability.

Bruce

---

From: David Barrett <[email-masked]>
Date: Sat, 14 Jul 2007 15:52:24 -0700

I agree with Bruce: a top priority for P2PSIP 1.0 is to be highly 
compatible between multple vendor's implementations.  As such, choosing 
a single DHT seems like a sensible risk reduction -- even if later we 
decide to change it.

And I agree with Henning -- I'm sure there are ways to create an 
uber-DHT that is algorithm independent.  But it's harder and riskier, 
and I think that problem needn't be solved today.  I agree it's not 
magic, but there is a surprising amount of voodoo to get any of this 
working outside the lab.

Overall, I still think this quest for eternal agility and fear of 
commitment is keeping us from moving forward at all.  I'd happily give 
up some short-term flexibility if it meant improving the odds of getting 
*anything* working.  Problems are so much easier to solve when you 
actually have them, versus when we merely theorize them.

-david

---

From: Henning Schulzrinne <[email-masked]>
Date: Sat, 14 Jul 2007 19:00:27 -0400

I don't know what beyond actual proposals and running code you'd  
like. Fear-mongering and mentioning demons in the closet isn't going  
to help us. If you have concrete concerns about any of the proposals,  
please describe them. Otherwise, I don't find these generic "it's  
really, really difficult" statements all that helpful in making  
progress.

Henning

On Jul 14, 2007, at 6:52 PM, David Barrett wrote:

> compatible between multple vendor's implementations.  As such,  
> choosing a single DHT seems like a sensible risk reduction -- even  
> if later we decide to change it.
>
> uber-DHT that is algorithm independent.  But it's harder and  
> riskier, and I think that problem needn't be solved today.  I agree  
> it's not magic, but there is a surprising amount of voodoo to get  
> any of this working outside the lab.
>
> commitment is keeping us from moving forward at all.  I'd happily  
> give up some short-term flexibility if it meant improving the odds  
> of getting *anything* working.  Problems are so much easier to  
> solve when you actually have them, versus when we merely theorize  
> them.
>
> -david
>

---

From: "David Barrett" <[email-masked]>
Date: Sat, 14 Jul 2007 16:41:09 -0700

I want running code on a large scale, in the real world, outside the lab.
It doesn't matter what algorithm it uses, how the packets are laid out, etc.
I just want something that solves a real problem for some relevant set of
users, and then I want to iterate.

Thus I have "concrete concerns" with any proposal that deviates from this
goal.  Adding more flexibility to a nonexistent system isn't useful.  Paving
the way for experimental algorithms that might exist on paper or on a few
buddies' computers isn't useful.  Planning for esoteric networking
environments with exceedingly difficult constraints isn't useful.  The only
thing that's useful is running code at a meaningful scale, instrumented in
such a fashion to learn if it works and what needs fixing.

I understand that it might not be exciting to hear my proposals.  "Blah
blah, reality, blah blah, actual users."  It's always more fun to add more
features and more flexibility, but it's not useful (for any meaningful
definition of "useful" that is somehow tied to actual users communicating).

I respect we all come to this forum with our own experiences.  And I'll
grant that others might have experienced success with alternate strategies.
But in my experience developing commercial P2P applications deployed on
hundreds of thousands of active computers around the world, until you have
something that actually does what you want to do -- even if it doesn't do it
great -- any distraction from this goal isn't useful.

So again, I say we pick the best DHT available today, build around that to
the exclusion of all others, and actually ship a product that people can
use.  Then, once people are using it, we'll know what works and what
doesn't, and then we can focus our attention accordingly.

I understand and respect there are other proposals, but this is mine.

-david

> From: Henning Schulzrinne [mailto:[email-masked]]
> Sent: Saturday, July 14, 2007 4:00 PM
> To: David Barrett
> Cc: P2PSIP WG

---

From: "Eunsoo Shim" <[email-masked]>
Date: Sat, 14 Jul 2007 23:43:13 -0400

I agree that it should be the market that drives the choice of the DHT
algorithm eventually.

On the other hand, currently no specific DHT algorithm has been established
as a widely accepted one for P2PSIP overlays. So I am wondering what DHT
algorithms a manufacturer would implement if they implement P2PSIP nodes.
And if manufacturers choose all kinds of variations of various DHT
algorithms, it will give significant headache and limited choices of devices
to even large service providers, not to mention smaller ones and enterprises
who like to use multi-vendor solutions.

The SIP community did not have to recommend or choose any voice codec. Other
organizations developed various voice codec standards and there were widely
accepted voice codecs such as G.711. So it was not much of an issue what
voice codec should be implemented or selected for SIP devices. The situation
with DHT is different in my understanding. There are too many options and
one can easily try to tweak a known DHT algorithm.

I am not aware of any standardization body or industry organization that is
working to define any standard DHT.

I believe at least some kind of recommendation on DHT algorithms, if not a
mandatory implementation requirement, would help a lot.

If the P2PSIP community does not produce it for P2PSIP overlays, who would
do it?

Thanks.

Eunsoo

---

From: "Eunsoo Shim" <[email-masked]>
Date: Sat, 14 Jul 2007 23:54:46 -0400

> of nodes need to act as peers at any given time, this may not be much

I am not sure whether this is a desirable approach.
Having a commond DHT and thus being able to use more devices as full peers
seems to be more desirable to me.

> "fancy"/"better" one less universally spoken.

I am not sure whether it is always the case that, given two DHT algorithms,
one can be considered as a baseline universal one and the other a
fancier/better one.

This incurs
> reachability cases particularly as new nodes are added, but it could

I am not convinced why we would want to pursue this complex approach instead
of choosing/recommending one common DHT.
We can change the common DHT or add one more later.

Thanks.

Eunsoo

Henning

---

From: "Eunsoo Shim" <[email-masked]>
Date: Sat, 14 Jul 2007 23:58:21 -0400

You made a good point that choosing a common DHT algorithm does not
guarantee automatic interoperation among various devices.
However, it still provides a much better situation than where
developers/manufacturers implement arbitrary sets of arbitrary DHT
algorithms. In the former situation, at least one can configure devices to
use the common algorithm. In the latter situation, interoperation may not be
able to be achieved at all.
Thanks.

Eunsoo

---

From: "Roy, Radhika R." <[email-masked]>
Date: Sun, 15 Jul 2007 00:01:48 -0400

The example of G.711 codec is NOT exactly right here although some aspects of this are applicable in the DHT argument. A couple of things are as follows:

1.	
	All codecs including G.711 use the same SIP signaling protocol
2.	
	G.711 codec has been used as a reference for all other codecs for interoperability purpose. For example, if audio bridging with multiple different codecs are done, then all of them need to be converted into a single common format, and G.711 is used as a reference common format.

So, if we analyze items 1 and 2 very carefully, it again proves the fact, as David pointed out, we need one standard thing as a reference to start with as follows:

*	Only "one" reference standard for the signaling protocol (in this case it is SIP) to provide interoperability
*	Only "one" reference standard codec that will offer a reference standard format that will be used for inter-operability among different codecs

As such, what is the lesson from the above from standardization point of view when we apply those experiences in DHT standardization?

Is it not what David has pointed out?

Best regards,

Radhika

________________________________

From: [email-masked] on behalf of Eunsoo Shim
Sent: Sat 7/14/2007 11:43 PM
To: Jonathan Rosenberg
Cc: P2PSIP WG
Subject: Re: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion

I agree that it should be the market that drives the choice of the DHT algorithm eventually.

On the other hand, currently no specific DHT algorithm has been established as a widely accepted one for P2PSIP overlays. So I am wondering what DHT algorithms a manufacturer would implement if they implement P2PSIP nodes. And if manufacturers choose all kinds of variations of various DHT algorithms, it will give significant headache and limited choices of devices to even large service providers, not to mention smaller ones and enterprises who like to use multi-vendor solutions. 

The SIP community did not have to recommend or choose any voice codec. Other organizations developed various voice codec standards and there were widely accepted voice codecs such as G.711. So it was not much of an issue what voice codec should be implemented or selected for SIP devices. The situation with DHT is different in my understanding. There are too many options and one can easily try to tweak a known DHT algorithm. 

I am not aware of any standardization body or industry organization that is working to define any standard DHT. 

I believe at least some kind of recommendation on DHT algorithms, if not a mandatory implementation requirement, would help a lot. 

If the P2PSIP community does not produce it for P2PSIP overlays, who would do it?

Thanks.

Eunsoo

	

	

	
	
	-Jonathan R.
	--
	Jonathan D. Rosenberg, Ph.D.                   600 Lanidex Plaza
	Cisco Fellow                                   Parsippany, NJ 07054-2711 
	Cisco Systems
	[email-masked]                              FAX:   (973) 952-5050
	http://www.jdrosen.net <http://www.jdrosen.net/>                          PHONE: (973) 952-5000 
	http://www.cisco.com <http://www.cisco.com/>

---

From: "Roy, Radhika R." <[email-masked]>
Date: Sun, 15 Jul 2007 00:02:22 -0400

You are right Eunsoo as David also has made the same point. - Radhika

From: [email-masked] on behalf of Eunsoo Shim
Sent: Sat 7/14/2007 11:58 PM
To: Kundan Singh
Cc: P2PSIP WG
Subject: Re: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion

You made a good point that choosing a common DHT algorithm does not guarantee automatic interoperation among various devices.
However, it still provides a much better situation than where developers/manufacturers implement arbitrary sets of arbitrary DHT algorithms. In the former situation, at least one can configure devices to use the common algorithm. In the latter situation, interoperation may not be able to be achieved at all. 
Thanks.

Eunsoo

---

From: "Henry Sinnreich" <[email-masked]>
Date: Sun, 15 Jul 2007 15:52:13 -0700

Henning Schulzrinne wrote:

>it just can't be a full peer.

Can we interpret this as an endorsement of the client protocol and/or
another argument in its favor?

Thanks, Henry

-----Original Message-----
From: Henning Schulzrinne [mailto:[email-masked]] 
Sent: Friday, July 13, 2007 5:02 PM
To: David A. Bryan
Cc: P2PSIP WG
Subject: Re: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion

Henning

---

From: Henning Schulzrinne <[email-masked]>
Date: Sun, 15 Jul 2007 19:19:16 -0400

> another argument in its favor?

I admit not having followed the discussion in detail, so I'm hesitant  
to answer, but I suspect, yes. My feeling is that the client protocol  
can probably be structured as a proper subset of the peer maintenance  
protocol.

I'll also note in passing that, so far, I have not found it helpful  
to restrict the DHT in designing P2PP. In other words, if the WG  
charter said "Thou shalt use foobar DHT only", I don't think the  
protocol would have been much simpler or different, except maybe for  
a small part of the discovery/introspection features. I'd be curious  
if designers of other protocol proposals have had different experiences.

Henning

---

From: "David A. Bryan" <[email-masked]>
Date: Sun, 15 Jul 2007 21:11:49 -0400

> if designers of other protocol proposals have had different experiences.

Agree that restricting to one DHT will not make the protocol
substantially simpler (in both dSIP and RELOAD we had multiple DHTs,
and other than using somewhat more generic labels for fields, I don't
think picking a particular DHT would make that easier.

In some ways, that is my worry about not defining a must implement
DHT...it is pretty easy to use the protocols to encode any DHT one
likes, so if we don't define one, expect each and every vendor to use
a different, non-interoperable one (at least until market strength and
a patent from one of the big players forces a default implement
anyway, encumbered by a license fee at that point...)

David

-- 
David A. Bryan
[email-masked]
+1.757.565.0101 x101
+1.757.565.0088 (fax)
www.SIPeerior.com

---

From: "David Barrett" <[email-masked]>
Date: Sun, 15 Jul 2007 18:20:00 -0700

> think picking a particular DHT would make that easier.

I agree the messages on the wire are probably about the same irrespective of
the DHT algorithm.  But would you agree that standardizing on a particular
algorithm would make things easier?

For example, if I'm using Kademlia and you're using Chord -- even we're
using compatible packet formats -- won't that create a dysfunctional overlay
network?

-david

---

From: "David A. Bryan" <[email-masked]>
Date: Sun, 15 Jul 2007 21:31:20 -0400

Um, I guess I wasn't clear. Sorry. That was precisely the point I was
making, particularly in my second paragraph below (and did in the
email response to Johnathan that started this thread)

I was trying to say that while valid, Henning's point that the
protocol won't be different if we pick a DHT is irrelevant to the
argument about whether to choose a must implement or not. If we don't
have at least one mandatory to implement, we will have islands of each
vendor's device not working together, and worse, may be forced into
using one specified by a player with large market share that might
have nasty strings attached.

So to again make my position clear:

We should allow modular DHTs.
We should specify one or a small number of must implement for compatibility.

That said, I do agree with the point Henning made that if we picked
only one DHT and didn't allow modularity (which I don't think anyone
is really suggesting, anyway), it wouldn't really significantly
simplify the protocol. I think Henning's argument had more to do with
why we should allow modular DHTs than anything about if a mandatory is
a good idea or not.

David

-- 
David A. Bryan
[email-masked]
+1.757.565.0101 x101
+1.757.565.0088 (fax)
www.SIPeerior.com

---

From: Henning Schulzrinne <[email-masked]>
Date: Sun, 15 Jul 2007 21:37:23 -0400

One way to deal with this is to have a separate document that defines  
"how to use P2PSIP protocol X for DHT Y". This will likely bias the  
implementor choice.

I have to admit that I'm torn between several arguments on this. For  
once, my perception is that the difference in performance, once you  
subtract out the DHT-independent optimizations, isn't all that large,  
at least among the core set of DHTs discussed here (Chord, CAN,  
Kademlia and kin). I also suspect that implementing several of them  
isn't all that hard. Just like today's voice clients typically  
implement a handful of codecs, at least the non-encumbered ones, I'm  
guessing that almost all P2P clients will implement several of the  
well-known, non-encumbered DHTs, making the problem of  
interoperability far less significant.  This doesn't guarantee  
interoperability, but anything short of prohibiting building clients  
that support other than the default set won't. As Jonathan pointed  
out, just defining a must-implement DHT doesn't help a whole lot as  
long as a group of clients can choose something else and thus lock  
out the plebes.

---

From: "David Barrett" <[email-masked]>
Date: Sun, 15 Jul 2007 18:42:30 -0700

> From: David A. Bryan

Got it.

> simplify the protocol.

Well, I'd suggest that before we've done it once for real, we probably don't
know enough to make smart modularity decisions.  If we just picked one DHT
and focused on nothing but that, we stand the highest chance of getting
*something* working in short order for P2PSIP 1.0.  Then I expect we'd be in
a good position to make modularization decisions for P2PSIP 2.0.

But regardless, I'm with you on the need for picking a baseline DHT and
focusing in that -- I'm just a bit more aggressive on scope reductions. :)

-david

---

From: "Bruce Lowekamp" <[email-masked]>
Date: Sun, 15 Jul 2007 22:16:17 -0400

> a good position to make modularization decisions for P2PSIP 2.0.

Nearly all of the current peer protocol proposals support pluggable
DHTs.  dSIP actually had two separate drafts to demonstrate how it
could work.  Marcia didn't have time to rewrite them for RELOAD, but
essentially it's just a matter of providing attribute numbers  for
what were headers in dSIP.  P2PP supports different DHTs and actually
has tried to even support unstructured overlays.

Given that there are specific, concrete proposals out already, do you
have specific criticisms or concerns after reading the drafts?

Bruce

---

From: David Barrett <[email-masked]>
Date: Sun, 15 Jul 2007 20:50:32 -0700

What I read in the P2PSIP charter is we commit to one DHT while leaving 
the option open for more.  I interpret that to mean we pick one DHT now 
for certain, and at some undefined point in the future we determine how 
to generalize that.  In other words, I feel a modular DHT system isn't 
strictly required at this time, and anything not strictly required 
shouldn't be discussed.

So I've got nothing against the drafts.  I just think they solve a 
problem we don't need to solve today, and thus shouldn't be.

Perhaps it would help if we began talking in terms of versions: I think 
P2PSIP v1.0 should take an absolute minimalist interpretation of the 
fewest, most crucial features necessary to have something that works in 
the most constrained possible way.  Everything else -- even discussion 
about anything else -- should be deferred until v1.0 is done, and 
modular DHTs fits in that category for me.

Do you agree we should take a minimalist, iterative interpretation of 
scope?  Do you feel modular DHTs absolutely must be in the first 
iteration?

-david

---

From: Ted Hardie <[email-masked]>
Date: Sun, 15 Jul 2007 22:07:31 -0700

At 9:31 PM -0400 7/15/07, David A. Bryan wrote:
>is really suggesting, anyway)

I'm really glad to see this, as your earlier statements looked like
they were saying just that.  Designing for one to the exclusion of
others certainly sounded like that to my ears.

The point I think that is missing here in much of this discussion is that
the agility-with-a-default means, in essence, that the default will be
around forever.  Some set of overlays will be created with it and some
set of devices/programs will be written such that only the default is
supported.  The protocol is agile, but the programs aren't, and the justification
in this case is the standard.  Those of us coming from other areas have
seen a lot of cruft piled on cruft to deal with installed base in those cases.
Coming from that background, this decision sounds like a very short-term
win, with a long term cruft penalty that will keep biting for a long, long
time. 

Yes, the vendor lock-in problem is real (where Vendor A does Chord and vendor B does  Kademlia), but unless Vendor A pics a DHT which is not available to B for
implementation, the answer to that emerges quickly.  B codes Chord and
eliminates A's lock-in; A must either then code Kademlia or lose clients to B.

I think that the working group should consider language that says "for interop
testing required to reach Draft, the group will use DHT X".  Anybody who wants
to play right now will implement X.  That's a useful thing to do, without any
lock-in at the protocol level that will haunt us later.

Still the weekend, so no new cents have come in.  Please accept my,
				regards,
					Ted Hardie

---

From: Ted Hardie <[email-masked]>
Date: Sun, 15 Jul 2007 22:11:19 -0700

Just realized that I probably had a confusion of Davids in the message below; my
apologies for attributing some of David Barrett's sentiments to David Bryan.

			Ted

---

From: "He, Jingtong" <[email-masked]>
Date: Mon, 16 Jul 2007 17:25:11 +0800

Sorry to interrupt!
I am wondering if we are designing a widely-used internet protocol
once-and-for-all and update it (if necessary) when we meet with new
problems/requirements. Or we are designing the first version of software,
just using minimal methods to satisfy minimal requirements, and update/patch
it year-by-year.

--
Regards,
Jingtong
>From: David Barrett [mailto:[email-masked]]
>Sent: Monday, July 16, 2007 11:51 AM
>To: Bruce Lowekamp
>Cc: Henry Sinnreich; P2PSIP WG

---

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 16 Jul 2007 06:59:15 -0400

The IETF experience with trying to do a version 2.0 of anything has  
not been good. We generally do a RFCXYZbis, which fix glaring  
problems in RFCXYZ, but the core functionality remains the same. See  
SMTP, SIP and HTTP, for starters.

Thus, I disagree with your conclusion; this is my last post on the  
subject, since this is a largely theoretical debate. As mentioned,  
everybody who actually does protocol work in the working group has  
decided that modularity comes at almost no cost. I don't see a reason  
to be stifled by FUD. You are obviously free to propose solutions  
that satisfy your artificial constraints; the working group can then  
decide which ones it likes better.

Henning

On Jul 15, 2007, at 11:50 PM, David Barrett wrote:

> What I read in the P2PSIP charter is we commit to one DHT while  
> leaving the option open for more.  I interpret that to mean we pick  
> one DHT now for certain, and at some undefined point in the future  
> we determine how to generalize that.  In other words, I feel a  
> modular DHT system isn't strictly required at this time, and  
> anything not strictly required shouldn't be discussed.
>
> Perhaps it would help if we began talking in terms of versions: I  
> think P2PSIP v1.0 should take an absolute minimalist interpretation  
> of the fewest, most crucial features necessary to have something  
> that works in the most constrained possible way.  Everything else  
> -- even discussion about anything else -- should be deferred until  
> v1.0 is done, and modular DHTs fits in that category for me.
>
> Do you agree we should take a minimalist, iterative interpretation  
> of scope?  Do you feel modular DHTs absolutely must be in the first  
> iteration?
>
> -david
>

---

From: "David Barrett" <[email-masked]>
Date: Mon, 16 Jul 2007 12:13:43 -0700

Got it, this is useful to know: it's really a question of which design
process we're using.  You're proposing a standard waterfall process:

1) Design everything you'll ever need
2) Build it
3) Patch if necessary

I'm proposing an iterative process:

1) Design the minimal set of features necessary to do minimal requirements
2) Build that and nothing else
3) Update design based on real-world lessons
4) Repeat again and again forever

So, that'd explain why any attempt to reduce scope must sound so terrifying
because you're assuming we only have one shot to get it right.  Totally
makes sense.

Where does everyone else fall?  Shall we take the one-shot waterfall
approach, or iterative?

-david
 
> Sent: Monday, July 16, 2007 3:59 AM
> To: David Barrett
> Cc: Bruce Lowekamp; Henry Sinnreich; P2PSIP WG

---

From: "Henry Sinnreich" <[email-masked]>
Date: Mon, 16 Jul 2007 13:12:03 -0700

David,

You have gotten it almost right here :-), except the following steps are
missing:

1 Publish the code to get cooperation, 
2 Use and/or develop simulation tools, 
3 Run simulations,
4 Run the protocol over the Internet,
5 Report the test results
6 Go back to step 2.

If the code is a commercial product, then at least steps 2-6 should be
implemented.

Your companion message to the BEHAVE WG is an excellent role model:

========
I'm dealing with measuring peer connectivity of my own system (non-ICE),
and here are some of the results for various NAT types:

Type        Count       Connectivity Open Time
Fullcone    150 (9%)    92%          1056ms
Fullstatic  397 (25%)   94%          867ms
Fw          68  (4%)    2%           374ms
Rstrcone    598 (37%)   89%          706ms
Rstrstatic  82 ( 5%)    91%          769ms
symmetric	293 (18%)   94%          906ms
=======

How is anyone supposed to spend resources on a paper design only?
This includes our P2P SIP WG where paper designs seem already in
oversupply.

Henry

-----Original Message-----
From: David Barrett [mailto:[email-masked]] 
Sent: Monday, July 16, 2007 12:14 PM
To: 'Henning Schulzrinne'
Cc: Henry Sinnreich; 'P2PSIP WG'
Subject: RE: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion

1) Design the minimal set of features necessary to do minimal
requirements
2) Build that and nothing else
3) Update design based on real-world lessons
4) Repeat again and again forever

So, that'd explain why any attempt to reduce scope must sound so
terrifying
because you're assuming we only have one shot to get it right.  Totally
makes sense.

-david

---

From: "David Barrett" <[email-masked]>
Date: Mon, 16 Jul 2007 13:23:03 -0700

So if I read you right, it sounds like you vote for:

- MORE real-world iteration
- LESS paper design.

Is that right?  If so, I am in complete agreement with you.

Anybody else?

-david

> From: Henry Sinnreich [mailto:[email-masked]]
> Sent: Monday, July 16, 2007 1:12 PM
> To: David Barrett
> Cc: P2PSIP WG; Henning Schulzrinne

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 16 Jul 2007 22:06:20 -0400

Henning,

Just to make sure I understand - I clearly agree that a single protocol 
can support structured overlay maintenance for large classes of DHTs 
without (much) new protocol machinery for each new DHT. However, I do 
not think it is possible that, within a single overlay, you have 
different clients using different DHTs with the same protocol. In other 
words, some clients run Kademlia, and the others run Chord, all within 
the same DHT. Do you agree?

-Jonathan R.

Henning Schulzrinne wrote:
> A number of proposals have moved beyond this stage and demonstrated, in 
> running code or plausible description, that using a single maintenance 
> and look-up protocol is quite feasible across multiple different DHTs. 
> Naturally, there is no way to prove that every conceivable DHT that 
> somebody may invent in the future can be handled by these protocols, but 
> I see no reason to regress below working code and good reason to suspect 
> that at least structured overlays can be maintained by these protocols 
> if they are designed correctly, i.e., without making unwarranted 
> assumptions.
> 
> essentially graphs, where vertices are network nodes. Just like a 'graph 
> maintenance protocol' within a host can build new data structures 
> beyond, say, trees, it is very likely that the protocol proposals can 
> maintain new distributed structures, as long as they follow that basic 
> model. The protocol is really not much more than a pointer establishment 
> mechanism, with lots of machinery to take into account that network 
> 'pointers' aren't as nicely behaved as pointers in C/C++.
> 
> Henning
> 
>> its real-word issues and can thus figure out the right abstraction 
>> layer to enable pluggable DHTs.
>>

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 16 Jul 2007 22:10:29 -0400

> have nasty strings attached.

Careful here. The choice is really between one of the following two:

1. we don't pick any must-implement DHT; each 'provider' mandates a 
specific one. We can write specs (and others can too) for how to use the 
protocol for various DHT. But this is a market-pressure decision.

2. because the DHT is set when the fist node joins and then basically 
lasts a really long time, you end up with all p2p networks using the one 
and only mandatory-to-implement DHT. In that case, you haven't really 
defined a mandatory-to-implement one, as you've picked the one-and-only 
DHT for p2psip.

-Jonathan R.

---

From: Jonathan Rosenberg <[email-masked]>
Date: Mon, 16 Jul 2007 22:15:21 -0400

inline.

> win, with a long term cruft penalty that will keep biting for a long, long

This is a problem I am also very worried about. Unfortunately, it can't 
be solved by mandating DHT or picking a mandatory-to-implement one or 
anything like that. The only way to solve this is by building into the 
protocol, from day one, standards on how one forms parallel p2p 
networks, one using a new DHT and one using the old, and how you 
gradually (over months perhaps) migrate in a coordinated way. This was 
one of the features in our proposal.

Providing this as a core part of the protocol design is one of the real 
values that IETF can bring to the table. If the protocol, independent of 
the DHT, doesn't provide for things like security, extensibility and 
migration, nat traversal, and so on, its just a bit-format. I personally 
don't see a lot of value in standardizing something that is nothing more 
than an encoding format for real protocols that someone else defines.

-Jonathan R.

---

From: Henning Schulzrinne <[email-masked]>
Date: Mon, 16 Jul 2007 22:24:22 -0400

Agreed; a DHT "mash up" isn't likely going to work.

On Jul 16, 2007, at 10:06 PM, Jonathan Rosenberg wrote:

> Henning,
>
> Just to make sure I understand - I clearly agree that a single  
> protocol can support structured overlay maintenance for large  
> classes of DHTs without (much) new protocol machinery for each new  
> DHT. However, I do not think it is possible that, within a single  
> overlay, you have different clients using different DHTs with the  
> same protocol. In other words, some clients run Kademlia, and the  
> others run Chord, all within the same DHT. Do you agree?
>
> -Jonathan R.
>
>> A number of proposals have moved beyond this stage and  
>> demonstrated, in running code or plausible description, that using  
>> a single maintenance and look-up protocol is quite feasible across  
>> multiple different DHTs. Naturally, there is no way to prove that  
>> every conceivable DHT that somebody may invent in the future can  
>> be handled by these protocols, but I see no reason to regress  
>> below working code and good reason to suspect that at least  
>> structured overlays can be maintained by these protocols if they  
>> are designed correctly, i.e., without making unwarranted assumptions.
>> follow that basic model. The protocol is really not much more than  
>> a pointer establishment mechanism, with lots of machinery to take  
>> into account that network 'pointers' aren't as nicely behaved as  
>> pointers in C/C++.
>> Henning
>>> The one change I'd make to your proposal (as I understand it) is  
>>> to first just pick a single DHT as if it were the only choice,  
>>> build everything exactly around that DHT, and call that P2PSIP  
>>> 1.0.  Then, after we've done it once, we'll have a very detailed

---

From: "Bruce Lowekamp" <[email-masked]>
Date: Mon, 16 Jul 2007 22:57:19 -0400

> protocol for various DHT. But this is a market-pressure decision.

Consider the "Managed, Private Networks" in Section 3.4 of
draft-bryan-p2psip-usecases-00.  There isn't a "provider" in any of
these cases.  There may be multiple equipment vendors, in which case
we'll probably have the same situations as we do now with audio
codecs.  (Although since the negotiation isn't pairwise, there will
probably be a greater affinity for the base, but there needs to be a
base to start with so that equipment from Cisco, Avaya, and Polycom
can interoperate.)

> defined a mandatory-to-implement one, as you've picked the one-and-only

I don't think this is really a significant concern.  While it is
listed as a usecase (3.1.2), I disbelieve that there will be a
large-scale open p2p network that will be formed starting with one
individual user and then last forever.  In this case, there will
likely be providers (or at least an enrollment server), which implies
that whoever is responsible for administering the overlay will make
the choice based on market-pressure.

In the first case, I think we need a mandatory DHT to ensure interop.
 In the second case, since there's a central authority, I don't think
the mandatory DHT is relevant.

Bruce

---

From: "Roy, Radhika R." <[email-masked]>
Date: Mon, 16 Jul 2007 23:00:01 -0400

Folks:
 
>From recent Jonathan's emails, it appears as follows:

*	
	For each DHT, there will be one DHT-specific P2PSIP overlay network, but one single common P2PSIP protocol.
*	
	That is, all DHT-specific P2PSIP overlay networks will use a single common P2PSIP protocol.

Is this what P2PSIP WG wants to do?

Is it the best interest of P2PSIP standardization? 

Will it not cause problems, as David pointed out and Jonathan also agreed, the islands of "non-interworking" P2PSIP overlay networks negating the most benefits of standardization?

Can anyone summarize at a very high-level what values the very "standardization" of so-called P2PSIP protocol will bring to the table with non-standardized DHT?

Best regards,

Radhika

From: [email-masked] on behalf of Roy, Radhika R.
Sent: Sun 7/15/2007 12:01 AM
To: Eunsoo Shim; Jonathan Rosenberg
Cc: P2PSIP WG
Subject: RE: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion

1.	

Best regards,

Radhika

---

From: "Roy, Radhika R." <[email-masked]>
Date: Mon, 16 Jul 2007 23:00:28 -0400

I also agree with Bruce - Radhika

From: [email-masked] on behalf of Bruce Lowekamp
Sent: Mon 7/16/2007 10:57 PM
To: Jonathan Rosenberg
Cc: Henry Sinnreich; P2PSIP WG
Subject: Re: [P2PSIP] Mandatory-to-implement DHTs - a dissenting opinion

Bruce

---

From: "Brian Rosen" <[email-masked]>
Date: Wed, 18 Jul 2007 10:53:02 -0400

The whole "pluggable DHTs" is not a protocol issue at all, it's an
implementation issue.  It affects how the code on one peer is written, not
how peers interact.  

I look at the question pretty simply: if I build an implementation and you
build an implementation they will interoperate if we have a common DHT and
they won't if they don't.  It does not bother me AT ALL that it locks in a
particular DHT.  I think that problem is so far down the list of worries,
that it should not enter into the discussion.  I think the problem of any 10
implementations not having a DHT in common is not only likely, but
guaranteed unless there is a mandatory-to-implement DHT.

I think a particular overlay could choose another DHT knowing that not all
implementations would work.  No problem.  If an overlay wants all
implementations to work, they pick the mandatory-to-implement one.  No
problem.  Especially in an ad-hoc case, at least for the use cases I'm
interested in, it is vital that every device works.  I need a choice that I
know will work.

The only cost to a mandatory to implement DHT is the code that implements
it.  If you all are into pluggable, it's probably common open source code.
This means a tradeoff of universal compatibility WHEN YOU WANT IT for some
test time.  That's a no-brainer to me.

Brian 

> Sent: Monday, July 16, 2007 10:24 PM
> To: Jonathan Rosenberg
> Cc: P2PSIP WG

---

From: Ted Hardie <[email-masked]>
Date: Wed, 18 Jul 2007 17:00:35 -0700

At 10:53 AM -0400 7/18/07, Brian Rosen wrote:

<Snip>

>It does not bother me AT ALL that it locks in a
>that it should not enter into the discussion.

<snip>

> If an overlay wants all
>problem. 

(I'm assuming your opinion here is as a technical contributor, and that you
are not ruling as chair that this should not enter into WG discussion)

You've just outlined the scenario that means whatever baseline is
selected as mandatory to implement *right now* will be the basis
of most DHTs from now on.  Anyone wanting to support arbitrary
clients chooses that one.  Any code or protocol work done to
enable arbitrary clients using different overlay strategies to interwork will
get exercised rarely, since forming a new one with the old, tried
and true baseline is guaranteed to work.  Put another way, any limitation we
have now in whatever DHT gets selected  is kept pretty much forever.

I invite you to go look at the discussion in EAI for a moment, and see how much
trouble it has cost the email community (or the DRUMS archives, or lots of
other places).  Keeping compatibility with systems that were near end-of-life
at the time 822 was written has cost code complexity, operational complexity,
and limited the end users.  And that was for a relatively trivial problem--the
number of bits being 7 or 8.  The rate of evolution in *this* area is high enough
that we don't even have a decent guess at what the costs of maintaining
this backward compatibility will be.

Selecting  a mandatory to implement is a very short-sighted choice;
it may increase the uptake in the early days, but it will cost you pain
in the long run.  If the other popular IETF protocols are any guide, it
can be a pretty significant amount of pain.

I have suggested a compromise in which the WG names a DHT for
use in the interop testing which will lead to Draft.  I think that will
give a nice, large pool of clients that support the same DHT.  Anyone
expecting to form a large or multi-client overlay would be able to pick
from that pool and deploy that DHT.  But it does not bake into the clay of the
protocol an overlay strategy that we know is likely to be surpassed in the
future.

			regards,
				Ted Hardie

---

From: "Brian Rosen" <[email-masked]>
Date: Thu, 19 Jul 2007 15:18:05 -0400

<as individual> (sorry I did not make that clear in my initial message)

I disagree. 

If the spec says DHTs can change, plan on it, implement at least this one
for now, then later, we can add another mandatory to implement one.

The cost is the old code.  No big deal.

Brian

> From: Ted Hardie [mailto:[email-masked]]
> Sent: Wednesday, July 18, 2007 8:01 PM
> To: Brian Rosen; 'Henning Schulzrinne'; 'Jonathan Rosenberg'
> Cc: 'P2PSIP WG'

---

From: Ted Hardie <[email-masked]>
Date: Thu, 19 Jul 2007 12:37:32 -0700

>for now, then later, we can add another mandatory to implement one.

Add or replace?  That is, do you expect to be able to deprecate the original
mandatory-to-implement, or add one such that there are a minimum of two?

>The cost is the old code.  No big deal.

On this point, we really do disagree.  Old code that must be exercisable
has considerable costs.
			Ted

---

From: "Brian Rosen" <[email-masked]>
Date: Thu, 19 Jul 2007 16:01:54 -0400

I wouldn't mind deprecate, although I doubt you would see it in practice.

I think the cost of maintaining an old DHT algorithm is small enough to be
ignorable, but I can agree to disagree on that point.

Brian

> Sent: Thursday, July 19, 2007 3:38 PM
> >> enable arbitrary clients using different overlay strategies to
> interwork
> >> will
> >> and true baseline is guaranteed to work.  Put another way, any
> limitation
> >> we
> >> I invite you to go look at the discussion in EAI for a moment, and see
> how
> >> much
> >> other places).  Keeping compatibility with systems that were near end-
> of-
> >> life
> >> and limited the end users.  And that was for a relatively trivial
> problem-
> >> -the
> >> number of bits being 7 or 8.  The rate of evolution in *this* area is
> high
> >> enough
> >> from that pool and deploy that DHT.  But it does not bake into the clay
> of
> >> the

---

From: "Bill Mccormick" <[email-masked]>
Date: Fri, 20 Jul 2007 09:35:15 -0400

I agree with Ted.   Maintaining working versions of old protocols is
expensive and shouldn't be undertaken lightly.

Bill.

-- 
Bill McCormick