Subject: [P2Prg] Dynamic Networking

From: Justin Chapweske <[email-masked]>
Date: Thu, 22 Apr 2004 13:47:15 -0500

> I have absolutely no problem with HTTP, being one of the "team of two
> plus a
> few" who released Tomcat to the world in '99. The problem I have with
> the
> HTTP application programming model is entirely in the realm of
> deployment.
> Server as largely assumed to be reachible via static addresses, IP. My
> interests
> are in "everything is a server, everthing is a client" such that
> servers can
> be instantiated, provissioned, and reclaimed dynamically w/ no harm
> and in
> fact "to great advantage" to the application network. In this model
> the app
> developer is literally constrained as to what the deployment network
> looks
> like. With JXTA, the app developer *owns* the network, can change it
> as the
> need arises, etc. BTW, this is the very epiphany I had that clued me
> in that
> there had to be a better way moving forward.

I'm CC'ing this message to decentralization and p2p-hackers to engage
the larger community on this interesting topic.

James, I heartily agree with your goal of transcending the limitations
of static addresses and allowing the network to be much more dynamic.

However, I think we have different approaches for achieving this goal.

My approach is to treat dynamic networking as a naming or URI problem. 
JXTA appears to take the approach that dynamic networking is a routing
or messaging problem.

Now, there are multiple levels at which one can solve dynamic networking
from a naming perspective.  The first level corresponds to solving the
problem at the IP level using multicast and anycast addresses.  A
wonderful example of this is zeroconf/rendezvous.  By allowing URIs/IP
addresses to be resolved to any valid machine on the LAN, transparent
dynamic networking is now enabled by any application that sits on top of
IP and DNS.

Of course, the limitation of dynamic networking at the IP level is that
it only works for "anycast" type services where a number of nodes can
provide the desired service.  But, it does not provide for
point-to-point dynamic networking services where a single specific node
(possibly roaming) can provide the desired service.

To solve the issue of contacting a single, possibly roaming, node with
no reliable static IP address, we can look at the next layer up the
naming stack - DNS.  To solve dynamic networking at the DNS level, we
make use of two different approaches: Akamai-style request routing via
DNS, and dynamic DNS that routes to a single  roaming node.  The Akamai
approach provides the anycast type functionality where a single name may
resolve to any number of valid endpoints that can facilitate the desired
service.  The dynamic DNS approach allows a single roaming node to be
contacted via a sturdy identifier where ever it may be.

Now, the main limitations of the DNS approach is the reachability of
nodes that are behind NAT/Firewalls and are unroutable via IP and the
centralization of DNS infrastructure.  The first issue can be solved by
something like IPv6/Teredo or simply introducing public rendezvous
points that proxy TCP streams to hidden nodes.  This could be easily
accomplished with SSH port forwarding.  The second problem of
centralization of DNS infrastructure is probably not an issue for most
applications since all you need is a single static server to handle the
DNS serving, and you can easily have a fully decentralized name space
underneath your top level registered domain.

Now, assuming that the NAT and centralization limitations of DNS are too
limiting for your application, or you simply don't feel comfortable
developing an application at the DNS level, the next layer in the naming
stack is URIs.

In order to provide dynamic networking at the URI naming level, you
simply need to switch a portion of your URIs to non-DNS-based URIs. 
There is no need to switch all of your URIs to be location independent,
and indeed in Swarmcast we freely mix static HTTP URIs with location
independent UUID URIs.

Location independent URIs, by their very nature, imply support for both
an anycast model, as well as a single-end-node or point-to-point model. 
The only difference between these two cases is the number of
authoritative sources that can provide the service associated with the
URI in a secure fashion.  In an anycast model, a number of nodes can
authoritatively serve a URI, and in a point-to-point model, only a
single end node can authoritatively serve a URI.

Now, even though these location-independent URIs are not HTTP URIs, it
doesn't mean that they can't still be served via HTTP.  There are two
ways to serve a non-HTTP URI via HTTP, the first is by proxying, and the
second is by providing explicit URI resolution services.  I'll skip over
proxying for now and focus on URI resolution services.  

URI resolution services are very straight-forward. They are basically a
mechanism to discover other equivalent URIs for a given URI.  The goal
of URI resolution in the context of dynamic networking is to resolve a
location independent URI into one or more authoritative HTTP URIs.  This
concept is laid out in THTTP/RFC 2169 and the Content-Addressable Web
(http://open-content.net/specs/draft-jchapweske-caw-03.html), though the
latter is in severe need of updating/refinement.

Under the covers, there are numerous ways that one can implement URI
resolution, but I believe one should be careful to hide the details of
the URI resolution specifics and allow developers to focus on simply
naming and consuming resources.  

For example: 

o Gnutella provides a form of URI resolution by mapping keywords to
URNs, then using THTTP to further resolve to specific HTTP end points. 
With magnet-URIs, people have done exactly that.

o DHT's are an obvious candidate for URI resolution under the covers - I
would love to see a DHT that provides a simple RFC 2169/THTTP interface
for looking up URIs.

o JXTA could also be used under the covers for URI resolution, using its
security and group communication features as a value add.  Once the URI
is resolved, the end-user application would establish a direct
connection to the resulting URI(s) to finish the transaction.  

Now, the last piece of this whole thing is HTTP proxying and caching. 
Now, even after mentioning earlier how DNS/HTTP-based URIs may have
certain limitations, this all goes out the window when you consider HTTP
caching semantics.

HTTP defines very clear semantics for allowing a non-point-of-origin
node to service a request directly and it can also fulfill requests for
non-HTTP URIs.  This allows fully dynamic URI-based networking to take
place hidden completely behind the abstraction of an HTTP proxy.  This
approach also allows additional features like automatic-failover to be
automatically added to applications w/o any modification to the original
application

My point behind this is that routing and messaging are not the answer to
facilitating dynamic networking.  An end application should not need to
be concerned with how to route a message to a dynamic receiver, nor
should it be concerned with protocols at a higher level than absolutely
necessary -  If you need dynamic networking on a LAN, use zeroconf.  If
you need dynamic networking among publicly addressable servers, use
DNS.  If you need dynamic networking to NAT'd nodes, then use URIs and
HTTP.  No matter which route you take, the code in the end-application
is largely the same - it understands IP, DNS, and URIs.

Imagine if Google required every client that connected with their
service to participate in the proprietary Google query routing/messaging
system.  Of course, they chose to hide their service behind the
abstraction of HTTP and their web services.

In closing, JXTA has some great features, but end applications should be
able to access JXTA services w/o knowing or caring that they are JXTA
services.  

Peace,

-Justin

_______________________________________________
P2prg mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2prg

---

From: Justin Chapweske <[email-masked]>
Date: Thu, 22 Apr 2004 13:47:15 -0500

For example: 

Peace,

-Justin

---

From: Paul Prescod <[email-masked]>
Date: Sat, 24 Apr 2004 18:52:06 -0700

> On Thu, Apr 22, 2004 at 01:47:15PM -0500, Justin Chapweske wrote:
> 
>>There is no need to switch all of your URIs to be location
>>independent, and indeed in Swarmcast we freely mix static HTTP URIs
>>with location independent UUID URIs.

I can't believe that after all these years I am going to step back into 
the name versus location debate but I suppose I am just kind of nostalgic.

Rather than write a long email I've written up a little essay. Please 
excuse the fact that it is barely edited:

http://www.prescod.net/rest/combining_names_and_locations/

>>Location independent URIs, by their very nature, imply support for
>>both an anycast model, as well as a single-end-node or point-to-point
>>model.  The only difference between these two cases is the number of

These features all hold regardless of whether the URI has a syntax 
starting with "http://" or "uuid:" You can choose to use an anycast 
model for HTTP URIs, you can have multiple authoritiative sources for 
HTTP URIs (haven't you ever used Google's cache as more authoritiative 
than the referenced domain?) etc.

  Paul Prescod