Subject: [ogpx] Beyond the monolithic client protocol endpoint (was Re: Synchronization of gestures)

From: Morgaine <[email-masked]>
Date: Sat, 5 Dec 2009 18:14:53 +0000

I agree with Lawson, but the topic goes far beyond gestures, so this is a
new thread.

Putting it another way, the days of the monolithic client are over, in the
same way as the days of the walled garden and single service provider are
over.

In VWRAP, we are formalizing the unbundling of service provision so that
server-side is becoming distributed, but we don't seem to have grasped the
corresponding concept of non-monolithic clients yet.  Perhaps the client
needs more flexibility than we are currently considering.

Although we are not dealing with client application architecture here, we *
are* dealing with protocol endpoint architecture, and the design of clients
has a direct impact on that.  The protocol should not require nor give
preference to any one particular client architecture, in the same way as we
seek not to hardwire any single choice of server-side deployment pattern.
It seems reasonable to apply this same philosophy to the client endpoint, so
that the protocol can allow for many different kinds of client-side
deployment.

Midway through the two years of operation of the Architecture Working Group
in SL, we explored a variety of client-side scenarios, including this design
for a multi-process
client<https://wiki.secondlife.com/wiki/Multi-Process_Client_VAG_--_draft>.
 This split out all functional parts of a client into separate
processes,
all interconnected through a "Mediator" hub process via sockets, while also
accepting point-to-point and shared-memory communication paths as
accelerators between subprocesses where required for speed.

Such a client architecture is a perfect match for this new age of multicore,
among a huge list of other benefits enumerated on that page.  It can be
considered the fully distributed end of the large spectrum of possible
clients.  Lawson's SL client- Squeak bridge represents another split, libomv's
gridproxy <http://lib.openmetaverse.org/wiki/SLProxy> and its Jabber
plugin<http://forge.opensimulator.org/gf/project/jabberimproxy/>shows
another, while LL's own system for media plugins is yet another.  All
these moves take us away from the notion of a VW client being a monolithic
entity, and correspondingly, we should no longer assume that the client-side
protocol endpoint is monolithic either.

To that end, it might be an effective approach to model VWRAP as a *protocol
over sets of sources and sinks for data*.  One set is server-side, which we
are already modelling as a multiplicity of service endpoints, and another
set is client-side, representing the many subsystems that deal with
different aspects of interacting with VW worlds and talking to their
services.  Those subsystems are clear candidates for distinct protocol
endpoints of their own.

Such a model would more accurately reflect how client applications are
actually built (in other words, in a modular fashion), instead of funneling
everything through a single client-side endpoint.  The current
single-client-endpoint approach is hardwiring an unnecessary bottleneck into
the client model, as well as encouraging a poor design for clients through
making all data appear at a single doorway.

Perhaps we need to reflect on this a little before rushing ahead with
yesterday's design philosophy for protocol endpoints.

Morgaine.

=================================

On Sat, Dec 5, 2009 at 2:00 PM, Lawson English <[email-masked]> wrote:

> All this goes back to what I just posted on the SLDEV list about taking
> initiative and coding something up. Having the ability for clients to talk
> to each other rather than some 3rd party service might blur the line
> concerning what VWRAP is supposed to be talking about, but, IMHO, the line
> is getting blurred anyway.
>
> What's the difference between a plugin sitting on someone's desktop in
> order to augment the client in some way, and a plugin that actually allows
> clients to talk to each other directly in order to collectively augment them
> in some way? As far as I'm concerned, these discussions about clients
> talking to each other are just extensions totalks about what
> responsibilities client plugins might have.
>
> My current project is based on this idea:
>
>
> http://wiki.secondlife.com/wiki/User:Saijanai_Kuhn/Plugins_discussion#Proposed_Extension_to_Media_Plugin
>
> I'm writing a SL client- squeak bridge (starting off easy with a port of my
>  do-nothing Python client for practice) that will allow me to test new
> client plugin ideas. The Holy Grail for the project will be to integrate
> Cobalt/Croquet with SL in some way(s) and that is just the tip of an iceberg
> that is being held under water by a large outcropping of land. There's more
> potential in P2P-VWRAP hybrids than there is in  P2P (Croquet/Cobalt) OR
> VWRAP (clear-cut client-server worlds) taken separately. While
> VWRAP/OGP/whatever's charter may not consider the P2P approach as relevant,
> the fact is, P2P is potentially in ANY extension of services involving
> clients which aren't talking exclusively to an old school SL simulator.
>
>
> Lawson
>
>
>
>
> Carlo Wood wrote:
>
>> On Fri, Dec 04, 2009 at 06:11:10PM -0800, Han Sontse wrote:
>>
>>
>>> With all due respect,  Carlo, I am finding that I disagree with your
>>> analysis.
>>>
>>>
>>
>> There is no analysis.. it is something has been said before
>> on the list: if YOU don't want something, then that doesn't
>> mean there will be no need for it in the future by others.
>>
>> You shouldn't take YOUR experience and desires and then say:
>> "we will never need this, NOBODY will EVER need this",
>> while it's relatively easy and simple to add the *support*
>> for it to the protocol!
>>
>> By adding the *possibility* for viewers to wait for others,
>> you do not force anyone to do it different from what you
>> describe. But you also don't take away the possibility to
>> do it how other people WANT it (like me, I personally think
>> this is important - so thus far we have a 50% - 50% vote for
>> the NEED of this wait feature).
>>
>> So, to be really undiplomatic blunt: who are you to deny
>> me this experience? While adding just a little to the protocol
>> we BOTH can be happy? You can do it your way, and I can do it
>> my way? But you say: "You don't need it" and want to remove
>> the possibility from the protocol so that you can do it your
>> way and I CANNOT do it the way I want ?!
>>
>> THAT is the reasoning behind NOT leaving out rather trivial
>> options to the protocol. And really, this has been said on
>> the list before a few times imho (and no, I'm not going to
>> find that back).
>>
>>
>>
>>> I have a lot of practical experience in SL getting animations
>>> as well as sound and textures to synchronize under a lot
>>> of different conditions by taking into account the client's existing
>>> cacheing behavior.   I'm drawing on that experience for my assertions.
>>>
>>>
>>
>> I am too, and I say I need this.
>>
>>
>>
>>> I can't think of a single reasonable use case that would *require*
>>> clients
>>> to hold for the slowest link.  The examples you have suggested
>>>
>>>
>>
>> I can.
>>
>>
>>
>>> don't require it to achieve the outcome you desire.
>>>
>>
>> ??? Then you didn't read very well what I said.
>> If you want to do what I said then you *NEED* the possibility to wait
>> for other (some) clients (some limited time), and why not, have the
>> possibility to abort, or continue anyway (as you want).
>>
>> Having those options you can synchronize the experience well within
>> lag-time, well within typing-delay time (ie, under 1 second). If you
>> don't wait, the playing time could run up till 30 seconds or more!
>>
>> Having those options you can synchronize the experience well within
>> 1 second AND you can do what you want: just state don't wait for anyone
>> and play stuff the second you get it.
>>
>> [...]
>>
>>
>>> So here I'll walk through an example... the hug:
>>>
>>> To synchronize a hug all that is needed is for the
>>>
>>
>> "all that is needed" to accomplish YOUR needs and views.
>> For me, I realllllly need to know when other clients are
>> ready too.
>>
>> [...snip...]
>>
>>
>>> be longer for some clients than others.  The exact wall clock delay does
>>> not
>>> matter since the end user has no idea when the event was actually sent.
>>>
>>>
>>
>> Yes we totally disagree here, and you are wrong: there are many other
>> shared experiences (typed text, and remaining movement of the avatar
>> before and after; not to mention voice chat) that need to be more or
>> less in sync with gestures; the user WILL have an idea when the event
>> was sent.
>>
>> The wall clock DOES matter - certainly when the delay can run
>> up to a whole minute.
>>
>> You are "right" for delays less than a second, and I argue
>> that we should try hard to get the shared experience synchronized
>> within one second IF THE USER(S) SO DESIRE.
>>
>> I already gave you examples where it is notable when the delay
>> is larger, but I guess every example is personal. If YOU don't
>> care that your friends see you bolt for the door and once outside
>> you jump and shout "Whooohooo!", while you really intended to
>> first jump and shout and THEN run for the door, then well--
>> that is entirely your choice. Now let me, and others, have mine.
>>
>> Another example, your hug: You initiate a hug request (your client
>> already has the needed assets or starts to download them immediately).
>> By the time that your partner clicks 'Ok', your viewer is ready
>> and immediately starts to play the hug. While hugging you say:
>> "Mmmmmmmmmmm, so nice". And YOU don't care if your partner sees
>> that text while they don't see you hugging yet; others do however.
>> Because others DO, we NEED support in the protocol to know when
>> other clients are ready (at least, have all the needed data
>> cached; any other delay is too short to be bothered with-- I'm
>> with you on that then).
>>
>>
>>
>
> _______________________________________________
> ogpx mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ogpx
>

---

From: Lawson English <[email-masked]>
Date: Sun, 06 Dec 2009 02:24:42 -0700

Thanks Morgaine. The concept of a plugin-driven client is simply your 
modular client idea built off of the existing LL client.

The final extension of your client is a collection of interacting 
plugins with a central mediator and possibly  connections between 
plugins and/or services that bypass the mediator. So the network looks 
like a star network of plugins with a central mediator and a conduit of 
connections between plugins and services. The final design would make 
the mediator an optional central point to establish new connections, and 
process scripting/macro/GUI commands, while the actual functions are 
carried out by the outlying modules, which may not even be on the same 
computer. Similarly, the "server" side might be equally as distributed 
and the distinction between client and server disappears completely with 
clients handling some 3D element generation and physics and assets, etc, 
while the server broadcasts 2D mashups of the client's perspective to 
cell phones.

Etc.

So at one end, we have pure client-server designs like Second Life, at 
the other extreme, purely p2p worlds like Croquet, and every possible 
combination/permutation in-between.

Lawson

Morgaine wrote:
> I agree with Lawson, but the topic goes far beyond gestures, so this 
> is a new thread.
>
> Putting it another way, the days of the monolithic client are over, in 
> the same way as the days of the walled garden and single service 
> provider are over.
>
> In VWRAP, we are formalizing the unbundling of service provision so 
> that server-side is becoming distributed, but we don't seem to have 
> grasped the corresponding concept of non-monolithic clients yet.  
> Perhaps the client needs more flexibility than we are currently 
> considering.
>
> Although we are not dealing with client application architecture here, 
> we /are/ dealing with protocol endpoint architecture, and the design 
> of clients has a direct impact on that.  The protocol should not 
> require nor give preference to any one particular client architecture, 
> in the same way as we seek not to hardwire any single choice of 
> server-side deployment pattern.  It seems reasonable to apply this 
> same philosophy to the client endpoint, so that the protocol can allow 
> for many different kinds of client-side deployment.
>
> Midway through the two years of operation of the Architecture Working 
> Group in SL, we explored a variety of client-side scenarios, including 
> this design for a multi-process client 
> <https://wiki.secondlife.com/wiki/Multi-Process_Client_VAG_--_draft> 
> .  This split out all functional parts of a client into separate 
> processes, all interconnected through a "Mediator" hub process via 
> sockets, while also accepting point-to-point and shared-memory 
> communication paths as accelerators between subprocesses where 
> required for speed.
>
> Such a client architecture is a perfect match for this new age of 
> multicore, among a huge list of other benefits enumerated on that 
> page.  It can be considered the fully distributed end of the large 
> spectrum of possible clients.  Lawson's SL client- Squeak bridge 
> represents another split, libomv's gridproxy 
> <http://lib.openmetaverse.org/wiki/SLProxy> and its Jabber plugin 
> <http://forge.opensimulator.org/gf/project/jabberimproxy/> shows 
> these moves take us away from the notion of a VW client being a 
> monolithic entity, and correspondingly, we should no longer assume 
> that the client-side protocol endpoint is monolithic either.
>
> To that end, it might be an effective approach to model VWRAP as a 
> /protocol over sets of sources and sinks for data/.  One set is 
> server-side, which we are already modelling as a multiplicity of 
> service endpoints, and another set is client-side, representing the 
> many subsystems that deal with different aspects of interacting with 
> VW worlds and talking to their services.  Those subsystems are clear 
> candidates for distinct protocol endpoints of their own.
>
> actually built (in other words, in a modular fashion), instead of 
> funneling everything through a single client-side endpoint.  The 
> current single-client-endpoint approach is hardwiring an unnecessary 
> bottleneck into the client model, as well as encouraging a poor design 
> for clients through making all data appear at a single doorway.
>
> On Sat, Dec 5, 2009 at 2:00 PM, Lawson English <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     All this goes back to what I just posted on the SLDEV list about
>     taking initiative and coding something up. Having the ability for
>     clients to talk to each other rather than some 3rd party service
>     might blur the line concerning what VWRAP is supposed to be
>     talking about, but, IMHO, the line is getting blurred anyway.
>
>     What's the difference between a plugin sitting on someone's
>     desktop in order to augment the client in some way, and a plugin
>     that actually allows clients to talk to each other directly in
>     order to collectively augment them in some way? As far as I'm
>     concerned, these discussions about clients talking to each other
>     are just extensions totalks about what responsibilities client
>     plugins might have.
>
>     I'm writing a SL client- squeak bridge (starting off easy with a
>     port of my  do-nothing Python client for practice) that will allow
>     me to test new client plugin ideas. The Holy Grail for the project
>     will be to integrate Cobalt/Croquet with SL in some way(s) and
>     that is just the tip of an iceberg that is being held under water
>     by a large outcropping of land. There's more potential in
>     P2P-VWRAP hybrids than there is in  P2P (Croquet/Cobalt) OR VWRAP
>     (clear-cut client-server worlds) taken separately. While
>     VWRAP/OGP/whatever's charter may not consider the P2P approach as
>     relevant, the fact is, P2P is potentially in ANY extension of
>     services involving clients which aren't talking exclusively to an
>     old school SL simulator.
>
>
>     Lawson
>
>
>
>
>     Carlo Wood wrote:
>
>             With all due respect,  Carlo, I am finding that I disagree
>             with your analysis.
>                
>
>
>             I can't think of a single reasonable use case that would
>             *require* clients
>             be longer for some clients than others.  The exact wall
>             clock delay does not
>     [email-masked] <mailto:[email-masked]>
> ------------------------------------------------------------------------
>

---

From: Morgaine <[email-masked]>
Date: Sun, 6 Dec 2009 17:11:51 +0000

>From a protocol perspective, the key observation in this area is that
different data paths from VWRAP services to the client no longer all
terminate in the same client-side process --- that is only one extreme end
of the possible spectrum of implementations.

In today's concurrent execution environment, the "application" may comprise
multiple processes on the same client-side host, or be distributed across
several client-side hosts (either virtualized or real), and quite possibly
involve multiple remote hosts on the Internet as well.  The concept of
"application" has broadened since its early years, in part because of the
arrival of multicore, and in part because of ever-growing user expectations
fueled by the Internet and social networking.

It's not up to us to prejudge nor prescribe how VW applications will be
structured.  The fact that advanced ones will involve multiple disjoint
processes is a given, and our protocol endpoint design must take that into
account, or it will be obsolete before it is even ready.

Fortunately, the capability model embraces this concept perfectly.  Having
acquired a capability from an AD or from an RD, this capability represents
all that is needed for anyone to perform whatever action is controlled by
that capability --- it does not prescribe *who* may do so.  That capability
can (in *principle*) be passed around or copied to any other party whom the
original acquirer permits to carry out that action for them.

Unfortunately, practice does not always follow principle, and a bad
implementation of capabilities could prevent a capability being expressed by
any party except the party to whom it was granted.  That would be a
corruption of the concept of "capability" as a rights token, confusing its
clean semantics with those of the endpoint-permission model that
capabilities were designed to improve.

It's not easy to identify what our early specs offer on this score, since
they seem to assume a single client-side endpoint but don't actually address
the issue --- it appears that they were drafted against a background of the
original monolithic client application representing the "norm".  This will
have to be examined in more detail to prevent newer non-monolithic clients
from being constrained by that original implementation.

Does anyone know of any current restrictions on acquiring a capability from
one host IP address but using it from another address, or acquiring it from
one source port but using it from another source port on the same host
address?  (The latter is the single-host, multi-process scenario.)

Morgaine.

======================================

On Sun, Dec 6, 2009 at 9:24 AM, Lawson English <[email-masked]> wrote:

> The final extension of your client is a collection of interacting plugins
> with a central mediator and possibly  connections between plugins and/or
> services that bypass the mediator. So the network looks like a star network
> of plugins with a central mediator and a conduit of connections between
> plugins and services. The final design would make the mediator an optional
> central point to establish new connections, and process scripting/macro/GUI
> commands, while the actual functions are carried out by the outlying
> modules, which may not even be on the same computer. Similarly, the "server"
> side might be equally as distributed and the distinction between client and
> server disappears completely with clients handling some 3D element
> generation and physics and assets, etc, while the server broadcasts 2D
> mashups of the client's perspective to cell phones.
>
> Etc.
>
> So at one end, we have pure client-server designs like Second Life, at the
> other extreme, purely p2p worlds like Croquet, and every possible
>> Although we are not dealing with client application architecture here, we
>> /are/ dealing with protocol endpoint architecture, and the design of clients
>>  It seems reasonable to apply this same philosophy to the client endpoint,
>> so that the protocol can allow for many different kinds of client-side
>> deployment.
>>
>> Group in SL, we explored a variety of client-side scenarios, including this
>> design for a multi-process client <
>> https://wiki.secondlife.com/wiki/Multi-Process_Client_VAG_--_draft> .
>>  This split out all functional parts of a client into separate processes,
>> multicore, among a huge list of other benefits enumerated on that page.  It
>> can be considered the fully distributed end of the large spectrum of
>> possible clients.  Lawson's SL client- Squeak bridge represents another
>> split, libomv's gridproxy <http://lib.openmetaverse.org/wiki/SLProxy> and
>> its Jabber plugin <
>> http://forge.opensimulator.org/gf/project/jabberimproxy/> shows another,
>> while LL's own system for media plugins is yet another.  All these moves
>> take us away from the notion of a VW client being a monolithic entity, and
>> correspondingly, we should no longer assume that the client-side protocol
>> endpoint is monolithic either.
>>
>>
>> /protocol over sets of sources and sinks for data/.  One set is server-side,
>> which we are already modelling as a multiplicity of service endpoints, and
>> another set is client-side, representing the many subsystems that deal with
>> On Sat, Dec 5, 2009 at 2:00 PM, Lawson English <[email-masked]<mailto:
>> [email-masked]>> wrote:
>>

---

From: Carlo Wood <[email-masked]>
Date: Mon, 7 Dec 2009 01:17:03 +0100

> account, or it will be obsolete before it is even ready.

I'd like to grab this opportunity to recall my proposal for
client-server protocol negotiation.

I really, really think that this is very important.

If we could get the protocol completely right and finished
the first time and it would never have to be changed again
in the future, because it would suit every possible future
demand and desire, then sure, we wouldn't need this...

However, apart from that the fact that we'll make mistakes,
we certainly can't predict what will be needed in the future!

Therefore we really should allow for protocol changes by
means of negotiation of optional and mandatory protocol
features.

I also think that we should use MY proposal for this negotiation;
and not some other "protocol negotiation" that just won't cut
it once things really start to change.

To recall:

* The protocol negotiation would allow to shift the protocol
base over time to a COMPLETELY arbitrary new protocol (that
garantees that any possible future demand CAN be met).
* It is possible to keep backwards compatibility for as long
as desired (ie, a few years), at which point old clients
may be forced to upgrade if they lack then mandatory protocol
features.
* The proposal comes with a well thought-out sheme of
distributed documentation of the protocol (extensions):
For documentation purposes there would be the need for
one central website having links to the websites
of every administration with protocol
extensions/enhancements (and from there to documentation
of each feature separately).

Leaving out the details for now, as they don't really
matter imho. Not until we agree that this is something
that we want.

Having such negotiation defined, the protocol can and
will be subject to an evolutionary process: Client coders
will think of new features; certain VW's will implement
and support those - and the best features will, under
user demand, be copied by other grids. Once every grid
uses some new feature and every viewer supports it,
the feature can be made an integral part of the protocol
as if it had been from the very start of it's design.

As an example (from IRC (I have been the main developer
and coder for undernet IRC for seven years)): In 1993
I introduced "Time Stamps" on undernet to battle net.riding.
Today every IRC network uses time stamps, and it's
unthinkable that IRC would work without. Other extensions
that I added on top of the RFC which have slowly been
copied by other networks*) are: bans stop every type
of channel flooding; SILENCE stops private flooding at the
server of the sender; delayed joins (to allow large
event channels with thousands of listeners); "max targets"
flood limiting (to slow down mass spamming to one message
per 2 minutes); BURST message to synchronize channels
on net.join atomically; numeric nicks to stop seven types
of possible desyncs as a result of changing nicks in
combination of MODE's, etc. But here it stopped... and
left me very frustrated about the protocol: IT TURNED OUT
TO BE IMPOSSIBLE TO MAKE NEEDED IMPROVEMENTS! Why? Because
for subsequent essential improvements I needed to change
the client-server protocol and that was IMPOSSIBLE because
of the lack of a good client-server protocol negotiation.
(After I left, this negotiation was added, but in a bad
way, still not allowing for the changes that are really
needed).

-- 
Carlo Wood <[email-masked]>

*) Not literally, but the Idea: each IRC network is
   a completely isolated set of servers, often running
   their own private server versions, so there is no
   need for any kind of compatibility there.
   That is different for the client-server protocol,
   since the SAME clients connect to different networks
   (but, as said before-- for a long time any such
   negotiation was lacking; resulting in hackish ways
   to "detect" on what network one was ('undernet',
   'efnet', 'dalnet') and then guessing what features
   are needed (as no versioning was being done) for
   a long time, with uncontrolled wild growth as
   result.
   VWRAP will exist of many servers and viewers by
   *different* administrations/groups that all
   interconnect however! In that case a good protocol
   negotion sheme IS needed, from the very start!

---

From: David W Levine <[email-masked]>
Date: Sun, 6 Dec 2009 22:46:39 -0500

The current specs certainly are intended to be totally neutral on this. 
The only place I can see where one would have to pay attention would be in 
managing and establishing TLS pipes (HTTPS) and this is routinely done 
within web environments. As you observe, the semantic of capabilities is 
designed to support this. It would actually take some effort to break that 
semantic. 

That said, there are structures which work better, and worse, when they 
are widely distributed.Asset, Messaging, and Such distribute without much 
pain. The core state sharing application (What a region does for a living) 
is harder to spread out, and likewise, there are interesting 
synchronization challenges as you break apart client components. The less 
shared tasks (such as manipulating your inventory) are less sensitive. The 
data flows which manage the shared visual experience more so. 

>From the protocol perspective, I suggest the question is "Are there 
specific synchronization management issues we need to surface." I'm not 
expecting a lot, but if we know of some, we should get them on the table.I 
will also gently remind people of the notewell, when it comes to
any IPR encumbered issues in this space. 

- David / Zha 

Morgaine <[email-masked]> 
Sent by: [email-masked]
12/06/2009 12:11 PM

To
[email-masked]
cc

Subject
Re: [ogpx] Beyond the monolithic client protocol endpoint (was Re: 
Synchronization of gestures)

In today's concurrent execution environment, the "application" may 
comprise multiple processes on the same client-side host, or be 
distributed across several client-side hosts (either virtualized or real), 
and quite possibly involve multiple remote hosts on the Internet as well.  
The concept of "application" has broadened since its early years, in part 
because of the arrival of multicore, and in part because of ever-growing 
user expectations fueled by the Internet and social networking.

Fortunately, the capability model embraces this concept perfectly.  Having 
acquired a capability from an AD or from an RD, this capability represents 
all that is needed for anyone to perform whatever action is controlled by 
that capability --- it does not prescribe who may do so.  That capability 
can (in principle) be passed around or copied to any other party whom the 
original acquirer permits to carry out that action for them.

Unfortunately, practice does not always follow principle, and a bad 
implementation of capabilities could prevent a capability being expressed 
by any party except the party to whom it was granted.  That would be a 
corruption of the concept of "capability" as a rights token, confusing its 
clean semantics with those of the endpoint-permission model that 
capabilities were designed to improve.

It's not easy to identify what our early specs offer on this score, since 
they seem to assume a single client-side endpoint but don't actually 
address the issue --- it appears that they were drafted against a 
background of the original monolithic client application representing the 
"norm".  This will have to be examined in more detail to prevent newer 
non-monolithic clients from being constrained by that original 
implementation.

Does anyone know of any current restrictions on acquiring a capability 
from one host IP address but using it from another address, or acquiring 
it from one source port but using it from another source port on the same 
host address?  (The latter is the single-host, multi-process scenario.)

Morgaine.

The final extension of your client is a collection of interacting plugins 
with a central mediator and possibly  connections between plugins and/or 
services that bypass the mediator. So the network looks like a star 
network of plugins with a central mediator and a conduit of connections 
between plugins and services. The final design would make the mediator an 
optional central point to establish new connections, and process 
scripting/macro/GUI commands, while the actual functions are carried out 
by the outlying modules, which may not even be on the same computer. 
Similarly, the "server" side might be equally as distributed and the 
distinction between client and server disappears completely with clients 
handling some 3D element generation and physics and assets, etc, while the 
server broadcasts 2D mashups of the client's perspective to cell phones.

Etc.

Lawson

Morgaine wrote:

Although we are not dealing with client application architecture here, we 
/are/ dealing with protocol endpoint architecture, and the design of 
clients has a direct impact on that.  The protocol should not require nor 
give preference to any one particular client architecture, in the same way 
as we seek not to hardwire any single choice of server-side deployment 
pattern.  It seems reasonable to apply this same philosophy to the client 
endpoint, so that the protocol can allow for many different kinds of 
client-side deployment.

Midway through the two years of operation of the Architecture Working 
Group in SL, we explored a variety of client-side scenarios, including 
this design for a multi-process client <
https://wiki.secondlife.com/wiki/Multi-Process_Client_VAG_--_draft> . 
 This split out all functional parts of a client into separate processes, 
all interconnected through a "Mediator" hub process via sockets, while 
also accepting point-to-point and shared-memory communication paths as 
accelerators between subprocesses where required for speed.

multicore, among a huge list of other benefits enumerated on that page. 
 It can be considered the fully distributed end of the large spectrum of 

To that end, it might be an effective approach to model VWRAP as a 
/protocol over sets of sources and sinks for data/.  One set is 
server-side, which we are already modelling as a multiplicity of service 
endpoints, and another set is client-side, representing the many 
subsystems that deal with different aspects of interacting with VW worlds 
and talking to their services.  Those subsystems are clear candidates for 
distinct protocol endpoints of their own.

Such a model would more accurately reflect how client applications are 
actually built (in other words, in a modular fashion), instead of 
funneling everything through a single client-side endpoint.  The current 
single-client-endpoint approach is hardwiring an unnecessary bottleneck 
into the client model, as well as encouraging a poor design for clients 
through making all data appear at a single doorway.

Morgaine.

On Sat, Dec 5, 2009 at 2:00 PM, Lawson English <[email-masked] <mailto:
[email-masked]>> wrote:

   Lawson

   Carlo Wood wrote:

           I have a lot of practical experience in SL getting animations
           as well as sound and textures to synchronize under a lot
           of different conditions by taking into account the
           client's existing
           cacheing behavior.   I'm drawing on that experience for my
           assertions.

       I can.

       ??? Then you didn't read very well what I said.
       If you want to do what I said then you *NEED* the possibility
       to wait
       for other (some) clients (some limited time), and why not,
       have the
       possibility to abort, or continue anyway (as you want).

       Having those options you can synchronize the experience well
       within
       lag-time, well within typing-delay time (ie, under 1 second).
       If you
       don't wait, the playing time could run up till 30 seconds or more!

       Having those options you can synchronize the experience well
       within
       1 second AND you can do what you want: just state don't wait
       for anyone
       and play stuff the second you get it.

       [...]

       [...snip...]
        
           be longer for some clients than others.  The exact wall
           clock delay does not
           matter since the end user has no idea when the event was
           actually sent.

       Yes we totally disagree here, and you are wrong: there are
       many other
       shared experiences (typed text, and remaining movement of the
       avatar
       before and after; not to mention voice chat) that need to be
       more or
       less in sync with gestures; the user WILL have an idea when
       the event
       was sent.

       already has the needed assets or starts to download them
       immediately).

---

From: Han Sontse <[email-masked]>
Date: Sun, 6 Dec 2009 21:19:56 -0800

On Dec 6, 2009, at 7:46 PM, David W Levine wrote:

>
> The current specs certainly are intended to be totally neutral on  
> this. The only place I can see where one would have to pay attention  
> would be in managing and establishing TLS pipes (HTTPS) and this is  
> routinely done within web environments. As you observe, the semantic  
> of capabilities is designed to support this. It would actually take  
> some effort to break that semantic.
>
> That said, there are structures which work better, and worse, when  
> they are widely distributed.Asset, Messaging, and Such distribute  
> without much pain. The core state sharing application (What a region  
> does for a living) is harder to spread out, and likewise, there are  
> interesting synchronization challenges as you break apart client  
> components. The less shared tasks (such as manipulating your  
> inventory) are less sensitive. The data flows which manage the  
> shared visual experience more so.
>
> specific synchronization management issues we need to surface." I'm  
> not expecting a lot, but if we know of some, we should get them on  
> the table.I will also gently remind people of the notewell, when it  
> comes to

Loosely speaking we need to be able to deal with a cloud-to-cloud  
relationship.   This fuzzy notion seems to extend to client-clouds and  
server-clouds, as well as P2P clouds.   To the extent that a given  
simulation space might contain other simulations, one could even  
consider each client as a server and each server as a client.

maybe a way to begin to break down the complexity in these  
relationships it to look at the different classes of service that may  
be declared:

Protocol constraints
One-shot  (UDP)
simple Ack (UDP-UDP)
Transactional short term (TCP)
Complex (Multi-port/multi-host???)

QoS
Isochronous -- definite real time interval; predefined sequence order
Synchronous --  predefined sequence ordering, indefinite time interval
Asynchronous -- indefinite sequencing, indefinite timing

Payload class declaration
	content streams
	parametric events
	bulk transfers
	OoB Signalling

Payload structure declaration
	intrinsics (uint32, bool, etc)
		direct
		indirect
	structured
	complex

Topology (Client and server are sub-classed peers)
	Ringed/chained P2P (a peer only knows 1 or 2 neighbors in the context)
	Open P2P( a peer knows all other peers in the context)
	Hierarchical P2P (peer knows a small set of super-peers and an  
arbitrary number of local peers)
	???

and so on....
~HS~

---

From: Joshua Bell <[email-masked]>
Date: Mon, 7 Dec 2009 10:30:25 -0800

> client-server protocol negotiation.

Am I correct that this represents the previous discussion on the topic?

Carlo's post:

http://www.ietf.org/mail-archive/web/ogpx/current/msg00363.html

Meadhbh's reply:

http://www.ietf.org/mail-archive/web/ogpx/current/msg00362.html

Can you respond to Meadhbh's comments? Dealing with version skew is
addressed at various levels in the protocol drafts already, from
presence/absence of named capabilities to HTTP nuances to LLSD semantic
rules. That's not to say that this is sufficient, but it does appear to
cover a broad spectrum of version skew issues, so identifying those that
aren't covered - or those that could be addressed more explicitly with
numbered versions - would be extremely valuable.

Joshua

---

From: Meadhbh Hamrick <[email-masked]>
Date: Mon, 7 Dec 2009 10:57:00 -0800

right. as i remember it (and i may be doing carlo's proposal a great
disservice) the proposal is:

a. at connection time, the client tells the server what new protocols
it knows about.
b. if the server doesn't know about the new protocols, then it uses
the base protocol

if so, we should also ask how this is different than a HTTP Upgrade.

-cheers
-meadhbh

---

From: Carlo Wood <[email-masked]>
Date: Mon, 7 Dec 2009 20:58:25 +0100

On Mon, Dec 07, 2009 at 10:30:25AM -0800, Joshua Bell wrote:
> Can you respond to Meadhbh's comments? Dealing with version skew is addressed
> at various levels in the protocol drafts already, from presence/absence of
> named capabilities to HTTP nuances to LLSD semantic rules. That's not to say
> that this is sufficient, but it does appear to cover a broad spectrum of
> version skew issues, so identifying those that aren't covered - or those that
> could be addressed more explicitly with numbered versions - would be extremely
> valuable.

Well, the more the protocol as-is allows corrections, improvements and
extensions later on without the need for client (coders) to know what
network/grid they are connected to, and/or what server versions are
being used, because it will all "automatically" work out, the better!

But-- it's very likely that not everything can be handled that way
and that viewers will more and more have to fall back to kludgy
heuristics to guess what new features are demanded or supported
on the current grid they connect to. I state that we CANNOT know
what changes of that type we will need in the future, I just know
that it will happen; that we'll wish that a feature negotiation
was in place from the beginning.

As a coincidence (or is it really a coincidence...) right NOW we
need this (see my other post): At the moment there is a need for
a new capability (the url for the map).

Working:

* If the client doesn't know about it, it won't ask for it (check).

* If the service doesn't know about it it won't return a capability.

Not working:

* In the latter case (the service not knowning about it) the viewer
  needs to fall back to the old style map on opensim and to a hardcoded
  S3 url on Second Life (which DOES support this capability, but just
  doesn't know about the "capability" url itself).

* Once Linden Lab (or any other grid that will implement the same
  way that map works) stops supporting the "old style map", there
  is no way to tell the client explicit that THIS "feature" (the
  new way of how map works) is now mandatory, other than the
  existing method of requiring a minimal *viewer* version... which
  of course only works for the few viewers that are supported this
  way and which is a very crude way to state that one feature is
  now mandatory (bound to fail once there are lots of features
  like this to be negotiated).

Sure, we can make this work-- in some kludgy way or another--
requiring hardcoding special cases and "grid detection" etc.
But this is just one, or the first of many that will come,
and to deal with that is neat and controlled way, we need
protocol negotiation.

If I'm wrong, and no protocol negotiation will ever be needed,
then we will not have lost anything: the feature tables will
just stay empty and there will not be anything to negotiate.
No harm done.

---

From: Carlo Wood <[email-masked]>
Date: Mon, 7 Dec 2009 21:14:44 +0100

> if so, we should also ask how this is different than a HTTP Upgrade.

The details are probably only important once we agree that this
is needed (or at least handy to have :p)...

Nevertheless, a better description would be:

a. At connection time, the server tells the client what implementation
   it represents (ie "opensim" or "lindenlab") (called "label" below),
   it's protocol version, and a "compatibility version"
   (multiple label/version/compversion triplets may be provided).

b. If the client knows about compversion or later for that implementation,
   then it proceeds; otherwise it would be incompatible (it will lack
   a feature that the service requires) and the user has to upgrade
   before they can connnect.

c. Since the client knows (was written for) "label" and version X,
   where X >= compversion, it will know what features are mandatory
   and which are optional (new optional feature of a later version
   it won't know about, but it also won't support those). Therefore
   it can request those optional features that it supports and wants.
   (your points 'a', above).

d. The server sends a final handshake where it tells the client with
   which features it will proceed (this is necessary as the server
   might DENY requested optional features, because it is possible
   that the current version of the server doesn't support those
   anymore; which doesn't make it incompatible with the viewer).

Thus:

TO client   <-- NETWORK label1 version1 compversion1 [label2 version2 compversion2 ...]
>From client --> PROT REQ label1 versionX <list of features>  (where compversion1 <= versionX <= version)
TO client   <-- PROT SET label1 versionX <pruned list of features>

---

From: Suzy Deffeyes <[email-masked]>
Date: Mon, 7 Dec 2009 17:40:30 -0600

>  like this to be negotiated).

So in the map tile case you mention, if the viewer gets a cap to the new way
of fetching tiles, it wouldn't try to do the old way.   In this particular
case, it seems like getting a cap for http fetch of map tiles deprecates
client use of the old mechanism.

"Mandatory" is something that should be dictated by a VWRAP spec version
number. So in version 1.0 the map capability is not mandatory, and when we
rev the version to 1.1, we might decide to make it mandatory. A viewer
supporting version 1.1 of the spec could reliably ditch the old code.

One thing I do ponder that might be related is how do I specify/decide that
I do *not* want to trust a specific packet type from the region. For
instance, if I get a cap for inventory operations from my agent domain, then
maybe I don't want to trust UDP packets related to Inventory that I get from
Joe's Prim Ripper region.  Will it always be obvious what the desired
behavior in the viewer is?  In the map tile example, should I blow off any
map tile packets i get from a region that also claims to support the map
tile HTTP cap?

Suzy Deffeyes/Pixel Gausman
IBM

---

From: Carlo Wood <[email-masked]>
Date: Tue, 8 Dec 2009 14:57:30 +0100

On Mon, Dec 07, 2009 at 05:40:30PM -0600, Suzy Deffeyes wrote:
> So in the map tile case you mention, if the viewer gets a cap to the new way of
> fetching tiles, it wouldn't try to do the old way.

Well, if it DOES get a cap, then clearly the service DOES know about the
feature, so there is little need for the old style map.

> In this particular case,
> it seems like getting a cap for http fetch of map tiles deprecates client use
> of the old mechanism.

Yup. The problem that I tried to demonstrate that needs protocol/feature
negotiation is for the case that the server does NOT return a cap.

Wouldn't it an ideal world if in the end we'll have One Big Virual World ;)
(rethorical question). But, one world or not (and certainly in the case
where we have many different worlds) in the end we will have many different
branches of servers. Server implementations written by different people,
with different goals and ideas.

At the client side the same thing will happen. Right now we ONLY have six
or so separately maintained viewers; that number will grow a lot in the
future up till the point where it is unpractical to maintain communication
between client and server coders.

If we all can convince ourselves that the standard that we will produce
here is THE BEST-- and does not need ANY extensions or extra features,
then surely we will decide that what we make here is the Best For Everyone,
and Nobody Needs More. However, we are not the (only) server developers
of the future, and I predict that there will be a growing number of
server developers in the future that WILL feel the need for some new,
till then unsupported feature.

What is going to happen is that there WILL be a growing set of "features"
on top of the VWRAP standard, and no controlled way to communicate all
those features to the viewer developers... To make a long story short,
it will become a mess. Soon you will only really be able to connect to
server type "opensim" if you use their viewer; and only connect to
Second Life if you use Linden Lab's viewer.

The viewer coders will get frustrated about that, because they want
to allow their users to connect whereever. So, they need to make their
viewer adjust to the protocol as function of the server type that they
connect to.

At first, it would be sufficient to add some kludge to detect the major
grids (as it works for "their own" viewer), but the growing number of
servers and features will make a nightmare of this. Server version
detection will be needed, and then the viewer coders will need to start
to keep track of which versions of each server-type support what
features. The methods used to determine if some feature is optional
and whether or not a viewer can or want to use it will also be
random (everyone implementing their own way). Some server developers
will only support a few viewers and keep a list of versions of those
viewers and then "negotiate" the features silently by assuming that
viewer MyOwnViewer version 4.35 is compatible and has features
A,E,J,M,N,X and Z, while FurryViewer version 1.11 is not compatible
but 1.12 is but only supports features A,E,X and Z. Now LittleViewer
can't connect at all (except for using the base VWRAP protocol)
except by spoofing it's version and saying it is really FurryViewer,
however LittleViewer's coder didn't really want to add feature X,
rather wanted just to support feature F...

Buttom line, this isn't about ONE application (a server) with a
compatible viewer (client) that is controlled and maintained by
the same people as the server. This is about a large network of
servers and clients maintained and developed dynamically by many
different groups and we DO NEED TO SUPPORT protocol evolution
in order to avoid a complete future mess.

If anyone doesn't believe that (or thinks that we can do this
with just caps and the other methods already proposed in OGP)
then I beg you to add this anyway: if no features are added
then we're talking about three or four small, standard messages
(two each direction) for each client-server connection, and
no harm done.

If anyone doesn't WANT this, because they don't want to see the
protocol evolve, because a *standard* means that everyone uses
the same, and therefore building in ways to add random features
would only invite people to make viewers incompatible; then please
reconsider. I strongly believe that not adding support will
not stop people. Besides, it isn't bad if the protocol changes
over time at all imho; look at it as evolution: support for the
best features will be added to all viewers and servers, other
features won't make it and die again. As long as it is crystal
clear who supports what, and the documentation of each and every
feature can easily be found back on the web, then I think we
only gained.

---

From: Morgaine <[email-masked]>
Date: Wed, 9 Dec 2009 06:12:59 +0000

VWRAP version numbers should change only when the base protocol itself is
incompatible between versions, and never just because a particular feature
is or is not supported by the client or by the server.  There will be
hundreds or thousands of features in the future of VWRAP, an ever-evolving
set of them, and the protocol needs to be generic enough to support such
diversity.  Protocol version numbers are the wrong place for selecting
features or feature sets.

The map tile access mechanism is definitely not the kind of thing that VWRAP
version numbers would be expected to select.  Such a choice needs to be
dynamic because it's not fixed for the life of the session.

Maps are determined by region policy or virtual world policy, and are not in
any way related to the session that commences with authenticating with the
agent domain.  The topological layout of regions or region domains is a
matter for each virtual world to control, while the visual data for each
tile is a matter for each region or region domain to determine.
Capabilities for map retrieval will have to be obtained accordingly, in a
dynamic manner, because the mechanism that is required will vary as one
travels around.  There will probably be several mechanisms in common use.

It should be noted that VWRAP users will be visiting many different worlds,
each one with its own feature set, so non-uniformity should be expected to
be the norm.  Even our first two sample points of SL and OSgrid already
illustrate lack of uniformity in numerous features, and we can expect this
to continue and to expand.  It's a situation that we must embrace through
designing a protocol that is inherently extensible, and in which features
are selected dynamically.

Morgaine.

==========================================

On Mon, Dec 7, 2009 at 11:40 PM, Suzy Deffeyes <[email-masked]> wrote:

> So in the map tile case you mention, if the viewer gets a cap to the new
> way of fetching tiles, it wouldn't try to do the old way.   In this
> particular case, it seems like getting a cap for http fetch of map tiles
> deprecates client use of the old mechanism.
>