Subject: questions on Keith's draft from an implementor....

From: Michael Mealling <[email-masked]>
Date: Tue, 9 Apr 2002 23:50:37 -0400

Speaking as an implementor and not as a co-chair I've come up with
some issues that I think might be worth discussing. I'm writting this
as I'm writing my RC query/response encoder so intimate knowledge
of the packet formats is needed to parse some of this stuff:

1) Support in the Query result structure for the QUERY_RECURSE flag being set 
is handled by an array of blob's that contain answers with the first
being the one for the resource expressed in the query and subsequent
array values being 'additional information'.  This seems fine in the case
where you have and want that additional information. But in the 'normal'
case you end up 'wasting' a whole new set of blob headers for just one
blob when in actuality it will always be there. Would it be more efficient
to save a layer of encoded blobs and have the query_response structure
look like this:

BEGIN query_response
string request_id
int query_status
int version_hi
int version_low
struct<> assertions
struct<> signatures
struct<> additional
END

Thus saving (if my math is right) 16 very redundant bytes for _every_ query
response.

2) One thought keeps bugging me: will we have the case where a request
for an attribute for a given resource needs arguments? Each time I've
had the need for one I simply tagged it onto the end of the attribute 
name URI. But that just seems insufficiently specified for some gut
reason. 

3) The situation with not knowing what size UDP packet to use and just
'hoping' the other end gets all of the fragments is bothersome. I can't
find any fragmentation delivery statistics but I've been told that
anything above 1500 bytes creates 'problems' with failed delivery of all
the subsequent fragments. The suggestion is to start out using TCP but that 
just seems overkill. Could we add something to the UDP packet that allows
for a RESCAP response to be packaged for client reconstruction?

Just thoughts I'd like some feedback on....

-MM

-- 
--------------------------------------------------------------------------------
Michael Mealling	|      Vote Libertarian!       | urn:pin:1
[email-masked]      |                              | http://www.neonym.net

---

From: Keith Moore <[email-masked]>
Date: Wed, 10 Apr 2002 11:32:17 -0400

> Thus saving (if my math is right) 16 very redundant bytes for _every_ query

you're right about the bandwidth savings, but the server and client
code are simpler if all responses have the same structure.  
I decided to compromise in favor of simplicity.  

in general, there are lost of ways to tweak the blob structure to
be more efficient - e.g. for blobs < 64k, 2 bytes would suffice 
for each offset.  again, it's a compromise - my choice was to
keep blobs flexible enough to handle larger objects without making
them more complex than necessary.  
 
> for an attribute for a given resource needs arguments? 

someday you might want to match patterns of attribute names other 
than by prefix.  a new RC function could be defined to do that; 
with old servers the client would just have to ask for  all attributes
and filter them itself.  

> for a RESCAP response to be packaged for client reconstruction?

we definitely need to supply good advice about when to use TCP vs
when to use UDP - my gut sense is that if the result is more than
3 IP fragments then you're better off using TCP than resending 
the request, but I'd like to see that verified by a statistical model.

OTOH, I'm not sure about the value of adding fragmentation/reassembly to RC.
I actually tried this in an earlier version, and it was a pain.  It requires
the server to maintain state about each pending request (since the data
can change between the first and subsequent requests).  and basically you
need about the same services you would get from TCP anyway. 

Keith

---

From: Michael Mealling <[email-masked]>
Date: Wed, 10 Apr 2002 17:22:36 -0400

> I decided to compromise in favor of simplicity.  

But they still have the same structure. The issue is that, since
_every_ query response will have at least one assertion for the URI
in the query, why not break it out of the array of blobs into the
query result blob itself. That way you only have to parse/use the
blobs in the additional information if you need to....

> them more complex than necessary.  

But this isn't a suggestion to tweak the blob structure but to tweak
RC's query structure....

> with old servers the client would just have to ask for  all attributes

But I'm wondering if its worth doing now.... Has anyone else run into 
this situation?

> the request, but I'd like to see that verified by a statistical model.

But why TCP just to get one more packet through?

> need about the same services you would get from TCP anyway. 

I'm not talking about full packet state and acks. I'm just talking about
the server sending a packet sequence number so that the client can put
blocks back together (and is also aware of what is being sent). If it
fails then the client either resends the request or reconnects via TCP.
The server wouldn't need to maintain any state.

-MM