Subject: Re: I MOVE we simplify the query language.

From: "John Stracke" <[email-masked]>
Date: Tue, 26 Mar 2002 16:10:12 -0500

>> You probably wouldn't be able to compose such queries by hand; but a 
CUA 
>> might have hard-coded queries that are that complicated.  For example, 
the 
>> above query would be useful for Fred's admin.
>
>Sounds like a great example for a stored query :-)

No, because, in the standard, a stored query cannot do anything that a 
non-stored query cannot do.

>>>            then let the CUA sort it out what it wants?
>> 
>> That doesn't scale on the CS, though.
>
>I am NOT saying that we don't need a QL-FULL, I am saying we need
>a QL-MIN that is an exact and limited subset of QL-FULL 

And I am saying that we already have a minimal subset; we can't trim it 
down any further and still have something useful.

>so that
>small device CSs can be implemented 

You ignored my question of why a small device would run a CS.

>> Pushing the query back towards the 
>> store makes for less bandwidth and less CAP processing.
>
>I would suspect that a CUA won't ask for what it can't handle.

That's not feasible.  The CUA, almost by definition, has no idea how much 
stuff is going to come down.

Anyway, without the USING constructs, it is impossible for the CUA to make 
some of the most important queries (e.g., "today's alarms"), which 
guarantees that the CUA will frequently have to pull down too much stuff.

>And why would expecting a CS that serves N number of CUA, scale
>better than 1 CUA doing the same work?

Because it does.  If I'm a CS, stepping through VEVENTs to send down, then 
examining each one to decide whether it matches a query takes much less 
time than transmitting it.  That means that pushing the query back towards 
the store saves work all along the way.

This is why database stored procedures exist: because they're faster than 
bringing all the data down to the database client to be processed.  At 
eCal, I initially opposed using stored procedures, on the grounds that 
they would increase the database's load.  I was wrong; when we went to 
stored procedures, our performance went up enormously.

>>  For 
>> database-based CSes (the only ones that will scale), you want to run 
the 
>> query in the DB itself, to get as little data as possible coming back 
over 
>> the wire from the DB.
>
>Does your database server support 'USING_PROPERTIES()', if not - no it
>can't be at that level. If it can't be mapped 1:1 to your database
>query language, then the CS is going to have to do 

Red herring.  Obviously the database can't run the CAP query language 
directly--if nothing else, queries on multivalued properties are going to 
have to be rewritten into joins.  So you compile the CAP query into SQL. 
No big deal; your only alternative is to interpret the CAP query, which is 
almost as hard as compiling it.

>exactly what you
>say won't scale.

No, I said that CSes that don't use a DB backend at all won't scale.  It's 
possible to have the CS do filtering on what comes back from the DB, if 
you really can't compile the CAP query into a SQL query (or don't want to 
take the time to implement that compilation); and it wastes less 
processing & bandwidth than having the CUA do the filtering, since the 
round-trip time to the database is usually less than to the CS.

>There are forces on this list that do not want an SQL language.
>So we invented things that break every database server language.

There are forces on this list that do not want a non-SQL language.  So we 
invented things to keep SQL from being broken by the iCalendar data model. 
 These extensions are painful, but they're the only way to keep SQL and 
get a query language that actually works.

>> It also hurts CUA performance, especially for limited-bandwidth cases 
like 
>> PDAs and cellphones, where users get charged by the byte.
>
>I would suspect that those vendors will be using stored queries

Again, if they're standard CAP queries, then being stored is irrelevant. 
On the other hand, if standard CAP queries don't meet the CUAs' needs, 
then we've failed.

>I am not trying to limit a CS from doing a QL-FULL. I am trying
>to enable small devices and small implementations to not have to
>buy a full database server in order to do CAP-QL.

You don't need a SQL-based server to implement the CAP query language; you 
can interpret the queries yourself.

/===========================================================\
|John Stracke                    |Principal Engineer        |
[email-masked]   |Incentive Systems, Inc.   |
|http://www.incentivesystems.com |My opinions are my own.   |
|===========================================================|
|I am destined to build a bridge to China out of live sheep.|
\===========================================================/

---

From: Doug Royer <[email-masked]>
Date: Tue, 26 Mar 2002 14:50:29 -0700

> non-stored query cannot do.

Okay? So it still sounds like a good canidate for a stored query.

> You ignored my question of why a small device would run a CS.

Because I can sell it.
Because it was a CAP requirement to support thin devices.
Because I want to.

> stuff is going to come down.

It is a TCP connection. Read what you can process when you
can process it. We don't have to invent flow control.

If you don't like what you are getting, close the BEEP channel
or abort the command.

Or the CUA will have set the MAX size when it connected in
its capabilities and the CS will have never sent large replies.

> guarantees that the CUA will frequently have to pull down too much stuff.

OR - my PDA can call a stored query that my mega-windows-GUI created
that gets what the PDA wants. Even when the PDA could never understand
the contents of the query, and even if the PDA could have never
generated the complicated query that it is using by QUERYID only.