Subject: Re: (unlimited) The intent/meaning of EXPAND and its usefullness

From: "Preston Stephenson" <[email-masked]>
Date: Fri, 12 Dec 2003 13:54:56 -0700

Just my too bits worth.

In our implementation, we expand out recurrence items on creation (so
STORES:EXPANDED:TRUE).
We set RECUR-LIMIT to specify the max we will fan out on creation.
For access of the items after that the other recur
properties/parameters don't make much sense.
For query, EXPAND doesn't make sense and we ignore it (whether or not
RECUR-EXPAND is set).
For query, RECUR-LIMIT doesn't make sense, since we only want it to
apply to storage.
Can we abstract the limit to just be a limit (say QUERY-LIMIT) of any
items returned?
For query, RECUR-EXPAND doesn't make sense, since the items are already
expanded.

There is a little problem on RECUR-LIMIT applying to storage and
query.
In the case of 2) below, the absence of  RECUR-LIMIT would mean no
limit.
The problem arises if you have to have it be two limits (storage and
query).

On creation we just expand out the item to the limit we can.
We have no way of passing back a status saying, we would have created
more.
(We've just documented it in the past and our customers know to work
around the limitations.)
We have no way of adding to an existing recurrence set.
How would you reflect that functionality in CAP,
"IF-YOU-CREATE-A-RECURRENCE-ITEM-AND-IT-EXCEEDS-THE-LIMIT" what happens
flag?

Thanks.
Preston

>>> <[email-masked]> 12/12/2003 8:53:24 AM >>>

Doug replied on 12/11/2003 03:28:33 PM:
> See RECUR-LIMIT.

That and STORES-EXPANDED (which is interrelated) but I dont think all
bases are covered yet. 

RECUR-LIMIT is defined as: 

   Purpose: This property specifies the maximum number of instances
the
  endpoint will expand instances into at query or storage time. 
[snip, snip] 
   Description: For implementations that have the "STORES-EXPANDED"
  value set to TRUE, then this value specifies the maximum number of
  instances that will be stored and fetched. For all implementations
  this is the maximum number of instances that will be returned when
  the "EXPAND" parameter is specified as TRUE and the results contain
a
  infinite or large number of recurring instances. 

and STORES-EXPANDED is defined as: 

   Purpose: This property specifies if the sending endpoint expands
  recurrence rules prior to storing them into the CS. 
[Snip, snip] 
   Description: If the value is TRUE then the endpoint expands
  recurrence rules and then stores the results into the CS. If this is
  TRUE then the "RECUR-LIMIT" property is significant because an
  infinitely recurring appointment will be stored no more than
  "RECUR-LIMIT" property values into the CS and all other instances
  will be lost. 

The unclear/unspecified bits I find are: 

1: RECUR-LIMIT says that its a limit for query results or for storage
time.  However this latter bit is inaccurate if STORES-EXPANDED is
FALSE.   Both STORES-EXPANDED and RECUR-LIMIT need to have prose in them
that clearly indicates that RECUR-LIMIT is not used as a storage
constraint if STORES-EXPANDED is FALSE (there is only text now to the
effect o STORES-EXPANDED:TRUE). 

2: There is no indication of how to indicate "no limit", if we want to
support that.  There is an example of RECUR-LIMIT:0 in the text however
this violates the ABNF for RECUR-LIMIT which says its a posint1 which is
defined as: 

    posint1     = posintfirst 1*DIGIT

                 ; A number starting with 1 through 9.
                 ;
   posintfirst = %x31-39 

so the example on p112 violates the ABNF or is intended to be "I allow
infinite expansion".  If we do not want to support "No limit" in CAP, we
need to add some explicit text to that effect.  If we do want to support
it, we need to say exactly how we indicate that. 

3: There is nothing to indicate how the endpoint can tell the requestor
"I have expanded to my RECUR-LIMIT and there are other instances that 
you did not get".   This is important for the CUA to know so it can
properly inform the CU that what they see is NOT all that should be
there.  In addition, if there is some way to 'resume' the expansion then
that should be clearly defined.  For example, if the CS says
RECUR-LIMIT:10 and STORES-EXPANDED:FALSE then if the CUA asks for
EXPAND:TRUE on a 30 instance repeat set there should be some clearly
defined way for the CS to expand the 1st 10 (RECUR-LIMIT) instances and
tell the requestor "Thats all ya get but thats not all I have."  The
requestor could then refine the query so that they get the next 10
instances (11-20) and the response "Thats all ya get but not all I
have." And so on until they are able to extract all of them from the CS
w/o having to do the unwinding themselves. 

This gets a little messy unless we clearly state that expansions are
ALWAYS done in chronological order so that the requestor simply tweaks
their QUERY to include a "AND DTSTART > "last returned instances"" kind
of clause.  It may be implicit so far but I would expect the CS to
return expanded results in chronological order but its not written CAP
that it has to so its possible the results come back in any arbitrary
order making this resuming of expansion hard to do. 

If we do not provide some means for this we run into some usability
issues.  A CU could store a RRULE;COUNT=364 repeating component into a
STORES-EXPANDED:FALSE CS whose has RECUR-LIMIT:100 just fine but they
would be unable to use EXPAND:TRUE to get the results back out and
possibly even without EXPAND:TRUE set (see #6 below). 

4: When a requestor tries to store something that would violate the
RECUR-LIMIT of the CS, there is no clear describing what the failure
results/state should be.  That is, is the entire request (for that
component) rejected entirely?  Is it processed to RECUR-LIMIT and the
rest ignored/lost?   

5: Does RECUR-LIMIT apply to manually stored instances or it is a limit
per component no matter how the instances are stored?  That is, if a CS
says they have RECUR-LIMIT:5 and I try to create a RRULE;COUNT=6 the CS
would reject it (subject to #4 above).  However, what if the CUA
manually rolls out the instances themselves (or they get created over
time as the CU gets added to more and more instances bit by bit!).  Can
my CUA create 6 separate instances of a particular component without
running astray of RECUR-LIMIT?  Since the CS is not doing the actual
expansion Im not certain that STORES-EXPANDED or RECUR-LIMIT would be
applicable.  After all, if I just ACCEPTed an invitation to a 6th
instance of the "Super Secret Product Ship Party Planning Committee"
then I would NOT expect to be rejected with some sort of "You tried to
store too many instances of this repeating component" failure since form
my POV I just accepted a single instance and not a bajillion repeats. 

6: It is unclear how RECUR-LIMIT is affected by NOT using EXPAND:TRUE. 
That is, is RECUR-LIMIT used to control the number of returned results
even if the caller did not request EXPAND:TRUE?  For example, if the CS
has RECUR-LIMIT:5 and a component repeats semi-weekly (W and F) then
would a query for all of this months entries fail even if I did NOT use
EXPAND:TRUE? 

Bruce 
PS: Note to CAP Editor(s): in RECUR-LIMITs Description, "EXPAND" is a
property, not a parameter.  Also, may I suggest that the Descriptoin be
split into 2 paragraphs at the point where it talks about the EXPAND
property (so each related property is in its own paragraph.) 
===========================================================================
Bruce Kahn                                INet:
[email-masked] 
Messaging & Collaboration                 Phone: 978.399.6496
IBM Software Group                         FAX: and nothing but the
FAX...
Standard disclaimers apply, even where prohibited by law...

---

From: Doug Royer <[email-masked]>
Date: Fri, 12 Dec 2003 15:17:24 -0700

Preston Stephenson wrote:

>We set RECUR-LIMIT to specify the max we will fan out on creation
I do not follow that last sentence. Are you saying that RECUR-LIMIT does 
not apply
to fetch because you have already expanded them? Or are you saying you
strip out RRULE and EXRULE and perhaps RDATE and EXDATE?
Or are you saying there are no more objects after RECUR-LIMIT?

(FYI 'fan out' has another unrelated meaning.).

>For query, EXPAND doesn't make sense and we ignore it (whether or not
Then the CUA will know this and also understand because you have
set STORES-EXPANED to TRUE.

>Can we abstract the limit to just be a limit (say QUERY-LIMIT) of any
If a CUA gets RECUR-LIMIT:L and STORES-EXPANDED:TRUE, then it that
is exactly you situation. The CUA will know that it will get up to 'L' 
instances
of a UID and that any over 'L' are not on that CS.

No need for a separate limit property or parameter  for STORES-EXPANED:TRUE
CS's, the limit can only apply to storage as such a CS does not expand 
the instances
at query time.

>For query, RECUR-EXPAND doesn't make sense, since the items are already
Yes, when it does apply is when STORES-EXPANDED:FALSE CS's
set RECUR-LIMIT:L, then you will know that you can just re-query 
starting from
the last date/time fetched and that you will get up to 'L' more.

>The problem arises if you have to have it be two limits (storage and
For a STORES-EXPANDED:TRUE CS, they will never be able to return more
than RECUR-LIMIT objects for a single UID. So they do not need a separate
fetch limit because you can not put in more objects in a EXPAND:TRUE
CS than what the CS itself stored when the CS expanded the instances.
They will always be the same for such a CS.

For a STORES-EXPANDED:FALSE CS, then they store the raw unexpended
object so no need for a separate store RECUR-LIMIT as it is meaningless.
For such a CS RECUR-LIMIT only applies to expanding of objects at fetch 
time.

One RECUR-LIMIT and STORES-EXPANDED covers it.

>We have no way of passing back a status saying, we would have created
No need as the CUA which sent the object already knows it is a recurring 
object (it
sent it) and it already had seen your RECUR-LIMIT capability.

>We have no way of adding to an existing recurrence set.
METHOD:ADD ?

Or did you mean once RECUR-LMIIT instances have been stored by your
implementation that your implementation can not ADD more?

Or did you men once RECUR-LMIIT instances have been stored by your
implementation that your implementation can not inform the CUA that
there should have been more? I might suggest that you store a private
object with the original recurrence rules in it, when the end is hit 
silently
expand SOME-PRIVATE-RECUR-LIMIT more, and then return or also include
the results to that? Then you can set RECUR-LIMIT large and
STORES-EXPANDED to TRUE and the CUA will never know that you
are somewhat expanding at run time.

>"IF-YOU-CREATE-A-RECURRENCE-ITEM-AND-IT-EXCEEDS-THE-LIMIT" what happens
The CUA knows it is a recurring object and it has seen your RECUR-LIMT.

-- 

Doug Royer                     |   http://INET-Consulting.com
-------------------------------|-----------------------------
[email-masked]                 | Office: (208)520-4044
http://Royer.com/People/Doug   |    Fax: (866)594-8574
                               |   Cell: (208)520-4044

               We Do Standards - You Need Standards