Subject: Re: Can we fix BYSETPOS? (Proposed Restrictions)

From: [email-masked]
Date: Mon, 11 Jun 2001 13:53:47 -0400

Eric Busboom <[email-masked]> wrote on 06/11/2001 11:32:45 AM:
> Or, you could just add another exrule:
> 
>  RRULE:FREQ=DAILY;BYHOUR=9,10,11,12,13,14,15,16;BYMINUTE=0,20,40
>  EXRULE:FREQ=DAILY;BYHOUR=12;BYMINUTE=0,20;
>  EXRULE:FREQ=DAILY;BYHOUR=16;BYMINUTE=40;

Or I could have used several RRULEs:

    RRULE:FREQ=DAILY;BYHOUR=9,10,11,13,14,15;BYMINUTE=0,20,40
    RRULE:FREQ=DAILY;BYHOUR=12;BYMINUTE=40
    RRULE:FREQ=DAILY;BYHOUR=16;BYMINUTE=0,20

or other several mixes.  The point is that BYSETPOS is useful for the 
combination I put together for a sub-day BYxxx subcomponent. 

> Certainly there are many rules that can be expressed with BYSETPOS -- 
for
> any restriction you can always invent a counter example. 

Which means that it is useful for those restriction cases you proposed.

>                                                      But we only need
> to implement each significant case once; we don't need to support every
> fringe case in a wide variety of ways.

Fringe cases are like needing to resolve how a CUA should behave if a 
repeat instance occurs during a TZ shift given its frequency of occurance. 
 If you mean that any sub-day BYxxx subcomponent is fringe then why not 
just argue to remove them entirely??  I see no logical explaination for 
removing BYSETPOS for recurrences that are sub-day (apart from the value 
range is +/-31M). 

If the grammar can express a set that large (or at a sub-day recurrence at 
all) then it is logical to think that someone may want (or even NEED) the 
ability to filter on particular instances just like in the super-day 
recurrence cases.

> Can you identify some rule of general significance that absolutely
> requires BYSETPOS?  If not, then we can keep the proposed BYSETPOS
> restrictions without impairing the utility of the recurrence rules.

Sure, for the same reason that any super-day recurrence set needs to have 
BYSETPOS; the sub-day recurrence set has the _exact_ same needs. 

> > There may be other usage models (ie: SKiCAL or some future ones) that
> > may need it to have sub-day repeats and need BYSETPOS to achieve the
> > same results that day (or larger) repeat intervals enjoy.
> 
> Why do we care if there are other situations where an implementation
> COULD use BYSETPOS? We should only care if they MUST use it. Given the
> problems we have already identified it, we should not keep unnecessary
> flexibility.

Whoa Hoss!  Thats a big step between COULD and MUST and I havent seen any 
justification for your distinctions.  BYSETPOS can occur in _ANY_ 
recurrence rule that has at least 1 BYxxx subcomponent in it (otherwise 
its non-sensical since the 'set' is just 1 instance).  If you need to see 
why it is necessary to have, revisit any of the examples in the RFC 2445 
(starting on p118) and just change them to be sub-day instead of super-day 
recurrence intervals. 

Im not sure exactly what "problems" we have already identified apart from 
"What is the 'set' referred to by BYSETPOS?" and the fact that it was 
published assuming super-day recurrence intervals (hence the limit of 
+/-366).  The former is simply a case of the authors not being clear in 
their text (and the WG not catching this during Last Call).  The latter is 
an acknowledged oversight that needs to be corrected.

<HAT IMPLEMENTOR=ON METHOD_REVIEWER=OFF>
What is the likelyhood that the will be some C&S system that uses sub-day 
recurrence?  Well given the model for SKiCAL Id say that it is not far 
fetched to assume it will happen.
What is the likelyhood that I make a CUA that generates something as 
abusive as the "Repeat every second, every day of the year" and that uses 
a BYSETPOS to select them all (making it useless in this case really)? 
Very very very very very (did I say very yet?) unlikely.
Am I likely to support an overly large BYSETPOS (or for that matter 
'abusive' recurrence rule)?  Probably not.  I have throttles in my code to 
prevent this kind of abuse (can any one say C&S Spam??)
</HAT>

So I ask you again then that if there there is a model that may need it 
(ie: SKiCAL or something else that will come down the pipeline) then why 
NOT keep it in for completeness?  If your (the royal you) implementation 
does not support sub-day recurrence anyway you wont need to be worried 
about "BYSETPOS=31000005,3100006" either now will you? 

Since we CAN generate a set of repeat instances with > 366 value, we NEED 
to have BYSETPOS be able to select them all.  Otherwise we could wind up 
with some instances that are not selectable or filterable.  I do not think 
we should 'revise' the grammar to be such that we need to 'rerevise' it 
later when someone new shows up with the need to have what we took out 
now.

> And that restricts interoperability. If every rule that uses BYSETPOS 
can
> also be implemented without it, and some implementation reject some
> BYSETPOS rules, then we have an unnecessary interoperability problem --
> those that use BYSETPOS unnecessarily will not be understood by those 
that
> do not accept BYSETPOS.

I am nearly tempted to agree w/your claim about alternate ways to code up 
recurrence rules w/o BYSETPOS but then I tried to do the "real world" one 
from the RFC 2445 that matches my pay schedule: "the last work day of the month".  This is currently a very simple rule:

     RRULE:FREQ=MONTHLY;BYDAY=MO,TU,WE,TH,FR;BYSETPOS=-1

but in 5 min of chewing on it I could not figure a different way to do it 
in RRULEs unless I multiple RRULEs per year that 'know' what months have 
workdays as 'last days' and which do not, and then repeated that set out 
for as many years as the sample which was infinite!.  Perhaps you can give 
me an alternate w/o a BYSETPOS that resolves to the same set of dates w/o 
it being several pages long...

There will always be some interop issues but I suspect they are more 
related to implementation architectures rather than to 'flaws' in the 
standards.  If a C&S system has absolutely no concept of seconds (ie: "We 
only allow scheduling on minute boundaries") then does this mean we should 
remove BYSECOND from our recurrence grammar? No.  What we need is a way 
for them to interop as best that we can hope for and have some logical 
behaviour for dealing with these cases (and that does belong in our 
specs!)

> We already have strong evidence that this will be the case -- CPL will 
not
> be able to accept every BYSETPOS rule under the current spec for RECUR.

And Outlook wont accept RDATEs.  Should we drop them from the spec?  Of 
course not!  The spec needs to have a clear and concise means for dealing 
with interop issues like this (ie: REQUEST-STATUS in our case).

> It is much better that we restrict BYSETPOS now and avoid these 
problems.

Not QED yet...  If the grammar can generate a set of instances that cannot 
be filtered or accessed by BYSETPOS then we have a problem, not a 
solution.

> So, Bruce, can you demonstrate that there exist common rules that cannot
> be constructed with the proposed restrictions to BYSETPOS? 

Sure, see above for the "Last workday of the month" example.

>                                                             If not, then
> let's keep the restrictions.

Guess then that the new restrictions are not yet in... 

As an implementor I like to see some restrictions in place to make my 
coding task easier.  They allow me to better tune my engine to be faster 
or allow me to cut some corners in some areas.

As the Method Reviewer I do not like to see changes that are overly heavy 
handed and not forward looking.  The proposed restrictions for sub-day 
recurrence just set off alarms to me when I look forward to future reuse 
of the grammar.

Bruce
===========================================================================
Bruce Kahn                                INet: [email-masked]
Iris Associates                          Phone: 978.392.5335
Westford, MA, USA 01886                    FAX: and nothing but the FAX...
Standard disclaimers apply, even where prohibited by law...

---

From: Eric Busboom <[email-masked]>
Date: Mon, 11 Jun 2001 12:38:26 -0700 (PDT)

> combination I put together for a sub-day BYxxx subcomponent.

It is the useful features that have resulted in recurrence rules being
absurdly complex. We should be focusing on the mandatory features.

The problem we have is that if we allow BYSETPOS to operate over the
entire range of seconds in a year, it is possible for CUAs to
generate rules that CPU-bound servers will reject because they are too
complex to evaluate in a small amount of time.

I believe that we can restrict recurrence rules to only using BYSETPOS for
day-scale events, thus eliminating the complexity problem for all but the
most feeble servers. I also think that any plausible rule that would use
BYSETPOS for sub-day intervals can be re-written without BYSETPOS, so the
restriction does not reduce the functionality of recurrence rules.

You are arguing that BYSETPOS for sub-day intervals is useful and that
there may be cases where some CUA will want to use them. That same
argument could support embedding TCL code in recurrence rules. Neither
case is necessary, and both of them cause interoperability problems
related to computational complexity.

The proposed restrictions are a compromise between a complexity problem
that we know exists and a set of future possibilities that we suppose
exist. We can't design for a vague an unknown future; if you've got some
plausible use cases that cannot be implemented without BYSETPOS on a
sub-day scale, then we can do some analysis. Otherwise, we are just
guessing.

So, can you show some rules for a plausible scenario that cannot be
implemented without BYSETPOS on a sub-day interval?

eric.

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 11 Jun 2001 16:51:49 -0400 (EDT)

> recurrence just set off alarms to me when I look forward to future reuse 

I'm not (overly) concerned about keeping coding tasks easy -- it's a *good
idea*, of course, but it's not an overriding concern.

I am, however, concerned about whether it's possible to implement a
specification efficiently.  (Where "efficiently" means using a predictable,
bounded amount of CPU time and memory to perform standard operations in the
specification.)

In iCalendar COS, we have a fairly classic "pick-two" situation.  We can
have at most two of:
1. Efficient implementation
2. Predictable interoperability
3. Full generality

RFC 2445, in theory, decided to drop point #1, though I doubt this was
realized at the time.  Implementors, in practice, usually dropped #3 and
thus #2.

What I'm proposing, instead, is to drop #3 formally, and thus keep #1 and
#2.

Here's some proposed wording as to how I'd accomplish this, while still
leaving room for future generality in appropriate environments.  I hope this
would be generally acceptable.

  Applications which interpret iCalendar COS objects MUST support any COS
  object which conforms to the following restrictions:

     [... list of restrictions, to be determined by the group.  This
     includes the restricted BYSETPOS rules.  It should be easy and
     efficient to check whether a COS object complies with the
     restrictions.]

  Applications which interpret iCalendar COS objects MAY also support COS
  objects which violate or exceed one or more of the above limitations.

  Applications which generate iCalendar COS objects SHOULD NOT generate a
  COS object which violates or exceeds any of the above limitations, unless
  they know, through means not defined in this specification, that every
  application which will attempt to interpret the COS object can do so
  sucessfully.

This solves the "what SkiCal needs" problem -- if SkiCal needs (e.g.)
sub-day BYSETPOS rules, it can define a looser set of restrictions for
SkiCal.

Will this proposal let us come to some consensus?  Are there thoughts about
what an appropriate list of these restrictions would be?  (I could
contribute some for the RRULE family, but not for the rest of the COS, which
I don't know very well).

-- 
Jonathan Lennox
[email-masked]

---

From: "Graham Gilmore" <[email-masked]>
Date: Mon, 11 Jun 2001 17:26:23 -0400

----- Original Message -----
From: "Eric Busboom" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: 11 June, 2001 15:38
Subject: Re: Can we fix BYSETPOS? (Proposed Restrictions)

> implemented without BYSETPOS on a sub-day interval?

    The obvious example would be to take Bruce's BYSETPOS poster case
and throw in a sub-day part:

    RRULE:FREQ=MONTHLY;BYDAY=MO,TU,WE,TH,FR;BYHOUR=12,13;BYSETPOS=-1,-2

    This isn't a terribly unlikely scenario, either.

    It seems that BYSETPOS is useful primarily when dealing with cases
where one time component doesn't "divide" evenly into another, for
example, not all months have the same # of days; the number of days &
weeks in a year varies from year to year; days with a timezone shift
don't have 24 hours (the infamous 1:30 am event could be specified
thusly to specify the first instance of 1:30 on appropriate days:
RRULE:FREQ=DAILY;BYHOUR=1;BYMINUTE=30;BYSETPOS=1).

    Graham

---

From: Eric Busboom <[email-masked]>
Date: Mon, 11 Jun 2001 16:46:48 -0700 (PDT)

Without BYSETPOS:

 RRULE:FREQ=MONTHLY;BYDAY=MO,TU,WE,TH,FR;BYMONTHDAY=-1;BYHOUR=12,13;

eric.

---

From: "Graham Gilmore" <[email-masked]>
Date: Mon, 11 Jun 2001 20:42:07 -0400

----- Original Message -----
From: "Eric Busboom" <[email-masked]>
To: "Graham Gilmore" <[email-masked]>
Cc: <[email-masked]>
Sent: 11 June, 2001 19:46
Subject: Re: Can we fix BYSETPOS? (Proposed Restrictions)

>  RRULE:FREQ=MONTHLY;BYDAY=MO,TU,WE,TH,FR;BYMONTHDAY=-1;BYHOUR=12,13;

    In this case, if the last day of the month is not one of
MO,TU,WE,TH,FR, that month would be skipped (since the intersection of
BYMONTHDAY and BYDAY is empty for that month).  Like Bruce, I tried for
several minutes to come up with an alternative representation, and I
couldn't.

    Graham

---

From: Eric Busboom <[email-masked]>
Date: Mon, 18 Jun 2001 13:05:26 -0700 (PDT)

> several minutes to come up with an alternative representation, and I

Ah -- you are right. I can't think of a way to solve it either, and it is
a very practical case. Looks like I'll have to abandon my proposal for
restrictions.

eric.