Subject: Re: Questions regarding recurrence rules

From: Sam Roberts <[email-masked]>
Date: Thu, 2 Jun 2005 11:27:30 -0400

Quoting [email-masked], on Tue, May 31, 2005 at 03:59:12PM +0200:
> Hi Guys,
> I have few questions regarding details of recurrence rules of rfc 2445. (Sorry 
> if I may sound more clueless than I really am. I have my assumptions, but I 
> don't want to influence any answers by bringing too much of my own arguments 
> and thoughts).

> 1) About the BYDAY rule part: Let's start with some examples
>   RRULE:FREQ=YEARLY;BYDAY=3SU
> Okay, that's the third sunday of the year. But what exactly is
>   RRULE:FREQ=YEARLY;BYDAY=5SU;BYMONTH=2
> Is this the fith sunday in the year, if it is also in february?

Yes.

   Each BYDAY value can also be preceded by a positive (+n) or negative
   (-n) integer. If present, this indicates the nth occurrence of the
   specific day within the MONTHLY or YEARLY RRULE. For example, within
   a MONTHLY rule, +1MO (or simply 1MO) represents the first Monday
   within the month, whereas -1MO represents the last Monday of the
   month. If an integer modifier is not present, it means all days of
   this type within the specified frequency. For example, within a
   MONTHLY rule, MO represents all Mondays within the month.

Whether it is an offset within a month or year depends on the FREQ, not on the
presence of any other modifiers. This might be a common mistake, Apple's iCal
does this wrong, I've found.

RFC2445bis should add an example for this since its an area of confusion.

> Or is it the fifth sunday in february? And it can be even less clear

No.

> from rfc 2445. What is
>   RRULE:FREQ=YEARLY;BYDAY=5SU;BYMONTH=2,7
> Is this the fifth sunday of the year, if it's in either February or July? 

Yes.

> Or is it the fifth sunday of both february and July?

No. That would be:

  RRULE:FREQ=MONTHLY;BYDAY=5SU;BYMONTH=2,7

> Or is it the fifth sunday of febrary and july together (i.e. in feb if the feb 
> in a leap year has 5 sundays, or the first sunday of july in all other 
> years)?

No way!

> If you think it's the latter, think about the following rule:
>   RRULE:FREQ=YEARLY;BYWEEKNO=4;BYDAY=5SU

> Is it the fifth sunday of all sundays that are in week #4?

Madness...

> Or is it the fifth sunday of the year if it is also in week #4?

Yes.

> Also, what does this rule mean:
>   RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU;BYYEARDAY=72

3rd month, and 2nd sunday in the year, and 72nd day in a year --> this won't
ever occur, of course.

> I suppose it doesn't mean the 72th day of the year, if that's the 2nd sunday 
> in March. Is there any way to express that recurrence in an RRULE?

How about:

   RRULE:FREQ=MONTHLY;BYMONTH=3;BYDAY=2SU;BYYEARDAY=72

> Also notice that rfc 2445 says "For example, within
>    a MONTHLY rule, +1MO (or simply 1MO) represents the first Monday
>    within the month, whereas -1MO represents the last Monday of the
>    month. ". So what does 
>   RRULE:FREQ=MONTHLY;BYYEARDAY=57,64;BYDAY=-1MO

Every month, the days that are the 57th&65th of the year (if in that month), and
that are on the last monday of the month.

> mean? The last monday of each month if it's also day #57 or #64 of the year 
> (which the quote implies), or the last monday each month in the set of all 
> days #57 and #64 of the year. In particular, in 2007, would the recurrence 
> set be Feb 27 (=#57 of the year, and the last monday in february) without any 
> occurence in march (march 5 is day #64, but it's not the last monday in the 
> month, since rfc 2445 says -1MO means last monday of the month)? Or would it 
> be Feb 27 (day #57 of the year, and last monday of all dates that match the 
> BYYEARDAY in the february interval) and March 4 ( day #64 of the year and 
> last monday of all dates in March that match the BYYEARDAY).
> 
> 
> 
> 2) In Section 4.3.10 RFC 2445 says "The COUNT rule part defines the number of 
> occurrences at which to range-bound the recurrence. The "DTSTART" property 
> value, if specified, counts as the first occurrence.". So look at this rrule:
> DTSTART;TZID=whatever:20050530T120000
> RRULE:FREQ=WEEKLY;COUNT=3;BYMONTH=6

I'm sorry, I can't find where it says it right now to quote, but I have the
very distinct memory that RFC2445 says that the DTSTART property MUST specify a
time that is valid according to the RRULE.

So, the above combination of DTSTART and RRULE is invalid, it doesn't mean
anything.

One of the problem with the way the iCalendar series of RFCs are laid out is
they tend to describe each small part in isolation, but the parts have complex
interactions, and where these are described is semi-random and hard to find.

> 3) About the DTSTART and recurrence rules: Is the DTSTART always taken to 
> match the rule (even if it doesn't fulfil the BY* parts)? In particular this 
> is important for EXRULES: Is the DTSTART always the first occurence of the 
> EXRULE? If that's the case, the DTSTART will always be excluded as soon as at 
> least one EXRULE is present. 
> And since exceptions overrule inclusions, there's no way to have an occurence 
> on the DTSTART in that case... Is this really the intended behaviour?

I haven't implemented EXRULE (or ever seen one), but your argument that the
DTSTART will always be excluded looks right to me.

That sounds like it would be a problem, but section 4.8.5.2 says this:

   The "EXRULE" property can be used to exclude the value specified in
                         ^^^
   "DTSTART". However, in such cases the original "DTSTART" date MUST
   still be maintained by the calendaring and scheduling system because
   the original "DTSTART" value has inherent usage dependencies by other
   properties such as the "RECURRENCE-ID".

They say "can", maybe should say "would always" :-)

It appears to me that DTSTART is made an exception, then, and must be kept.

So, don't make events whose alleged "start" is excluded from the set of events.
This actually seems reasonable, why not just start the event when it actually
starts?

Cheers,
Sam

---

From: Reinhold Kainhofer <[email-masked]>
Date: Thu, 2 Jun 2005 19:14:26 +0200

On Thursday 02 June 2005 17:27, Sam Roberts wrote:
> Whether it is an offset within a month or year depends on the FREQ, not on
> the presence of any other modifiers. 

At first I thought so, too. But then, there are two reasons which seem to 
contradict this:

1) The RFC speaks of "within the MONTLY or YEARLY RRULE" (notice that they 
don't say "within the frequency", like they do four lines below). So I 
wondered if this might be the only place where it's relevant in which order 
the BY* parts are evaluated.

2) All VTIMEZONEs (at least those used by Exchange, Apple's iCal, and by 
libical and so by evolution, korganizer etc.) use 
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU

> RFC2445bis should add an example for this since its an area of confusion.

Definitely.

> > Or is it the fifth sunday of febrary and july together (i.e. in feb if
> > the feb in a leap year has 5 sundays, or the first sunday of july in all
> > other years)?
>
> No way!

Actually, if the RFC really meant "within the RRULE", when that would be the 
correct interpretation. That's why I ask.

> Every month, the days that are the 57th&65th of the year (if in that
> month), and that are on the last monday of the month.

Depends. If you follow the RFC with the interpretation of "within the RRULE", 
you'll get:
In each month, that days that are days 57 or 64 of the year. Within this rrule 
(=within these days that match the rule so far), the last monday would be Feb 
27, 2007, and March 5, 2007....

> > mean? The last monday of each month if it's also day #57 or #64 of the
> > year (which the quote implies), or the last monday each month in the set
> > of all days #57 and #64 of the year. In particular, in 2007, would the
> > recurrence set be Feb 27 (=#57 of the year, and the last monday in
> > february) without any occurence in march (march 5 is day #64, but it's
> > not the last monday in the month, since rfc 2445 says -1MO means last
> > monday of the month)? Or would it be Feb 27 (day #57 of the year, and
> > last monday of all dates that match the BYYEARDAY in the february
> > interval) and March 4 ( day #64 of the year and last monday of all dates
> > in March that match the BYYEARDAY).

> > 2) In Section 4.3.10 RFC 2445 says "The COUNT rule part defines the
> > number of occurrences at which to range-bound the recurrence. The
> > "DTSTART" property value, if specified, counts as the first occurrence.".
> > So look at this rrule: DTSTART;TZID=whatever:20050530T120000
> very distinct memory that RFC2445 says that the DTSTART property MUST
> specify a time that is valid according to the RRULE.

No, definitely not. E.g. even look at the examples given in RFC 2445. It 
explicitly gives an example where the DTSTART does NOT match the RRULE:

   Every other week on Monday, Wednesday and Friday until December 24,
   1997, but starting on Tuesday, September 2, 1997:

     DTSTART;TZID=US-Eastern:19970902T090000
     RRULE:FREQ=WEEKLY;INTERVAL=2;UNTIL=19971224T000000Z;WKST=SU;
      BYDAY=MO,WE,FR
     ==> (1997 9:00 AM EDT)September 2,3,5,15,17,19,29;October
     1,3,13,15,17
         (1997 9:00 AM EST)October 27,29,31;November 10,12,14,24,26,28;
                           December 8,10,12,22

> > match the rule (even if it doesn't fulfil the BY* parts)? In particular
> > this is important for EXRULES: Is the DTSTART always the first occurence
> > of the EXRULE? If that's the case, the DTSTART will always be excluded as
> > soon as at least one EXRULE is present.
> > And since exceptions overrule inclusions, there's no way to have an
> > occurence on the DTSTART in that case... Is this really the intended
> > behaviour?
>

Yeah, the "can" is the issue. If I have a daily event that starts on a monday, 
and add an exrule, which excludes all events on thursdays and fridays (here 
the exrule uses the DTSTART, but it doesn't match the BY* parts). So, does 
the DTSTART always match the EXRULE by definition (i.e. is the first monday 
excluded, and all thursdays and fridays afterward), or does the first monday 
not match the EXRULE, and only all thursdays and fridays are excluded.

> So, don't make events whose alleged "start" is excluded from the set of
> events. This actually seems reasonable, why not just start the event when
> it actually starts?

Good question. I haven't thought of this too much, but I haven't yet seen a 
case where it would really make a different (if I understand RECURRENCE-ID 
correctly, I don't see where this should matter, although RFC 2445 says it 
does).

Cheers,
Reinhold
-- 
------------------------------------------------------------------
Reinhold Kainhofer, Vienna University of Technology, Austria
email: [email-masked], http://reinhold.kainhofer.com/
 * Financial and Actuarial Mathematics, TU Wien, http://www.fam.tuwien.ac.at/
 * K Desktop Environment, http://www.kde.org, KOrganizer / KPilot maintainer

---

From: Doug Royer <[email-masked]>
Date: Thu, 02 Jun 2005 13:15:27 -0600

This is more documentation supporting my assertion that recurrence
rules are not as portable as many claim.

I would suggest using RDATE's and no RRULE or EXRULE. They are
easy to unwind, only after clearly defined and consistently used.

-- 

Doug Royer                     | http://INET-Consulting.com
-------------------------------|-----------------------------

               We Do Standards - You Need Standards

---

From: Reinhold Kainhofer <[email-masked]>
Date: Fri, 3 Jun 2005 00:41:52 +0200

> easy to unwind, only after clearly defined and consistently used.

Using only RDATE means loosing data (namely the actual intent) and the 
possibility to edit the data later on. 
If you have a course with the following data (from a student's perspecitive):
Mar 2, 7, 9, 14, ...
Can you be sure that it is really meant as a course that happens every monday 
and wednesday? No, you'll have to look at your calendar for every date. If 
you see that the recurrence is every monday and friday, you don't need to 
look at your calendar, but sooner or later you'll remember.

Also, in KOrganizer we also use iCalendar to store the whole calendar on disk, 
and there we absolutely need a way to store the way in which a date is 
recurring. Simply discarding all RRULES and using RDATES instead might be a 
solution for sending invitations, but it's not a solution for storing on 
disk, and also not for exchanging events between two calendaring applications 
(i.e. import/export of events). 

Cheers,
Reinhold

-- 
------------------------------------------------------------------
Reinhold Kainhofer, Vienna, Austria
email: [email-masked], http://reinhold.kainhofer.com/
 * Financial and Actuarial Mathematics, TU Wien, http://www.fam.tuwien.ac.at
 * K Desktop Environment, http://www.kde.org/, KOrganizer / KPilot maintainer

---

From: "Laird Nelson" <[email-masked]>
Date: Fri, 3 Jun 2005 10:46:48 -0400

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of 
> Reinhold Kainhofer
> 2) All VTIMEZONEs (at least those used by Exchange, Apple's 
> iCal, and by libical and so by evolution, korganizer etc.) 
> use RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU

(Mr. Kainhofer is implying that they got it wrong; something with which I
agree.  Mr. Roberts points out in another email that really, yes, that RRULE
should read RRULE:FREQ=MONTHLY;BYMONTH=10;BYDAY=-1SU, with which I also
agree.)

However, consider this timezone definition excerpt from RFC 2445 (line 3719
in the .txt version):

     BEGIN:STANDARD
     DTSTART:19671029T020000
     RRULE:FREQ=YEARLY;BYDAY=-1SU;BYMONTH=10
                ^^^^^^
The RRULE here is identical to the "wrong" way of doing it that Mr.
Kainhofer describes above, and probably explains why Microsoft and Apple and
the libical folks all got it "wrong" in the same manner (i.e. because the
"wrong" way is the one that is given as an example throughout RFC 2445).
Apparently this is the way that RFC 2445 encourages you to write RRULEs
featuring integer-preceded BYDAY modifiers in conjunction with other
modifiers.

If this is correct and can be generalized, then it implies that BYDAY
modifiers apply to the current set of evaluated occurrences, not to the
FREQ.  This contradicts what Mr. Roberts was saying earlier.  The upshot is
that I'm very confused.  Perhaps my recurrence rule expansion library will
need to be written to operate in one of two different modes?

I am assuming from the relative silence on all of these discussions that Mr.
Dawson and Mr. Stenerson are no longer on this list and cannot chime in to
offer the framers' intent.

Best,
Laird

---

From: Doug Royer <[email-masked]>
Date: Fri, 03 Jun 2005 10:38:23 -0600

The "DTSTART" property value, if

Dawson & Stenerson          Standards Track                    [Page 42]

RFC 2445                       iCalendar                   November 1998

    specified, counts as the first occurrence.

> 
> After re-reading the rfc several times, and reading a thread on the libical 
> mailing list from July 2002 
> (http://www.softwarestudio.org/mail-archives/libical/0680.html, see the 
> thread following this initial mail for the relvant comments), I don't think 
> that the DTSTART is meant to be automatically part of the recurrence set 
> generated by the RRULE (only if it matches the BY* parts). So, basically, 
> this means that 
>   DTSTART;VALUE=DATE:20050603
>   RRULE:FREQ=DAILY
>   EXRULE:FREQ=WEEKLY;BYDAY=MO,WE;COUNT=3
> excludes only Mo June 6, We June 8, and Mo June 13 (the count applies only to 
> the nr of dates that actually match the rule, as explained above; and the 
> DTSTART is not automatically excluded by the EXRULE, since otherwise the 
> phrase "can be used to exclude ... DTSTART" doesn't make sense).
> 
> Can anyone confirm this?
> 
> Cheers,
> Reinhold
> 

--

---

From: Reinhold Kainhofer <[email-masked]>
Date: Fri, 3 Jun 2005 20:24:36 +0200

Hi Doug,

>     specified, counts as the first occurrence.

Yes, and? I was discussing exactly this sentence. Did you read what I wrote in 
the paragraph before the one that you quote?

On Friday, 3. Juni 2005 17:47 I wrote:
> I thought about this issue some more, and I tend to think that the phrase
> "if specified" in the RFC 2445 is actually meant to mean "if the DTSTART
> matches the recurrence rule" (not if the DTSTART is present in the VEVENT
> at all).

The issue here is what "if specified" means. 
It can't mean "if a DTSTART property is available", because if there is no 
DTSTART (implied or explicit), then an RRULE doesn't make sense at all. So 
what else can it mean?

It's definitely there for a reason, so the only explanation I found was that 
"specified" in that context was meant to be "if the first recurrence of the 
RRULE matches the DTSTART, it counts towards the COUNT". 
This might not seem relevant (or rather, it might seem pretty obvious), but if 
you look at the larger picture it serves as clarification: 
The whole recurrence set is generated by DTSTART plus the recurrences 
generated by the RRULEs and the RDATEs, minus the EXRULES and EXDATEs. 
Also, rfc 2445 says "Duplicate instances are ignored". The phrase you quote 
simply means that although the the dtstart will be ignored in the RRULE 
(because it's already in the recurrence set due to its being DTSTART), it 
still counts towards count.

Also notice the wording: it speaks of the property value, not of the property 
itself. The property needs to be specified (which implies that a value for 
DTSTART is specified in the VEVENT), so the question here is specified *by 
what* ? In the event? (No, I already negated this and brought arguments for 
this) By the recurrence rule? 

At least that's the explanation I found for that exact phrase in the RFC. I 
may be wrong, but I haven't yet heard a better explanation to what "if 
specified" really means there. Please correct me, if I'm wrong!

Cheers,
Reinhold

>
> > After re-reading the rfc several times, and reading a thread on the
> > libical mailing list from July 2002
> > thread following this initial mail for the relvant comments), I don't
> > think that the DTSTART is meant to be automatically part of the
> > recurrence set generated by the RRULE (only if it matches the BY* parts).
> > So, basically, this means that
> > excludes only Mo June 6, We June 8, and Mo June 13 (the count applies
> > only to the nr of dates that actually match the rule, as explained above;
> > and the DTSTART is not automatically excluded by the EXRULE, since
> > otherwise the phrase "can be used to exclude ... DTSTART" doesn't make
> > sense).
> >
> > Can anyone confirm this?
> >
> > Cheers,
> > Reinhold

---

From: Reinhold Kainhofer <[email-masked]>
Date: Fri, 3 Jun 2005 17:47:29 +0200

Hello guys,
I did some more digging in various mailing list archives, re-read rfc 2445 a 
few times and throught a little more on the issues.

On Tuesday 31 May 2005 15:59, Reinhold Kainhofer wrote:
> 2) In Section 4.3.10 RFC 2445 says "The COUNT rule part defines the number
> of occurrences at which to range-bound the recurrence. The "DTSTART"
> property value, if specified, counts as the first occurrence.". 
[...]
> Now if the DTSTART is not specified the quote from above would imply that
> the start doesn't count as a recurrence, so looking at
> DTEND;TZID=whatever:20050530T150000
> DURATION:PT3H
> would give the following resulting recurrence dates?
>   1) 20050530 (DTSTART is always the first occurence, )
>   2) 20050630
>   3) 20060630
>   4) 20070630 (DTSTART doesn't count towards COUNT)

I thought about this issue some more, and I tend to think that the phrase "if 
specified" in the RFC 2445 is actually meant to mean "if the DTSTART matches 
the recurrence rule" (not if the DTSTART is present in the VEVENT at all). In 
particular this would mean that
  DTSTART;VALUE=DATE:20050603
  RRULE:FREQ=WEEKLY;BYDAY=MO,WE;COUNT=3
gives the recurrence set June 3 (DTSTART), June 6, 8, and 13 (the first three 
occurences defined by the RRULE).

Also, in the rule
  DTSTART;VALUE=DATE:20050603
  RRULE:FREQ=WEEKLY;BYDAY=WE,FR;COUNT=3
the recurrence set is generated by the DTSTART plus the first three occurences 
of the RRULE (June 3, 8 and 10). As a result the recurrence set would only be 
June 3, 8, and 10 (June 3 is duplicate from the DTSTART and the RRULE, but 
rfc 2445 says that means it only appears once).

> this is important for EXRULES: Is the DTSTART always the first occurence of
> the EXRULE? If that's the case, the DTSTART will always be excluded as soon
> as at least one EXRULE is present.

Can anyone confirm this?

Cheers,
Reinhold