Subject: RE: RECURRENCE-ID discussion

From: [email-masked]
Date: Mon, 4 Aug 2003 16:15:19 -0400

I have sent another "nudge" note to the author in hopes of getting his 
interpretation and/or understanding of what the drafts "mean."   Pretty 
soon I'm going to have to get into "Mom" mode and that won't be pretty. 
So, let's see if I can get the text/understanding from them this week. 

Satya Vempati <[email-masked]>
Sent by: [email-masked]
08/04/2003 15:19

        To:     "[email-masked]" <[email-masked]>
        cc: 
        Subject:        RE: RECURRENCE-ID discussion

Unfortunately the RFCs are not consistent. You can quote one part of the
RFC which agrees with your position, and others can quote the other parts
that seem to buttress theirs.
-----------------
For example, (RFC 2446, Section 3.7.1)

If the "Organizer" wishes to
change the "DTSTART", the original "DTSTART" value is used for
"RECURRENCE-ID" property and the new "DTSTART" and "DTEND" values
reflect the change. Note that after the change has occurred, the
"RECURRENCE-ID" has changed to the new "DTSTART" value.
-------------------

Additionally, portions of the document are not very clear and open to
multiple interpretations:

"When the definition of the recurrence set for a calendar component
changes...the "RECURRENCE-ID" for a given recurrence instance might also
change"

What is meant by the "definition of a recurrece set?" One can also argue
that the recurrence set changes whenever something is added to or deleted
from it.

Because of such inconsistencies, if we clearly understand the intent of
the original authors and agree on it, the minor textual inconsistencies
could be resolved amicably.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Monday, August 04, 2003 8:37 AM
To: [email-masked]
Subject: Re: RECURRENCE-ID discussion

Satya asked on 07/31/2003 06:43:38 PM:
> Is there an update on when the iCal authors will post their thinking on
> the subject?

I had hoped that reading the RFCs and searching the archives would have
put this to rest once and for all but perhaps not for some.  So let me
take some time to review both the RFC text and to technically analyze why
the iCalendar specified a fixed RECURRENCE-ID.  This is going to be a
somewhat long posting in order to cover both iCalendar, iTIP and the
claims that purportedly define a changing RECURRENCE-ID model.

First, lets start with a review of the RFCs.  The 2 RFCs in question are
2445 (iCalendar) and 2446 (iTIP).  iCalendars role is to define the
various properties and property parameters as well as specify their
behaviour / intent.  iTIPs role is to give the iCalendar properties
semantic meaning that all implementations can follow to interop.  So lets
start with reviewing the actual definition of RECURRENCE-ID (Section
4.8.4.4 Recurrence ID).

Description: The full range of calendar components specified by a
recurrence set is referenced by referring to just the "UID" property
value corresponding to the calendar component. The "RECURRENCE-ID"
property allows the reference to an individual instance within the
recurrence set.

[Snip]

The date/time value is set to the time when the original recurrence
instance would occur; meaning that if the intent is to change a
Friday meeting to Thursday, the date/time is still set to the
original Friday meeting.

The "RECURRENCE-ID" property is used in conjunction with the "UID"
and "SEQUENCE" property to identify a particular instance of a
recurring event, to-do or journal. For a given pair of "UID" and
"SEQUENCE" property values, the "RECURRENCE-ID" value for a
recurrence instance is fixed. When the definition of the recurrence
set for a calendar component changes, and hence the "SEQUENCE"
property value changes, the "RECURRENCE-ID" for a given recurrence
instance might also change. [Snip]

Some folks seem to blindly ignore the 2nd (above, 3rd in the actual RFC)
paragraph which clearly and unequivocally says that when a reschedule
takes place the RECURRENCE-ID value "is still set to the original" value!
This of course flat out defeats the claim that "RECURRENCE-ID changes on
each reschedule" as some have claimed so its easy to see why overlooking
it happens.

In looking at the oft cited last paragraph you will find that the key
phrase is "When the definition of the recurrence set for a calendar
component changes...the "RECURRENCE-ID" for a given recurrence instance
might also change".  The important bit that seems to be misread is
"definition of the recurrence set"; it does not say "definition of the
recurrence _instance_".   So its likely that this is partly to blame for
some misinterpreting the description to mean "Whenever the instance is
rescheduled, the recurrence instance also changes"  This does make a
couple minor but not insignficant changes to the phrasing such as changing
"might also change" into "always changes" but these changes are best
ignored if you want to change the model.

Of course these changes would conflict with the entire paragraph before it
so some would just ignore it in favor of their reinterpreatation of the
prose.  However it must be noted that iCalendar clearly proscribes a
"fixed" RECURRENCE-ID in at least 2 places.

In addition Doug has claimed that "'original' means currently booked
version" but thats nowhere near the meaning of the word original.  Instead
of arguing a common sense meaning Ill instead defer to Merriam-Webster who
define it as:

Main Entry: 2original
Function: adjective
Date: 14th century
1 : of, relating to, or constituting an origin or beginning : INITIAL <the
original part of the house>
2 a : not secondary, derivative, or imitative b : being the first instance
or source from which a copy, reproduction, or translation is or can be
made

So "original" means the the same as the "initial" value, NOT the "latest"
or "most recent".  This definition matches the paragraph in describing how
RECURRENCE-ID stayed at the Friday date/time.  The reason that a fixed
RECURRENCE-ID was specified over a varying one will become evident as we
move into analyzing iTIP so lets do that now.

iTIP defines its role in relation to iCalendar as:

iTIP complements the iCalendar object specification by adding
semantics for group scheduling methods commonly available in current
calendar systems.

so its role is NOT to impart (or modify) any defintion to the iCalendar
properties or property parameters, thats the function of iCalendar.  The
semantics for "group scheduling methods" are done in the various iTIP
methods or messages.  iTIP messages can be sent over any kind of media or
using assorted methods so the protocol had to first define some guidelines
to apply to all messages to properly deal with missequencing  on the
receiving end.  That is done in Section 2.1.5 Message Sequencing, before
any direct discussion of individual messages was done.  In Section  2.1.5
Message Sequencing we see:

1.  The primary key for referencing a particular iCalendar component
is the "UID" property value. To reference an instance of a
recurring component, the primary key is composed of the "UID" and
the "RECURRENCE-ID" properties.

2.  The secondary key for referencing a component is the "SEQUENCE"
property value.  For components where the "UID" is the same, the
component with the highest numeric value for the "SEQUENCE"
property obsoletes all other revisions of the component with
lower values.

Thus for repeating instances the primary key value used to find the
instance the message refers to is the UID / RECURRENCE-ID pair.  Once this
pair is used to find the instance in question the SEQUENCE property can be
used to determine if the message is newer or older than the one the
recipient has.  Or in iTIP terms, tell if the message has already been
obsoleted or if it should obsolete the recipients copy.  Nothing here
claims that RECURRENCE-ID changes on each reschedule and in order to be
fault tolerant and easily recoverable RECURRENCE-ID should not change
(iCalendar prose not withstanding).  This can be seen in the prose found
in other sections in iTIP such as Sections 3.2.2 REQUEST, 3.2.2.1
Rescheduling an Event, 3.2.2.2 Updating or Reconfirmation of an Event, or
even the often misquoted Section 4.7.2 Bad RECURRENCE-ID.

Section 3 of iTIP is where the actual protocol definitions happen.  Its
where iTIP defines what a meeting request or response look like, etc.
Section 4 of iTIP is where the authors provided assorted examples of
actual iTIP messages from Section 3.  As such its merely the demonstrative
part of the iTIP RFC and does not carry the same weight.  The authors were
always grumbling about the inclusion of examples ("If we put in an example
of X then everyone will code to that and not to the text" as they so often
lamented in meetings, emails and here on the list) and now this has come
back to haunt us based on a misreading of iCalendar and a line or two in
the examples section of iTIP (not even in the defintion section of iTIP).
In any case, lets continue the analysis promised.

iTIP Section 3.2.2 REQUEST in part reads:

The "UID" and "SEQUENCE" properties are used to distinguish the
various uses of the "REQUEST" method. If the "UID" property value in
the "REQUEST" is not found on the recipient's calendar, then the
"REQUEST" is for a new "VEVENT" calendar component. If the "UID"
property value is found on the recipient's calendar, then the
"REQUEST" is for a rescheduling, an update, or a reconfirm of the
"VEVENT" calendar component.

and if you apply the rules from Section 2.1.5 to this you should get that
if the UID/RECURRENCE-ID are not found in the recipients calendar then its
a new REQUEST, otherwise its an update.  Implicitly this concurs with
iCalendar and Section 2.1.5 because if the RECURRENCE-ID changed and one
or more iTIP REQUESTs were missed (or delayed!) then the recipient has NO
way to properly perform this match and thus they would treat the REQUEST
as a new invitation rather than a reschedule.

In case that was not clear enough for some Ill try to make it as clear as
I can: If I send you a REQUEST for a particular UID and RECURRENCE-ID that
you do not have then you are to treat it as a new invitation by the
definition of REQUEST.  So you want the UID / RECURRENCE-ID values to
never change from that point on or else you will think you got _another_
new invitation rather than a rescheduling of the one you already have!

If RECURRENCE-ID did change on each reschedule (iCalendar prohibitions
aside) then you have NO way to distinguish between a new invitation and
the case where you missed one or more reschedules.  This means that for
the model where RECURRENCE-ID changes on each reschedule, any single
delayed or missequenced REQUEST can _only_ be recovered by rescying up
_all_instances_ of the repeat set!  In addition if the messages were just
missequenced this results in _tons_ of extra wasted workflow thrashing.

Lets compare the iCalendar defined fixed RECURRENCE-ID model to the
changing RECURRENCE-ID model of some and see how they deal with
missequenced messages.  The scenario is I invite you to an instance of a
repeating meeting.  After the initial invitation I reschedule it two times
to accomodate other invitees.  So for simplicity in references Ill refer
to the initial invitation as SEQUENCE:0 and the two reschedules as
SEQUENCE:1 and SEQUENCE:2 respectively.  Because of network/server issues
you recieve them in the order of SEQUENCE:0 then SEQUENCE:2 and then
SEQUENCE:1; a not uncommon case when using email (aka iMIP).

First the fixed RECURRENCE-ID model analysis:

When you receive the SEQUENCE:0 invitation you apply Section 2.1.5 and
3.2.2 and determine that this is a new meeting invitation which you take
some action on to get it onto your calendar.

You then receive SEQUENCE:2 and you can match the UID / RECURRENCE-ID
values and determine that the message is newer and that it obsoletes your
current copy.  You may notice that its SEQUENCE value is more than 1
different but since the higher valued SEQUENCE "obsoletes all other
revisions of the component withlower values" it is not an issue that you
did not see SEQUENCE:1 (yet).  You can simply take action on it and send
back a REPLY with the correct UID / RECURRENCE-ID / SEQUENCE:3 info on it
and we are in sync.

You later receive the SEQUENCE:1 REQUEST.  You apply Section 2.1.5 and
determine that your version is newer and as such this REQUEST is obsoleted
and should be ignored.  No further action is required by you since you are
already in sync with me.

Now for the changing RECURRENCE-ID model analysis:

When you receive the SEQUENCE:0 invitation you apply Section 2.1.5 and
3.2.2 and determine that this is a new meeting invitation which you take
some action on to get it onto your calendar.

You then receive SEQUENCE:2 and you can NOT match the UID / RECURRENCE-ID
values in your calendar so you determine that "something has gone wrong"
(to borrow from the misquoted Section 4.7.2).  You MUST therefore take NO
action on the REQUEST since you cannot match the instance to one you
currently have.  You can ONLY send a REFRESH message back to me with just
the UID and NO RECURRENCE-ID.  This will cause me to generate another
REQUEST to you "with the latest description and version of the event" (per
iTIP Section 3.2.6 REFRESH).  So I send you another REQUEST with the
latest RECURRENCE-ID at SEQUENCE:2 which happens to _exactly_ match the
REQUEST you just tossed out.  If you receive this new REQUEST _before_ you
get the missequenced SEQUENCE:1 REQUEST then you can _only_ repeat this
loop indefinitely without recovery!

In the interim the SEQUENCE:1 REQUEST arrives and you are able to apply
Section 2.1.5 rules and find the correct UID / RECURRENCE-ID instance and
determine that the REQUEST obsoletes the SEQUENCE:0 one you have.  Now you
can apply the currently obsolete SEQUENCE:1 change to your SEQUENCE:0
instance and send back a REPLY to me.  I would of course detect that the
UID / RECURRENCE-ID / SEQUENCE:1 REPLY is for an old copy (by applying
2.1.5) and thus I would have to ignore it and send you yet another REQUEST
at SEQUENCE:2.  Now that you are at SEQUENCE:1 you can safely match the
UID / RECURRENCE-ID values and re-respond at SEQUENCE:3.

This model has lots of drawbacks (hence why we long ago opt'd for a
"fixed" RECURRENCE-ID model).  They include:

1: Any missequencing of iTIP messages cannot be recovered on a single
instance case.  You MUST resync up ALL instances in the set, not just the
instance in question.  This means potentially LOTS of extra wasted
workflow messaging.  All just because of _1_ missquenced REQUEST (or REPLY
if you invert the picture for my side as Organizer).

2: If the problem is NOT a missequencing of REQUESTs but rather recovery
from a lost REQUEST (ie: an interim copy got lost in a server crash, disk
failure, admin purge, etc) then it is not possible to recover when just
one or some instances are involved since RECURRENCE-IDs are always
involved in the REQUEST/REPLY(/COUNTER/...) messages when you correctly
apply the restriction tables from iTIP.  REQUEST clearly says:

RECURRENCE-ID   0 or 1  only if referring to an instance of a
recurring calendar component.  Otherwise it
MUST NOT be present.

so any initial or subsequent REQUESTs MUST have a RECURRENCE-ID on them
when referring to a particular instance.  Had you not received the
SEQUENCE:1 REQUEST at some point you would be infinitely stuck in the
REFRESH/REQUEST loop above.

3: Since each instance can be reschedule independently of its siblings, it
can be shown that if the RECURRENCE-ID changes it takes just 5 steps to
misidentify the correct instance in question in a REPLY (on the Organizers
side).  For the steps, go check the archives for the 2 times its been
pointed out.

The same kind of fault _intolerant_ behaviour can be seen for other cases
in iTIP if you lay out the scenarios (and ignore iCalendar too).

Now lets visit that misquoted iTIP Examples Section 4.7.2 Bad
RECURRENCE-ID lest some claim we are ignoring their citations for
justifying a changing RECURRENCE-ID model.  iTIP Section 4.7.2 Bad
RECURRENCE-ID is intended as an example section on how to deal with
problems matching UID / RECURRENCE-ID / SEQUENCE.  It says "there are
three cases in which an instance cannot be found." and describes them as:

1.  The component with the referenced "UID" and "RECURRENCE-ID" has
been found but the "SEQUENCE" number in the calendar store does
not match that of the ITIP message.

2.  The component with the referenced "UID" has been found, the
"SEQUENCE" numbers match, but the "RECURRENCE-ID" cannot be
found.

3.  The "UID" and "SEQUENCE" numbers are found but the CUA does not
support recurrences.

Case 3 is not germane and will be ignored for this analysis.

Case 1 is dealt with in:

In case (1), two things can happen. If the "SEQUENCE" number of the
"Attendee's" instance is larger than that in the "Organizer's"
message then the "Attendee" is receiving an out-of-sequence message
and MUST ignore it.  If the "SEQUENCE" number of the "Attendee's"
instance is smaller, then the "Organizer" is sending out a newer
version of the component and the "Attendee's" version needs to be
updated. Since one or more updates have been missed, the "Attendee"
SHOULD send a "REFRESH" message to the "Organizer" to get an updated
version of the event.

Here SEQUENCE is described as "smaller" and "larger" rather than "highest"
and "lower" found in Section 2.1.5 but the implicit behaviour still
matches.  In order to match a UID / RECURRENCE-ID / SEQUENCE combo where
the UID and RECURRENCE-ID  values are "found" but the SEQUENCE value is
"smaller" or "larger" then it should be intuitive that UID and
RECURRENCE-ID do not change.  Otherwise they could not be found to
mismatch SEQUENCE!  So this description matches the behaviour under
Section 2.1.5.

The last ine about "one or more updates have been missed" has been misused
as justification for the recipient (ie: you in the above examples) having
to send a REFRESH to the Organizer (ie: me...).  However 2 things need to
be noted here:

A: The case is described as the one where UID / RECURRENCE-ID matched but
SEQUENCE did NOT and
B: The text says that the recipient "SHOULD" send a REFRESH, not "MUST".

Now if you apply normal logic to the analysis you should see that bullet A
above cannot be done in a changing RECURRENCE-ID model since the
RECURRENCE-ID value changed!   However this is incongruence needs to be
overlooked to use Section 4.7.2 as justifying a changing RECURRENCE-ID
model.

Also, bullet B does not say "MUST" because it is NOT necessary! The
larger/higher SEQUENCE version obsoletes all smaller/lower SEQUENCE
versions and as such a REFRESH is not really necessary (except in a
changing RECURRENCE-ID model).

Case 2 is dealt with in:

In case (2), something has gone wrong.  Both the "Organizer" and the
"Attendee" should have the same instances, but the "Attendee" does
not have the referenced instance.  In this case the "Attendee" SHOULD
send a "REFRESH" to the "Organizer" to get an updated version of the
event.

Aha, we now have problem with the fixed RECURRENCE-ID model or do we?
Actually we do not.  If you receive a REQUEST for an unknown UID /
RECURRENCE-ID pair then if you apply the text from Section 3.2.2 REQUEST
correctly you simply view the REQUEST as a new invitation.  In this case
to a new instance you did not know about before.  No problem here.  Simply
treat it as what it is, a REQUEST to another instance and workflow
functions as before.

Ah, we do have a problem though if we used a changing RECURRENCE-ID model.
The "something has gone wrong" is that the message was missequenced.
Gee, something as simple as that means there is a problem??  Thats the
sign of a poor model (and one of the reasons the WG opt'd for the fixed
RECURRENCE-ID model) because to recover means lots of extraneous iTIP
messages in both directions.

In addition it should be noted that not all Attendees are invited to all
instances so the 2nd line is a bit misleading.  If I want to add you to an
instance (or subset of instances) of a repeating meeting Im organzing then
obviously you wont have the "the same instances".  I suspect that this is
an artifact of editing because we do not expect / require that all
attendees be invited to all instances.

Also, this example also uses "SHOULD" instead of "MUST".  Had the
RECURRENCE-ID model been one of a changing RECURRENCE-ID value then the it
would have to say "MUST" in order to attempt to resync the invitee with
the Organzier.  Some folks have seized on this Case as justification for a
changing RECURRENCE-ID model.  However they have modified it in the
process by changing the "SHOULD" into a "MUST" and ignoring all of the
other sections of iTIP that contradict them.

Ok by now it should be clear that iCalendar and iTIP specify and use a
fixed RECURRENCE-ID model.  So the next thing to consider is the Working
Group history on this subject and looking at how its been implemented by
folks.

As previously noted in recent discussions the question of a fixed or
changing RECURRENCE-ID model was raised at least as far back as July 1999.
The concensus then was that RECURRENCE-ID was fixed (essentially for the
reasons demonstrated above).  I invite anyone who still thinks that
RECURRENCE-ID should changing on a reschedule to go reread the archives
and search for yourself.

Finally, lets take a look at how at least the RFC authors have implemented
RECURRENCE-ID in their various products.  Surely they would have
implemented it the way the RFC intended it, yes?!

If you take a look at Microsofts Outlook 98 (and higher) and Exchange you
will find that all implementations (done by different development teams)
all implement a fixed RECURRENCE-ID.  You need to be configured for
Internet Mode to get iCalendar support (or you can use the Exchange server
if you configure it for iCalendar).  In any case they have a fixed
RECURRENCE-ID model.

If you take a look at Lotus Organizer, Lotus Notes and the eSeries
offerings you will find that all of them (also done by different
development teams) implement a fixed RECURRENCE-ID.

I do not have current access to Evolution or to Netscapes offerings but in
looking over my notes from CalConnects I see that we did not have any
interoperability issues with anyone participating so they must have also
implemented fixed RECURRENCE-IDs.

Ok, to recap this (assuming you are still with me to here):

1: iCalendar clearly and unequivocally says that on a reschedule
RECURRENCE-ID does not change; it keeps the "original" value.
RECURRENCE-ID "might also change" only in the case "When the definition of
the recurrence _set_ for a calendar component changes" and not "When the
definition of the recurrence _instance_ for a calendar component changes"
2: iTIP prose specifying Message Sequencing and messages themselves do not
disagree with iCalendar.
3: iCalendar defines the RECURRENCE-ID property; iTIP defines its
semantics.  As such iTIP cannot redefine RECURRENCE-ID, that is not its
job/function.
4: The changing or delta RECURRENCE-ID model has been demonstrated as
being fault intolerant and flawed in dealing with sequencing and recovery
of lost messages.
5: The WG already discussed this long ago and confirmed that RECURRENCE-ID
did not change on a reschedule.
6: ALL the implementations from all the authors use a fixed RECURRENCE-ID.
So either they ALL misunderstood their own model or they properly
implemented it based on iCalendar and iTIP.

RECURRENCE-ID is akin to "the UID for a particular instance" and as such
it needs to have the similar behavior as the UID property.  Otherwise iTIP
workflow gets lots more complicated and verbose and so does a CUAs design.

Bruce
===========================================================================
Bruce Kahn                                INet: 
[email-masked]
Messaging & Collaboration                 Phone: 978.399.6496
IBM Software Group                         FAX: and nothing but the FAX...
Standard disclaimers apply, even where prohibited by law...

---

From: Satya Vempati <[email-masked]>
Date: Tue, 05 Aug 2003 10:16:54 -0700

<<It sounds like you only need a short note from the authors to this
effect and you'd then accept the fixed RECURRENCE-ID model.  Would that be
accurate? >>
 
That is correct. Then we can work out the details for ourselves.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, August 05, 2003 9:14 AM
To: Satya Vempati
Cc: [email-masked]
Subject: RE: RECURRENCE-ID discussion

> that seem to buttress theirs.

2 things that are being forgotten so Ill repeat 'em again: 

1: This exact citation was the one Dan Hickman asked about back in July
1999 and the WG discussed.  We agreed that it was in error.  Given that of
250+ pages of RFCs and you find 2 sentences that contradict the rest is
not an overwhelming contradiction.  We had 5+ authors working on different
parts of 3 RFCs so 2 lines out of 250+ is not that significant.  There are
no other conflicting citations that Ive seen so far and ALL the others
citations support each other. 

2: The lines you cite is part of iTIP.  iTIP provides the semantics for
the iCalendar defined properties.  iTIP cannot change the iCalendar
defintion of the property or its behaviour; thats NOT its function. 

> that the recurrence set changes whenever something is added to or deleted

A: What about the paragraph you forgot to cite?  The one that expressly
says on a reschedule of an instance the RECURRNECE-ID stays at its
original value?  Is that unclear or open to multiple interpretations? 

B: We have _always_ been discussing the case of rescheduling instances, at
least thats always been the scenario Ive used and focused on (and Ive
always tried to keep others from mixing the scenarios).  Changing the
definition of the recurrence set by doing an adding or removing instances
is exactly what that sentence was about.  However rescheduling an instance
is doing NEITHER!  And as you seem to agree, adding or removing/deleting 
instances clearly says that the RECURRENCE-ID value "might also change". 
For example, if I add a new non-conflicting instance to the existing set
then there is no need for my RECURRENCE-ID to change obviously.   

Do NOT fall into Dougs misinterpreation that rescheduling an instance is
the same as changing the repeat instance set defintion, its not. 

> Because of such inconsistencies 

1 already WG discussed and declared bad iTIP bit from at least 1999 is not
anything new or a major inconsistancy in the RFCs.  The other phrase you
mention is not an inconsistancy in my reading; the only problem is how
some can misread "the definition of the recurrence set" as being "the
definition of the recurrence instance" but thats not the RFCs fault nor is
it an inconsistancy.   

It sounds like you only need a short note from the authors to this effect
and you'd then accept the fixed RECURRENCE-ID model.  Would that be
accurate? 

>                                  , if we clearly understand the intent of

I always try to be as polite but sometimes frustration seeps in.  I have
showed: 

1: Multiple supporting citations in both iCalendar and iTIP that describe
and support a fixed RECURRENCE-ID model 
2: Why the fixed RECURRENCE-ID model works well and is quite fault
tolerant.   
3: That the RFCs overwhelmingly describe and implicitly use a fixed
RECURRENCE-ID model.   
4: Several of the major flaws inherent in a changing RECURRENCE-ID model. 

Yet some folks still persist in saying that 1-2 lines in iTIP justify that
iCalendar/iTIP uses a delta RECURRENCE-ID model.  Wouldnt you get a littel
frustrated too after several weeks of this? 

Bruce 
===========================================================================
Bruce Kahn                                INet: [email-masked]
Messaging & Collaboration                 Phone: 978.399.6496
IBM Software Group                         FAX: and nothing but the FAX...
Warning: Dates in Calendar are closer than they appear.

---

From: Satya Vempati <[email-masked]>
Date: Tue, 05 Aug 2003 10:26:13 -0700

accurate?>>

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Tuesday, August 05, 2003 9:14 AM
To: Satya Vempati
Cc: [email-masked]
Subject: RE: RECURRENCE-ID discussion

Bruce

---

From: Satya Vempati <[email-masked]>
Date: Sat, 09 Aug 2003 04:59:40 +0530

An excellent summary and detailed analysis. I just wish that the RFC was
written better:

Instead of: 

   For a given pair of "UID" and "SEQUENCE" property values,
   the "RECURRENCE-ID" value for a recurrence instance is fixed.

if it said:

   For a given pair of "UID" and "SEQUENCE" property values,
   the "RECURRENCE-ID" values for all recurrence instances are fixed.

we would not be arguing about it today.

The current text does sound like it is talking about one instance.

Along with Bruce, the growing sense is that the fixed recurrence model is
simpler and indeed does work.

Actually --again as Bruce has already pointed out-- the discussion of
whether "Recurrence-Id" changes goes back to July, 1999. Dan Hickman
brought up the (now infamous) iTIP section that states that RECURRENCE-ID
should change to the new DTSTART and asked:

--------------- 
<SNIP>

The paragraph below is shown at the very end of section 3.7.1 in the
rfc2446
(iTIP).  The last two sentences seem to conflict in meaning.  The first
says
that "the original DTSTART value is used for RECURRENCE-ID" but the second
says that "RECURRENCE-ID has changed to the new DTSTART".

I thought we decided to leave the RECURRENCE-ID the same.  If the last
sentence is true, then it seems we would have to exclude the original date
in an "EXDATE" property and then include the new date in an "RDATE"
property.  Is this right?

</SNIP>

To which Frank Dawson replied:

----------------
<SNIP>

Dan: 
Good to hear from you. Hope your iCalendar activities are going great! 

The RECURRENCE-ID (along with the UID and SEQUENCE properties) are used to
reference (a particular version of) an individual occurrence for a
repeating event or to-do. As such, it will change after the DTSTART for
the occurrence changes. 

>The paragraph below is shown at the very end of section 3.7.1 in the
rfc2446
>(iTIP).  The last two sentences seem to conflict in meaning.  The first
says
>that "the original DTSTART value is used for RECURRENCE-ID" but the second
>says that "RECURRENCE-ID has changed to the new DTSTART". 

For example, if you scheduled a Monday, Wednesday and Friday meeting and
you want to reschedule the Wednesday meeting meeting to Thursday, then the
RECURRENCE-ID in the reschedule would specify the date/time of the
Wednesday meeting as specified in the original REQUEST. Now, after
rescheduling to Thursday, if you want to reference that rescheduled
occurrence you specify a RECURRENCE-ID that would have the date/time of
the Thursday meeting. 

>I thought we decided to leave the RECURRENCE-ID the same.  If the last
>sentence is true, then it seems we would have to exclude the original date
>in an "EXDATE" property and then include the new date in an "RDATE"
>property.  Is this right? 

Not sure I agree with the last part. You can currently specify a "EXDATE"
property in the original REQUEST that has the date/time value of a
particular occurrence. You could also specify a duplicate date/time value
in a RDATE that is the same as an occurrence specified in a RRULE or other
RDATE. However, the set of occurrences will only have one element with a
given date/time interval. 

You can also reschedule by specifying the revised DTSTART/DTEND for the
occurrence. However, if you use EXDATE/RDATE wouldn't that appear to be
"ADDing" an occurrence? We use the ADD method to do this. It seems to be
problematic if you try to use the EXDATE/RDATE to specify the date/time
for the rescheduled meeting. It would seem that we should be using the
DTSTART/DTEND properties for that. 

-- Frank 

</SNIP>

To summarize, the iTIP issue was discussed with the original authors but
it was deliberately left alone. It is not a miss. But it looks more and
more like an error.

-----Original Message-----
From: Michael Fair [mailto:[email-masked]]
Sent: Friday, August 08, 2003 4:22 PM
To: [email-masked]
Subject: Re: RECURRENCE-ID discussion

"Doug Royer" <[email-masked]> wrote in message
news:[email-masked]...
> > So now we go to update it again.  If we look at the objects
> > themselves.  I specified that SEQUENCE: 1 of UID: 1,
> > RECURRENCE-ID: ...T180000Z would DTSTART at ...T150000Z.
> > Note that no where in this object did I ever say, "oh and
> > by the way here's a new RECURRENCE-ID".  So now we go
> > to move it again.  Here's where the disagreement is.
>
> You do not need to say 'oh and by the way here's a new RECURRENCE-ID'
> because iCAL says:
>
>     Purpose: This property is used in conjunction with the "UID" and
>     "SEQUENCE" property to identify a specific instance of a recurring
>     "VEVENT", "VTODO" or "VJOURNAL" calendar component. The property
>     value is the effective value of the "DTSTART" property of the
>     recurrence instance.
>
> So once the SEQUENCE:1 object is booked, then the RECURRENCE-ID
> for that single instance object SEQUENCE:1 is the effective value
> of the "DTSTART" property - ...T150000Z and not ..T180000Z.
>
> So after that point if you do an update to RECURRENEC-ID ..T180000Z,
> there would be no object with that effective DTSTART date and
> thus no RECURRENCE-ID to match.

Indeed, I grant you that the use of the term "effective" does
tend to call for the use of a changing RECURRENCE-ID.  But it
doesn't, and I'll show you why a little later.

So further down in the IMAP definition, we find this text:

   For a given pair of "UID" and "SEQUENCE" property values,
   the "RECURRENCE-ID" value for a recurrence instance is fixed.

So here's how I read this:
There is a UID which describes a recurrence set.  That UID has
a SEQUNCE value which identifies the set it describes.  This
reading is supported by the part that describes how the SEQUNCE
changing reflects a set description change which in turn could
modify the RECURRENCE-ID of the set instances.

So for examples sake, let's again use UID: 1
and say that SEQUENCE: 0 describes every Friday this year.

So for UID 1/SEQUENCE 0 the RECURRENCE-ID for each instance
is "fixed" at the Fridays of this year.  If I move one of
those instances (say June, Friday the 13th) to Thursday.
Then I update that particular instance's SEQUENCE to 1.
This is only an update to the SEQUENCE for that specific
instance's calendar component not for the parent UID 1
set descriptor.  The UID/SEQUENCE pair has not been updated.
The (UID-RECURRENCE-ID)/SEQUENCE pair (or triplet depending
on how you want to describe it) has been updated and has
no relevance to whethere or not the RECURRENCE-ID is fixed.

This view is supported by order of precedence for locating
an instance, first the UID, then the RECURRENCE-ID are primary
lookup keys, the RECURRENCE-ID can be NULL which means
different things in different contexts, and then the SEQUENCE
is used as a secondary lookup.

In other words, by moving an instance to Thursday I have
not modified the description of the recurrence set specified
by UID 1/SEQUENCE 0 and therefore the RECURRENCE-ID remains
fixed.  Consequently, for the pair UID 1/SEQUENCE 0, the
RECURRENCE-ID for June, Friday the 13th remains fixed
regardless of what I do that particular instance of UID 1
in the context of SEQUNCE 0.  In essence, I've simply said
that June, Friday the 13th's instance actually occurs on
Thursday now.

So let's go back to that other quote that seemed to
imply a changing RECURRENCE-ID.

>     The property value is the effective value of the
>     "DTSTART" property of the recurrence instance.

The "effective value" here is referencing the UID/SEQUENCE
pair, not the effective value of an instance that's been
updated.  This must be the case because
"for a given UID/SEQUENCE pair the RECURRENCE-ID value
for a recurrence instance is fixed".

However, if you update the recurrence set description,
and consequently update SEQUENCE for that UID, then
you must recalculate all the instances in the set and
they may have a new DTSTART.  To account for the
ability for the RECURRENCE-ID to change in this way
you must use "the effective value of the DTSTART
propoerty of the recurrence instance".

The only model under which all the paragraphs make
sense is the one where the RECURRENCE-ID is fixed
within a given set description of an event.  The
simplest way to express this in terms programmers
can easily adhere to is that for a given UID/SEQUENCE
pair (meaning a particular recurring set description)
the RECURRENCE-ID value for a recurrence instance
is fixed.

Once UID 1/SEQUENCE 0 hits UID 1/SEQUENCE 1 (which would
mean something significant about the set of instances
has changed) all bets are off in regards to RECURRENCE-ID
values that previously existed.

> In Bruces model you can NEVER add someone to the series
> of recurring VEVENT after any change that effects the
> effective  DTSTART value of any instance as they could never
> be able to resolve the RECURRENCE-ID because they never
> had SEQUENCE:0.

This is just plain false.  Given my "every Friday for this
year" example above, with the June 13th modification to
June 12th, to invite someone to the entire series I would
send them a
VEVENT REQUEST containing:
UID: 1
SEQUENCE: 0
describing every Friday this year
with them as an ATTENDEE

and simoultaneously also send them a
VEVENT REQUEST containing:
UID: 1
RECURRENCE-ID: Friday, June 13th
SEQUENCE: 1
describing Thrusday, June 12th
with them as an ATTENDEE

In "Bruce's model" I could even send those two objects
out of order and you'd still be able to accurately process it
which debunks your claim that they couldn't resolve the
RECURRENCE-ID because they never had SEQUENCE: 0.

I don't mean to be trolling here, but isn't it in fact
the changing RECURRENCE-ID model that has the problem
with dealing effectively with lost / missing messages?

And lastly to cover the great example you offered:

> For a single VEVENT REQUEST with three instance (COUNT=3
> in a RRULE). It has three RECURRENCE-ID's none of which
> were ever explicitly created.
>
> So if the original REQUEST looks like this:
>
> METHOD:REQUEST
> SEQUENCE:0
> DTSTART:20030805T180000Z
> RRULE:FREQ=HOURLY;COUNT=3
> LOCATION:A
>
> Would have 3 RECURRENCE-ID's:
>
> ...T180000Z  1st instance (effective DTSTART)
> ...T190000Z  2nd instance (effective DTSTART)
> ...T200000Z  3rd instance (effective DTSTART)

So good so far.

> So now the ORGANIZER wants to change the 2nd instance to ..T193000Z.
> So per iTIP they could send a new REQUEST SEQUENCE:1 :
>
> METHOD:REQUEST
> SEQUENCE:1
> DTSTART:..T193000Z
> RECURRNCE-ID:..T190000Z

Also so good so far.

> So once the CUA accepts the new SEQUENCE:1 object, the object
> still has three RECURRENCE-ID's:
>
> ...T180000Z  1st instance (effective DTSTART)
> ...T193000Z  2nd instance (effective DTSTART)
> ...T200000Z  3rd instance (effective DTSTART)
>
> As the RECURRENCE-ID is the effective DTSTART value
> of the instance.

Oooops just violated:

   For a given pair of "UID" and "SEQUENCE" property values,
   the "RECURRENCE-ID" value for a recurrence instance is fixed.

Since you didn't provide a UID in your examples (how strange)
let's go ahead and use UID: 1

First you defined UID 1 with three events, then you updated
one of those instances.  This was fine, but when you listed
the new RECURRENCE-ID's you seemed to miss that you haven't
redefined the recurrence set of UID 1, just updated the DTSTART
of an instance whose effective DTSTART is still defined by
UID 1/SEQUENCE 0.

UID 1 and UID 1/RECURRENCE-ID ...T190000Z are two totally
separate calendar components which each track there own
SEQUENCE values.

> Now change the 3rd instance of the SEQUENCE:1 object to ...T190000Z
> and move its LOCATION to 'B'.
>
> METHOD:REQUEST
> SEQUENCE:2
> DTSTART:..T200000Z
> RECURRNCE-ID:..T190000Z
> LOCATION:B

Woaaahhh you have two problems here.

First the 3rd instance is actually RECURRENCE-ID: ...T200000Z
which you were trying to DTSTART to ...T190000Z and move to
location B so you're example message wrong, but I can work
with what you intended.

The second problem is that you just jumped two SEQUENCE values
when you've only made one change to this object.  You haven't
touched the 3rd instance prior to this point, and the update
above to the second instance did not update UID: 1, it updated
UID: 1/RECURRENCE-ID: ...T190000Z so this is actually a
SEQUENCE: 1 move for UID: 1/RECURRENCE-ID: ...T200000Z.

Remember that:
UID: 1 and
UID: 1/RECURRENCE-ID: ...T180000Z and
UID: 1/RECURRENCE-ID: ...T190000Z and
UID: 1/RECURRENCE-ID: ...T200000Z

are totally separate objects which persist their own SEQUENCE.

> So once the CUA accepts the new SEQUENCE:2 object, the object
> still has three RECURRENCE-ID's (new effective DTSTART times):
>
> ...T180000Z  1st instance (effective DTSTART)
> ...T190000Z  2nd instance (effective DTSTART)
>                       (used to be the 3rd instance)
> ...T193000Z  3rd instance (effective DTSTART)
>      (used to be the 2nd instance)

This is actually totally screwed up at this point with
exception of the first instance.

There are four components involved in this event at this
point and they are described as follows:

UID: 1 - describes three instances starting an hour apart
     at Location: A
UID: 1/RECURRENCE-ID: ...T180000Z - hasn't been touched
     at Location: A
UID: 1/RECURRENCE-ID: ...T190000Z - DTSTART: ...T193000Z
     at Location: A
UID: 1/RECURRENCE-ID: ...T200000Z - DTSTART: ...T190000Z
     at Location: B

This is important to note for you to see the flaw in your
later assertion that "in Bruce's model, you're broken".

> Now change the SEQUENCE:2 2nd instance to ...T200000Z:

This actually needs to read:
"Now change the SEQUENCE:1 version of the 2nd instance
 to ...T200000Z"

This will actually be a SEQUENCE: 2 update for the 2nd instance.

> METHOD:REQUEST
> SEQUENCE:3
> DTSTART:..T190000Z
> RECURRNCE-ID:..T200000Z

First off you totally screwed the pooch on mixing up the
RECURRENCE-ID and DTSTART again.  Secondly you're just being
sloppy or you're getting models confused because up to now
you seem to have been using a changing RECURRENCE-ID model
which would make the ID for the 2nd instance ...T193000Z.

> Now in Bruce's model, your broken because you just moved
> the SEQUENCE:0, LOCATION:A meeting to ...T200000Z to LOCATION:A.
> When in fact the move was to move the LOCAION:B meeting.
>
> Bruce's reply is to throw the object away and start over
> because its impossible to resolve.

Again, this is a totally bogus assertion.
The above message should actually read:

METHOD:REQUEST
SEQUENCE:2
DTSTART:..T200000Z
RECURRNCE-ID:..T190000Z

I don't quite understand what you are asserting is broken.
If you first create three events at Location: A, you then
move the time on the second, and then the time and the
location on the third, and then just the time again on
the second.  You end up with the first two events at
location A, one starting at 1800Z, the other at 2000Z,
and the thrid event at location B starting at 1900Z.
What's broken about that?

This would leave us with the following four objects:
[Note:  The first instance may or may not actually
be represented in the calendar store since it was
never actually modified from its original description]

UID: 1 - describes three instances starting an hour apart
     at Location: A - SEQUENCE: 0

UID: 1/RECURRENCE-ID: ...T180000Z - hasn't been touched
     at Location: A - SEQUENCE: 0

UID: 1/RECURRENCE-ID: ...T190000Z - DTSTART: ...T200000Z
     at Location: A - SEQUENCE: 2

UID: 1/RECURRENCE-ID: ...T200000Z - DTSTART: ...T190000Z
     at Location: B - SEQUENCE: 1

>  > Now I think this means that Doug is trying to say that
>  > the RECURRENCE-ID is set to the original value of the instance
>  > being modified which with a changing RECURRENCE-ID could be
>  > different.  But I find it impossible for Doug to support
>  > this claim because as far as I can see there is no where in
>  > the text that allows one to set a before and after RECURRENCE-ID.
>  > So the only way to get a differing RECURRENCE-ID is for CS/CUA
>  > to proactively change it on its own accord which the message
>  > never specifies to do.
>
> You do not change RECURRENCE-ID's directly. You update the object
> in ways that alter the effective DTSTART value for an instance.

The "effective DTSTART" is per the UID / SEQUENCE set description
pair, not the UID/RECURRENCE-ID / SEQUENCE pair.  Otherwise you
conflict with:

   For a given pair of "UID" and "SEQUENCE" property values,
   the "RECURRENCE-ID" value for a recurrence instance is fixed.

> And as its effective DTSTART value is changed, thus its RECRRENCE-ID
> changes to match its effective DTSTART value. And:
>
>     (iTIP)
>     An instance of a recurring event is assigned a unique identification,
>     "RECURRENCE-ID" property, when that instance is renegotiated.
>     Negotiation may be necessary when a substantive change to the event
>     or to-do has be made (such as changing the start time, end time, due
>     date or location). The "Organizer" can identify a specific recurrence
>     instance using the "RECURRENCE-ID" property. The property value is
>     equal to the date/time of the instance. If the "Organizer" wishes to
> You do not 'set' the 'before' and 'after' RECURRENCE-ID, you
> alter the effective DTSTART directly by updating one or more
> instances and the RECURRENCE-ID is defined to match the
> new effective DSTART.

I hold that the "Note" mentioned above which is a main crux for
the changing RECURRENCE-ID understanding is an actual RFC error
and was not the intent of the authors.  That text appears in one
place, however in numerous places throughout the RFC iTip goes
to great pains on several occasions to talk about how iTip must
be tolerant of missed/lost messages.  In the changing RECURRENCE-ID
model this idiom is severely violated since it is not tolerant
at all to missequenced messages and especially volatile in the
presence of lost messages.

Further, the "effective DTSTART", is the DTSTART of the instance
as described by the recurrence set in the event object and not
the current DTSTART for any given instance.

Ok, now this next part we all totally agree on!!!!

> Another way to change the RECURRENCE-ID's is to alter the times
> an existing VEVENT. So if the SEQUENCE:0 object were:
>
> METHOD:REQUEST
> SEQUENCE:0
> DTSTART:20030805T180000Z
> RRULE:FREQ=HOURLY;COUNT=3
> LOCATION:A
>
> Would have 3 RECURRENCE-ID's:
>
> ...T180000Z  1st instance (effective DTSTART)
> ...T190000Z  2nd instance (effective DTSTART)
> ...T200000Z  3rd instance (effective DTSTART)
>
> If the ORGANIZER sent a SEQUENCE:1 :
>
> METHOD:REQUEST
> SEQUENCE:1
> DTSTART:20030805T180000Z (same as the SEQUENCE:0 value)
> RRULE:FREQ=HOURLY;COUNT=3;INTERVAL=2
> LOCATION:A
>
> The SEQUENCE:1 object would 3 RECURRENCE-ID's 2 of which
> are different than the SEQUENEC:0 object:
>
> ...T180000Z  1st instance (effective DTSTART)
> ...T200000Z  2nd instance (effective DTSTART)
> ...T220000Z  3rd instance (effective DTSTART)
>
> The ORGANIZER moved two instances and their effective
> DTSTART values (RECURRENCE-ID).

Absosmurfly! Without question! and with violent agreement, Yes!

Updating to SEQUENCE: 1 on the base event object, by for
instance doing a recurrence set description update like
the above, in every way, shape, and form throws all
RECURRENCE-IDs from the SEQUENCE: 0 version into doubt.

I am in total agreement, as is Bruce from what I've read,
that this kind of change, one in which no RECURRENCE-ID
is referenced to limit the scope to a single instance
but instead operates on the whole set of instances, totally
presents the possibility that the RECURRENCE-ID of the
instances will change.  In fact, the new RECURRENCE-ID
will be the bewly updated "effective DTSTART" value.

This new set of RECURRENCE-IDs will remain "fixed" for
as long as the event set descriptor maintains its same
SEQUENCE value.  Should another change update the set
descriptor object to SEQUENCE: 2, the entire set of
RECURRENCE-ID MUST be recalculated to their new effective
DTSTART values.

It seems to me that main crux to the differences here is
a different understanding in the handling of the SEQUENCE
values in relationship to the several calendar components
that exist to serve a single recurring event.

I'm too tired to look it up now, but the section on what
objects must persist under what circumstances should
clear up that issue.  So at the moment, given lack of
text to back me up, I can only contend that the RFCs
specify that for each recurring event there are two types
of calendar components.  One type is the event descriptor
itself (UID with no RECURRENCE-ID) and the other type is
a recurrence instance within that set (UID and RECURRENCE-ID)
and that every one of these objects tracks its own
independent SEQUENCE value.

-- Michael --