Subject: Re: Correct handling of Recurrence-id

From: [email-masked]
Date: Fri, 23 May 2003 18:29:50 -0400

Arnaud Quillaud wrote on 05/23/2003 02:49:21 PM:
> But what is the advantage of the opposite interpretation of the spec
> (changing RECURRENCE-ID over time) ? Or what are the disadvantages 
> of fixed RECURRENCE-ID ? 

I thought I covered them already earlier today but Ill try to resummarize 
the probelms with the 'delta' model:

1: We cannot guarantee that every previous message has arrived at the 
recipients address.  As such, if any message _ever_ is lost then the 
entire workflow breaks down.  Also, data can arrive delayed and out of 
order making it impossible for workflow to be performed accurately (I 
often get Dougs replys to my notes waayy before I get my own postings via 
the IMC list server which gives my threading agents fits!) 

2: We cannot prevent users from deleting 'old' messages nor should we 
require that _all_ previous messages be preserved in case we need to 
perform some new scheduling action. 

3: There is no reliable way for a CUA to tell when the 'last' message in 
the delta chain has arrived and that workflow can safely be performed. 
There is no magic indicator property nor can one be safely added. 
Otherwise an UPDATE 3 (for above) would also have it and thus any CUA 
would mistake UPDATE 2 as the last one rather than UPDATE 3.  Its a 
chicken and egg type of problem recognizing "the last" or actual message 
to take workflow actions on. 

4: The WG burned lots of gray cells recognizing the fact that the only way 
to keep workflow functioning was to make each message a full and complete 
snapshot for that entry/instance.   

If you search the WG archives you will find more detailed discussions on 
delta vs full snapshots (or whatever we called it then) and why a delta 
design is unreliable.

>                          I think we now all understand the proposed 
> model. It would be great to now see why it might be the best approach.

Actually both models are just interpretations of the text in 2445, Section 
4.8.4.4 Recurrence ID:

   The date/time value is set to the time when the original recurrence
   instance would occur; meaning that if the intent is to change a
   Friday meeting to Thursday, the date/time is still set to the
   original Friday meeting.

   The "RECURRENCE-ID" property is used in conjunction with the "UID"
   and "SEQUENCE" property to identify a particular instance of a
   recurring event, to-do or journal. For a given pair of "UID" and
   "SEQUENCE" property values, the "RECURRENCE-ID" value for a
   recurrence instance is fixed. When the definition of the recurrence
   set for a calendar component changes, and hence the "SEQUENCE"
   property value changes, the "RECURRENCE-ID" for a given recurrence
   instance might also change.

The former paragraph is the justification for the "fixed and never 
changing RECURRENCE-ID" model .  The latter paragraph is the justification 
for the "it changes based on the 'booked' value of DTSTART" model.

As I said before, in the past we have found residual text in the spec that 
should have been removed when some change was made but it was not.  I 
strongly suspect that this another case of this.

> version 2: The RECURRENCE-ID is equal to the DTSTART of the instance
> at a given time. When sending an iTIP REQUEST that  corresponds to a
> change of DTSTART for a particular instance, the RECURRENCE-ID is 
> equal to the previous value of the DTSTART. In any subsequent 
> REQUEST, the RECURRENCE-ID is equal to the new DTSTART (that is, 
> until the DTSTART is changed again). Hence, the RECURRENCE-ID 
> identifying an instance might change over time.

No 'might' about it.  If the DTSTART changes then so would the 
RECURRENCE-ID.  Therefore unless DTSTART never changes (ie: the entry only 
gets longer/shorter) the RECURRENCE-ID will always change by this version.

> The only disadvantage of version 1 that I can think of is about 
> someone joining the party while some rescheduling has already 
> occured. Since the newcommer won't have the SEQUENCE: 0 of the 
> recurrence set, he might interpret the set of RECURRENCE-ID 
> differently. But maybe this is just because I don't understand what 
> the right iTIP workflow is.

Its not a problem for the non-delta model.  The Organizer simply sends a 
new invitee a REQUEST that contains the information they will need to 
correctly identify the instance and put it on the right date/time.  So, 
continuing the example I used earlier to add Arnaud to Friday 13-Jun-03 I 
would simply send the following REQUEST:

   BEGIN:VCALENDAR
   PRODID:-//ACME/DesktopCalendar//EN
   METHOD:REQUEST
   VERSION:2.0
   BEGIN:VEVENT
   ORGANIZER:Mailto:[email-masked]
   ATTENDEE;ROLE=CHAIR;PARTSTAT=ACCEPTED:Mailto:[email-masked]
   ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION;TYPE=INDIVIDUAL;CN=Doug:
Mailto:[email-masked]
   ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION;TYPE=INDIVIDUAL;CN=George:
Mailto:[email-masked]
   ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION;TYPE=INDIVIDUAL;CN=Tom:
Mailto:[email-masked]
   ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION;TYPE=INDIVIDUAL;CN=Ki:
Mailto:[email-masked]
   ATTENDEE;RSVP=TRUE;PARTSTAT=NEEDS-ACTION;TYPE=INDIVIDUAL;CN=Arnaud:
Mailto:[email-masked]
   DTSTART:20030613T140000Z
   DTEND:20030613T150000Z
   SUMMARY:Head bashing
   UID:[email-masked]
   SEQUENCE:3
   RECURRENCE-ID:20030609T140000Z
   DTSTAMP:20030523T225900Z
   END:VEVENT
   END:VCALENDAR

If the RECURRENCE-ID never changes then it matches the SEQUENCE:0 value. 
So, Arnaud would use the rules under 2446, Section 2.1.5 Message 
Sequencing to see that he has a new REQUEST from me he has never seen 
before and thus treat it like a new invitation.  He clicks the Accept 
button on his UI and his CUA sends me back the properly crafted REPLY. 
Since he has the RECURRENCE-ID that everyone else shares (and that never 
changes) any future reschedules to him will be just that, reschedules and 
not a new invitation. 

All of this is spelled out in iTIP, Section 3.2.2 REQUEST:

   The "UID" and "SEQUENCE" properties are used to distinguish the
   various uses of the "REQUEST" method. If the "UID" property value in
   the "REQUEST" is not found on the recipient's calendar, then the
   "REQUEST" is for a new "VEVENT" calendar component. If the "UID"
   property value is found on the recipient's calendar, then the
   "REQUEST" is for a rescheduling, an update, or a reconfirm of the
   "VEVENT" calendar component.

BTW: here UID is not meant to be literally just UID but UID/RECURRENCE-ID 
(as described under Message Sequencing in iTIP).  It was done this way for 
brevity (ask Frank/Steve if you doubt me).

The same cannot be said of the version 2 ("delta") model.  The only way 
for me to add Arnaud would be to send him the initial snapshot and then 
ALL of the reschedules up to the current point.  Otherwise he could not 
possibly share the same RECURRENCE-ID that I do and thus not be able to do 
proper workflow.  There of course also is the problem of getting the chain 
of changes to Arnaud intact.

Given the unambiguious description from 3.2.2 above the version 2 REQUESTs 
would result in an initial Friday 13-Jun-03 entry being added to his 
calendar.  However if I reschedule again, new RECURRENCE-IDs are created. 
By the iTIP text above Arnaud would be forced to treat any new SEQUENCE:4 
/ RECURRENCE-ID:20030610T140000Z as a totally new invitation; NOT AS A 
RESCHEDULE OF THE FRIDAY INSTANCE HE ALREADY HAS!.  This is a very strong 
reason why deltas are bad!  (I need to try decaf I think...)

Bruce
===========================================================================
Bruce Kahn                                INet: 
[email-masked]
Messaging & Collaboration                 Phone: 978.399.6496
IBM Software Group                         FAX: and nothing but the FAX...
Standard disclaimers apply, even where prohibited by law...

---

From: Doug Royer <[email-masked]>
Date: Fri, 23 May 2003 17:21:12 -0600

[email-masked] wrote:
> I thought I covered them already earlier today but Ill try to 
> resummarize the probelms with the 'delta' model:
> 
> entire workflow breaks down. 
 > ..........

All of this has been solved.

Just compare the  UID, SEQUENCE, Possibly RECURRENCE-ID and DTSTAMP
per iTIP and iCAL. If you have an old copy - do a REFRESH.

Done.

-- 

  Doug Royer                     |   http://INET-Consulting.com
  -------------------------------|-----------------------------
  [email-masked]                 | Office: (208)612-INET
  http://Royer.com/People/Doug   |    Fax: (866)594-8574
                                 |   Cell: (208)520-4044

                 We Do Standards - You Need Standards

---

From: [email-masked]
Date: Tue, 27 May 2003 13:38:43 -0400

> per iTIP and iCAL. If you have an old copy - do a REFRESH.

No "possibly" about it for the repeating case which we are talking about 
now.  iTIP is not conflicting on the regarding identifying instances and 
sequencing (Section 2.1.5 Message Sequencing):

   1.  The primary key for referencing a particular iCalendar component
       is the "UID" property value. To reference an instance of a
       recurring component, the primary key is composed of the "UID" and
       the "RECURRENCE-ID" properties.

   2.  The secondary key for referencing a component is the "SEQUENCE"
       property value. 

So for an instance of a repeating set the primary key to use is always 
UID/RECURRENCE-ID followed by a secondary key of SEQUENCE followed by 
DTSTAMP!

The text that Doug/George have cited about rewriting the RECURRENCE-ID 
means that the _primary key_ would be changed for every reschedule to a 
particular instance.  If the key changes then you cannot find the 
particular instance in question; you find nothing OR you find a different 
instance!  This BREAKS WORKFLOW no matter what they claim!  (Ill show you 
below in case you are not convinced.)

The only way to always match the proper instance every time no matter how 
many reschedules have been involved/missed/etc is to make sure that the 
key never changes!  Nothing Doug says can change that.

I think that those that espouse a changing RECURRENCE-ID value have not 
looked at iTIP and how it distinguishes a new invitation from a reschedule 
(or update).  Because the WG decided to simplify the list of METHODs in 
the base protocol from the original ~15+ down to the current set we had to 
overloade some METHOD values.  To help distinguish between the possible 
cases of new vs reschedule vs update we put the sections in iTIP such as:

3.2.2.1 Rescheduling an Event

   The "REQUEST" method may be used to reschedule an event. A
   rescheduled event involves a change to the existing event in terms of
   its time or recurrence intervals and possibly the location or
   description. If the recipient CUA of a "REQUEST" method finds that
   the "UID" property value already exists on the calendar, but that the
   "SEQUENCE" (or "DTSTAMP") property value in the "REQUEST" method is
   greater than the value for the existing event, then the "REQUEST"
   method describes a rescheduling of the event.

So lets take a look at how Toms CUA would respond to my and Dougs 
reschedule notices. 

Note: for brevity Im leaving off the time bits to keep the examples but 
they should be included for real.  Now, lets start with my version...

Since the RECURRENCE-ID never changes I would send Tom a new REQUEST with:

       DTSTART:20030613 
       DTEND:20030613 
       RECURRENCE-ID:20030609
       SEQUENCE:3

for that first instance.  Since Tom already has that RECURRENCE-ID then he 
would follow the behaviour from Section 3.2.2.1 and treat it as a 
reschedule notice for the correct instance (the one originally on M 
09-Jun-03).  Thus he knows that there is no a meeting on Monday 09-Jun-03 
and he can accept/decline properly!

Lets now take a look at the proposal to use a new RECURRENCE-ID per 
reschedule.  Following what Doug has described so far I would have:

       DTSTART:20030613 
       DTEND:20030613 
       RECURRENCE-ID:20030612
       SEQUENCE:3

since the previous DTSTART value was for R 12-Jun-03 (at SEQUENCE:2).  If 
I recieve a REPLY from Tom for RECURRENCE-ID:20030609 at SEQUENCE:0 then I 
have a quandry.  First off, I have to be able to match that RECURRENCE-ID 
to an instance of the repeat set but none will match.  Unless my CUA 
maintains a full history of RECURRENCE-ID and SEQUENCE values for every 
revision of every instance.  Ok, perhaps (yuck but perhaps)...

So then I follow my internal reschedule 'chains' to determine that Tom was 
replying to the instance that is currently RECURRENCE-ID:20030613 and I 
would thus send him a new REQUEST with:

       DTSTART:20030613 
       DTEND:20030613 
       RECURRENCE-ID:20030612
       SEQUENCE:3

for the first instance.  Since Tom cannot find that RECURRENCE-ID then he 
would follow the behaviour from Section 3.2.2.1 and treat it as a new 
request since he does not have that RECURRENCE-IE for the given UID.  So 
the 'update' I sent would be treated as a new invitation to a new 
instance!  So now Tom thinks that he has been invited to an instance on 
Monday 09-Jun-03 (which he already accepted) AND he was just invited to a 
new instance on Friday 13-Jun-03 and he has to decide to attende or not.

This means that Tom thinks there is a meeting on Monday (there isn't) and 
that there is another related instance on Friday.  In fact its not 
'another' instance but the one he originally accepted.  Workflow could be 
partly restored in that if I never reschedule the Friday 13-Jun-03 
instance again Tom will show up to it.  However he will also show up on 
Monday where there is no meeting and no matter how many REFRESHes Tom 
sends, he will NEVER get any new information about it because my side 
thinks the latest snapshot is Friday and thus Id reREQUEST him for Friday. 
 This breaks the workflow process.

As more reschedules happen the more 'orphaned' instances get left on the 
invitees calendars because the primary key used to find them has been 
changed.  This is not good and why we opt'd away from a 'delta' model so 
long ago.

It can become worse if instances in a repeat set 'cross' at some point 
such that the rekeyd RECURRENCE-ID is shared between different instances 
at different points in time.  The SEQUENCE values could match when the 
overlap happens so then it becomes impossible to accurately tell which 
instance is being referred to at any given time.  To see this, simply do 
the following 5 steps:

1: Create a 2 entry repeating set for every Wednesday @ a given time.
2: Reschedule the first instance to the Friday between the two Wednesdays 
(now SEQUENCE:1)
3: Reschedule it again to the day before (Thursday, SEQUENCE:2)
4: Reschedule the second instance to the Friday between the two Wednesdays 
(now SEQUENCE:1 and same RECURRENCE-ID)
5: Reschedule it again to another day.

So now if a REPLY for RECURRENCE-ID:THURSDAY / SEQUENCE:1 comes in, can 
you tell me which instance it was for AND more to the point what the 
correct new RECURRENCE-ID is??  Didn't think you could, at least not 
accurately...

Ok, by now I hope that everyone can see that changing the primary instance 
key of UID/RECURRENCE-ID can totally munge workflow and create a mess 
whereas keeping a fixed primary instance key results in a much much 
simpler process and NO confusion...  Any not see this?

Bruce