Subject: Fwd: Re: Synchronization [Was: Re: CAP: Last Call By March IETF: Need Volunteers!]

From: Mark Paterson <[email-masked]>
Date: Fri, 18 Jan 2002 09:08:34 -0500

For some  reason this didn't seem to ever show up on the list. Take
II.

Date: Thu, 17 Jan 2002 13:04:03
-0500

To: "[email-masked]" <[email-masked]>

From: Mark Paterson <[email-masked]>

Subject: Re: Synchronization [Was: Re: CAP: Last Call By March IETF:
Need  Volunteers!]

At 02:11 PM 1/16/2002 -0500, George Babics wrote:

Doug Royer wrote:

> > > > 5) Synchronization:

> > > > -------------------

> > > >   Investigate if there is enough in CAP to
allow for synchronization?

> > > >  I believe this is in the requirements
doc.

> > > It is. I think it does meet the synchronization
requirements.

> > > And we limited synchronization to a bare minimum. That
is

> > > that you must be able to upload and download an entire
calendar

> > > and what ever it takes to make two calendars look the
same.

> >   Have we gone through the exercise of making sure
that we

> > support synchronization? If we limit support to a bare
minimum

> > can it hurt performance of synchronization?

> >   A CUA is able to query on the last modified
LAST-MODIFIED

> > property to find all the components that have changed the
last

> > time a sync was done. Using the METHOD:DELETE it can find
all

> > components that have been deleted. Is this sufficient?

> >   Should we add a small section or sub-section on
CAP

> > describing how synchronization can be done?

> I think that we had agreed that this was an implementation

> specific thing for now. At the very least we specified that

> a CUA must be able to upload an entire calendar and do the

> diff work it self.

> No one has opposed a synchronization specification. I say we

> need one - AS A SEPARATE DRAFT. For now a CUA can get it done

> by uploading the entire calendar - or by guessing from any

> VQUERYs it can make.

However, if there is something that we can easily add to

CAP that can help synchronization, we may want to add it. On the

other hand, what I described earlier, on how a CUA can have

basic synchronization, by using query on last modified time

and METHOD:DELETE, should be enough.

Thus, if there are no objections, I will close the
issue.

I think there are a few issues to consider before closing this.

The key paragraph from the CAP Requirements document is as
follows:

"CAP MUST allow synchronization, meaning at a minimum that the CUA
is able to find and retrieve new, modified or deleted entries for a given
time period. The CUA MUST be able to find out which entries have been
added, modified or deleted since it last synchronized, in order to
operate in disconnected mode. This requirement may be satisfied using the
query requirements already defined."

In other words it must be possible for a sync engine (client, server,
whatever) to be able to get the answers to the following three
questions.

1) Within the given date range what entries are new?

2) Within the given date range what entries have changed?

3) Within the given date range what entries have been deleted?

Using a query on last modified time with a date range should allow you to
gain answers to 1) and 2). The sync engine just needs to keep track of
the time stamp from the last time they asked. Most sync solutions today
have a sync context or anchor (or some other fancy name) but when dive in
you'll find they all pretty much are just exactly that, the time stamp
from the last successful sync. The sync engine need merely enumerate
through the results and all the IDs it recognizes are changes and all the
ones it doesn't are new.

The only problem I can foresee has to do with instances of recurring
entries. If I change the time of an instance of a meeting or delete an
instance this will change the recurrence rule for this entry and
consequently the last modified time but if I am doing a query over a
specific time period  the first instance of the repeating may not be
in this time period so will I get it? What does DTSTART and DTEND mean in
this case? I think this is a basic problem that the people discussing the
CAP-QL need to address. If I am trying to get all the entries in a week
to populate a week view for example, I need some guarantee that I will
get the entries with recurrence rules that cause instances to fall within
my range. If the CAP-QL addresses this then I guess the issue for sync
will be taken care of as well.

Now we get to question 3). You suggest that we query for the method
DELETE within a date range. Combined with the last modified time I'll get
all the newly marked DELETE entries since the last time I sync'd. This
all seems great but is assumes that all CUAs will use a MODIFY command
with the DELETE method. The problem is that there is also a command
(CANCEL or is it DELETE??) that can wipe out an entry completely. If one
CUA does it one way and another the other then how can the sync engine be
sure that it is really being told about everything that has been deleted?
You don't want the sync engine having to query for ever entry ID it knows
about to see if it still there. Is there some guarantee that CAP
compliant CUAs will use the MODIFY command?

One last issue with deletes has to do with recurring entries. If I delete
an instance then there is now an exception in my rule that would keep an
instance from falling within my interested range. Even if the above
failing of CAP-QL is resolved I may not be able to get this? Is there a
way to query using a range and the last modified time combined with a
search for an EXDATE?

George

--

Mark Paterson

Director, Client R&D

Steltor

mailto:[email-masked]

http://www.steltor.com

--

Mark Paterson

Director, Client R&D

Steltor

mailto:[email-masked]

http://www.steltor.com

---

From: "Shannon J. Clark" <[email-masked]>
Date: Fri, 18 Jan 2002 10:40:35 -0600

Just a few comments:

1 - in looking at Sync it is probably very valuable to look at the work of
the SyncML Consortium - it includes most of the software companies building
products that require sync functionality and they have resolved many of
these issues.

2 - generally the best practice is NOT for the server (or sync engine) to
track the "last sync" - this does not scale very gracefully. Rather the
client application should send a request of the format "I'm current as of
THISDATE - what does it take to make me current?" - note that the actions
and issues are different in a SERVER --> CLIENT sync vs. a SERVER <-->
CLIENT two-way sync. - in the later case there is the real possibility that
the client is more current than the server.

3 - thus for clean sync functionality a Calendar server should be able to
answer the following question(s):

    A. What has changed since DATE? (in the context of a given user and
specific VCALENDAR (or VAGENDA?)

        Changed includes - NEW entries, MODIFIED entries, DELETED entries -
all have been "written" - if you look at "DELETE" as a special case of
"writing" - and if you are using TOMBSTONE it is.

    B (optional?) What is "different" between CUA's copy of a VCALENDAR and
CS's copy.

Hope this helps,

Shannon
  -----Original Message-----
  From: [email-masked]
[mailto:[email-masked]]On Behalf Of Mark Paterson
  Sent: Friday, January 18, 2002 8:09 AM
  To: [email-masked]
  Subject: Fwd: Re: Synchronization [Was: Re: CAP: Last Call By March IETF:
Need Volunteers!]

  For some  reason this didn't seem to ever show up on the list. Take II.

    Date: Thu, 17 Jan 2002 13:04:03 -0500
    To: "[email-masked]" <[email-masked]>
    From: Mark Paterson <[email-masked]>
    Subject: Re: Synchronization [Was: Re: CAP: Last Call By March IETF:
Need  Volunteers!]

      Doug Royer wrote:
      >
      > > >
      > > > > 5) Synchronization:
      > > > > -------------------
      > > > >
      > > > >   Investigate if there is enough in CAP to allow for
synchronization?
      > > > >  I believe this is in the requirements doc.
      > > >
      > > > It is. I think it does meet the synchronization requirements.
      > > > And we limited synchronization to a bare minimum. That is
      > > > that you must be able to upload and download an entire calendar
      > > > and what ever it takes to make two calendars look the same.
      > >
      > >   Have we gone through the exercise of making sure that we
      > > support synchronization? If we limit support to a bare minimum
      > >   A CUA is able to query on the last modified LAST-MODIFIED
      > > property to find all the components that have changed the last
      > > time a sync was done. Using the METHOD:DELETE it can find all
      > >   Should we add a small section or sub-section on CAP

        Thus, if there are no objections, I will close the issue.

    The key paragraph from the CAP Requirements document is as follows:

    "CAP MUST allow synchronization, meaning at a minimum that the CUA is
able to find and retrieve new, modified or deleted entries for a given time
period. The CUA MUST be able to find out which entries have been added,
modified or deleted since it last synchronized, in order to operate in
disconnected mode. This requirement may be satisfied using the query
requirements already defined."

    In other words it must be possible for a sync engine (client, server,
whatever) to be able to get the answers to the following three questions.

    Using a query on last modified time with a date range should allow you
to gain answers to 1) and 2). The sync engine just needs to keep track of
the time stamp from the last time they asked. Most sync solutions today have
a sync context or anchor (or some other fancy name) but when dive in you'll
find they all pretty much are just exactly that, the time stamp from the
last successful sync. The sync engine need merely enumerate through the
results and all the IDs it recognizes are changes and all the ones it
doesn't are new.

    The only problem I can foresee has to do with instances of recurring
entries. If I change the time of an instance of a meeting or delete an
instance this will change the recurrence rule for this entry and
consequently the last modified time but if I am doing a query over a
specific time period  the first instance of the repeating may not be in this
time period so will I get it? What does DTSTART and DTEND mean in this case?
I think this is a basic problem that the people discussing the CAP-QL need
to address. If I am trying to get all the entries in a week to populate a
week view for example, I need some guarantee that I will get the entries
with recurrence rules that cause instances to fall within my range. If the
CAP-QL addresses this then I guess the issue for sync will be taken care of
as well.

    Now we get to question 3). You suggest that we query for the method
DELETE within a date range. Combined with the last modified time I'll get
all the newly marked DELETE entries since the last time I sync'd. This all
seems great but is assumes that all CUAs will use a MODIFY command with the
DELETE method. The problem is that there is also a command (CANCEL or is it
DELETE??) that can wipe out an entry completely. If one CUA does it one way
and another the other then how can the sync engine be sure that it is really
being told about everything that has been deleted? You don't want the sync
engine having to query for ever entry ID it knows about to see if it still
there. Is there some guarantee that CAP compliant CUAs will use the MODIFY
command?

    One last issue with deletes has to do with recurring entries. If I
delete an instance then there is now an exception in my rule that would keep
an instance from falling within my interested range. Even if the above
failing of CAP-QL is resolved I may not be able to get this? Is there a way
to query using a range and the last modified time combined with a search for
an EXDATE?

      George

    --
    Mark Paterson
    Director, Client R&D
    Steltor
    mailto:[email-masked]
    http://www.steltor.com

  --
  Mark Paterson
  Director, Client R&D
  Steltor
  mailto:[email-masked]
  http://www.steltor.com

---

From: Mark Paterson <[email-masked]>
Date: Fri, 18 Jan 2002 12:06:04 -0500

At 10:40 AM 1/18/2002 -0600, Shannon J. Clark wrote:

Just a
few comments:

1 - in looking at Sync it is probably very
valuable to look at the work of the SyncML Consortium - it includes most
of the software companies building products that require sync
functionality and they have resolved many of these
issues.

I am quite familiar with the SyncML specs and yes they have solved lots
of Sync Issues but none of these are of any consequence to this group. To
do a sync, whether it is with SyncML or not, you need to be able to get
the answers to the 3 questions I listed. We simply need to resolve
whether or not we feel the CAP draft lives up to the sync requirements
listed within the CAP requirements document. I brought a couple of things
that worry me in the hopes that the gang who know the CAP draft off by
heart (George, Doug, John, etc...) will reply and convince me that my
worries are covered, in which case the issue can be considered
closed.

2 - generally the best practice is NOT for the
server (or sync engine) to track the "last sync" - this does
not scale very gracefully. Rather the client application should send a
request of the format "I'm current as of THISDATE - what does it
take to make me current?" - note that the actions and issues are
different in a SERVER --> CLIENT sync vs. a SERVER <--> CLIENT
two-way sync. - in the later case there is the real possibility that the
client is more current than the server.

I used the term "sync engine" to mean where ever the sync
smarts are located. It could be in a client or on a server. It is
implementation specific. Using the SyncML model it would be on the
server. SyncML also has a "Sync Anchor" and yes using the
SyncML model the SyncML client is responsible for saving it.

However none of this matters to this group. It is out of scope.

3 - thus for clean sync functionality a
Calendar server should be able to answer the following
question(s):

A. What has changed since
DATE? (in the context of a given user and specific VCALENDAR (or
VAGENDA?)

Changed includes - NEW entries, MODIFIED entries, DELETED entries - all
have been "written" - if you look at "DELETE" as a
special case of "writing" - and if you are using TOMBSTONE it
is.

Your A is simply my 3 questions rolled into one.

B (optional?) What is
"different" between CUA's copy of a VCALENDAR and CS's
copy.

You can always do a sync this way. Download a users whole agenda and
compare it yourself against what you have and figure out the changes
yourself. I wasn't around when the CAP requirements docs was first
written but from what I've read I am certain they were striving for A
since any sync solution based on your B approach is very slow and would
be very demanding on the CS.

Hope this helps,

Shannon

-----Original Message-----

From: [email-masked]
[mailto:[email-masked]]On
Behalf Of Mark Paterson

Subject: Fwd: Re: Synchronization [Was: Re: CAP: Last Call By
March IETF: Need Volunteers!]

For some  reason this didn't seem to ever show up on the list.
Take II.

Doug Royer wrote:

> > > >   Investigate if there is enough in CAP
to allow for synchronization?

doc.

requirements.

is

calendar

same.

minimum

LAST-MODIFIED

last

all

CAP

I think there are a few issues to consider before closing
this.

"CAP MUST allow synchronization, meaning at a minimum that the
CUA is able to find and retrieve new, modified or deleted entries for a
given time period. The CUA MUST be able to find out which entries have
been added, modified or deleted since it last synchronized, in order to
operate in disconnected mode. This requirement may be satisfied using the
query requirements already defined."

3) Within the given date range what entries have been
deleted?

Using a query on last modified time with a date range should allow
you to gain answers to 1) and 2). The sync engine just needs to keep
track of the time stamp from the last time they asked. Most sync
solutions today have a sync context or anchor (or some other fancy name)
but when dive in you'll find they all pretty much are just exactly that,
the time stamp from the last successful sync. The sync engine need merely
enumerate through the results and all the IDs it recognizes are changes
and all the ones it doesn't are new.

One last issue with deletes has to do with recurring entries. If I
delete an instance then there is now an exception in my rule that would
keep an instance from falling within my interested range. Even if the
above failing of CAP-QL is resolved I may not be able to get this? Is
there a way to query using a range and the last modified time combined
with a search for an EXDATE?

George

--

Mark Paterson

Director, Client R&D

Steltor

mailto:[email-masked]

http://www.steltor.com

--

Mark Paterson

Director, Client R&D

Steltor

mailto:[email-masked]

http://www.steltor.com

--

Mark Paterson

Director, Client R&D

Steltor

mailto:[email-masked]

http://www.steltor.com

---

From: Doug Royer <[email-masked]>
Date: Fri, 18 Jan 2002 10:33:54 -0700

Mark Paterson wrote:
> > is able to find and retrieve new, modified or deleted entries for a
> > given time period. The CUA MUST be able to find out which entries
> > have been added, modified or deleted since it last synchronized, in
> > order to operate in disconnected mode. This requirement may be
> > satisfied using the query requirements already defined."
> >
> > In other words it must be possible for a sync engine (client,
> > server, whatever) to be able to get the answers to the following
> > three questions.
> >
> > solutions today have a sync context or anchor (or some other fancy
> > name) but when dive in you'll find they all pretty much are just
> > exactly that, the time stamp from the last successful sync. The sync
> > engine need merely enumerate through the results and all the IDs it
> > recognizes are changes and all the ones it doesn't are new.

> > entries. If I change the time of an instance of a meeting or delete
> > an instance this will change the recurrence rule for this entry and
> > in this time period so will I get it?

Change the RRULE? If you add an EXDATE or EXRULE, it will change
the VEVENT, but not the RRULE. Or if an instance at the beginning
or end is deleted - then, yes - you could change the RRULE.
Both will change the DTSTAMP.

This is also why the EXPAND property exists.

EXPAND:FALSE - yes, because it is part of the VEVENT and part of
               the VEVENT falls within that range.

EXPAND:TRUE  - no, you will only get the instances of that VEVENT
               that fall in that range.

> > What does DTSTART and DTEND
> > mean in this case? I think this is a basic problem that the people
> > discussing the CAP-QL need to address. If I am trying to get all the
> > entries in a week to populate a week view for example, I need some
> > guarantee that I will get the entries with recurrence rules that
> > cause instances to fall within my range. If the CAP-QL addresses
> > this then I guess the issue for sync will be taken care of as well.

I think it does with RRULE.

> > ...What does DTSTART and DTEND
> > mean in this case?...

I do think that we need to add another note to VQUERY,
this point was brought up before, I don't recall if it was
ever in any version of a draft.

   When performing VQUERYs that use DTEND, the CS MUST calculate:
	
	The effective DTEND for all components that have a DURATION
	property and match on those if they fall into the range
	supplied by DTEND.

   When performing VQUERYs that use DURATION, the CS MUST calculate:

	The effective DURATION for all components that have a DTEND
        property and match on those if they fall into the range
        supplied by DURATION.

Above when you sent:

> > In other words it must be possible for a sync engine (client,
> > server, whatever) to be able to get the answers to the following
> > three questions.
> >
> > ...
> >

It is possible for the CUA to calculate the differences given
the current state of any component in the CUA and the result
of a VQUERY to the CS. So I think the requirement is met.

And we need to add text about METHOD:DELETE and how that is
the way to mark somehting as deleted.

---

From: Mark Paterson <[email-masked]>
Date: Fri, 18 Jan 2002 15:00:33 -0500

At 10:33 AM 1/18/2002 -0700, you wrote:

Mark Paterson wrote:

this.

follows:

deleted?

I think you are reading more into what I wrote then I meant. When I
stated that the recurrence rule (not the RRULE) would change I simply
meant something within everything that went into deciding where instances
would end up would change and thus the entry. I should have said
recurrence set as is used in the iCalendar RFC. Sorry for the confusion.
I'm pretty sure I mean the same thing as you.

This is also why the EXPAND
property exists.

EXPAND:TRUE  - no, you will only get the instances of that
VEVENT

that fall in that range.

Either way it sounds like a properly tailored CAP-QL query will take care
of what I was worried about. Good.

When performing VQUERYs that use DTEND, the CS MUST
calculate:

The
effective DTEND for all components that have a DURATION

property
and match on those if they fall into the range

supplied
by DTEND.

When performing VQUERYs that use DURATION, the CS MUST
calculate:

The
effective DURATION for all components that have a DTEND

property and match on those if
they fall into the range

supplied by
DURATION.

deleted?

of a VQUERY to the CS. So I think the requirement is
met.

You don't want the Sync CUA asking the CS about each component it knows
about to see if it has been deleted. This would be a very heavy
operation. You want one call to tell you what's been deleted and I think
it can be done if Calendar CUAs all properly use the METHOD DELETE to
delete things.

And we need to add text about
METHOD:DELETE and how that is

It really needs to be a MUST for CUAs.

OK. I think my 1st and 2nd issues have been dealt with here. The only one
remaining is the last one about deleting an instance of a recurring
entry.If the Sync CUA is making a query over a date range how can it be
sure to catch this? It should be able to do something combining the
DTSTAMP and EXDATE I think?

--

Mark Paterson

Director, Client R&D

Steltor

mailto:[email-masked]

http://www.steltor.com

---

From: Doug Royer <[email-masked]>
Date: Fri, 18 Jan 2002 14:57:43 -0700

Mark Paterson wrote:
> You don't want the Sync CUA asking the CS about each component it
> knows about to see if it has been deleted. This would be a very heavy
> operation. You want one call to tell you what's been deleted and I
> think it can be done if Calendar CUAs all properly use the METHOD
> DELETE to delete things.

Good thing you raised more issues, I forgot ...

The LAST-MODIFIED property is the date-time the object was
modified. It DOES NOT mean the it is the newest authoritative data.

If the SEQUENCE number is greater in one of the objects, it
does not matter what the LAST-MODIFIED says, for the same UID,
and SEQUENCE set, the larger SEQUENCE number is the newest object.
When this happens, it only means that the older data was updated
more recently and that is why the LAST-MODIFIED on older data
may be newer.

Use SEQUENCE. Only the ORGANIZER can update the SEQUENCE number.

From iTIP:

   . For the "PUBLISH" and "REQUEST" methods, the "SEQUENCE" property
      value is incremented according to the rules defined in [iCAL].

   . The "SEQUENCE" property value MUST be incremented each time the
      "Organizer" uses the "ADD" or "CANCEL" methods.

   . The "SEQUENCE" property value MUST NOT be incremented when using
      "REPLY", "REFRESH", "COUNTER", "DECLINECOUNTER", or when sending a
      delegation "REQUEST".

So, if the data is updated, the SEQUENCE MUST BE incremented.
Do we need to specify this is how you tell in CAP?

So for cap, METHOD:CREATE (or BOOKED or whatever it is), the
SEQUENCE will be greater in the newest object controlled by
the ORGANIZER.

> OK. I think my 1st and 2nd issues have been dealt with here. The only
> one remaining is the last one about deleting an instance of a
> recurring entry.If the Sync CUA is making a query over a date range
> how can it be sure to catch this? It should be able to do something
> combining the DTSTAMP and EXDATE I think?

If you update a component, it's SEQUENCE number will be larger
in the newer data. LAST-MODIFIED will just tell you when the CUA
or CS last modified the (possibly old) data.

For components marked METHOD:DELETE, they exist so a CUA knows
that the CS does (did) in fact know about the object, and which
ever has the larger SEQUENCE wins. If the CUA has a SEQUENCE of
10, METHOD:REQUEST and the CS SEQUENCE of 11, METHOD:DELETE, then
the CUA can delete the component from the CUA. And we do not specify
how the CUA or CS knows to delete the entry from the CS because
you may have more than one CUA or CUA-bot that does that kind of
thing for you.

---

From: Mark Paterson <[email-masked]>
Date: Thu, 31 Jan 2002 10:15:23 -0500

At 02:57 PM 1/18/2002 -0700, Doug Royer wrote:

...

Use SEQUENCE. Only the ORGANIZER
can update the SEQUENCE number.
....

Sorry for taking so long to get back on this thread. My day job got in
the way :-)

I didn't quite get what you meant about older data appearing newer then
newer data at first but I suppose some CS implementations where all
messages ever sent are kept I suppose this is possible but all this means
is that when we do a query for changes a sync CUA needs to be careful to
filter out those objects whose SEQUENCE appears older then the SEQUENCE
which the sync CUA last remembers. It doesn't resolve my questions. A
sync CUA is still not going to fetch each and every object it knows about
to check the latest SEQUENCE to determine changes. This would be way too
heavy on the CS.

Let's try pulling back on reigns and refocus on my original questions. My
claim was that in order for CAP to live up to the Sync requirements
listed in the CAP Requirements a Sync CUA must be able to get the answers
to the following 3 questions:

Having followed the CAP-QL thread its perhaps time I took a crack at
building my own queries. I've made a few simplifying assumptions. I've
just worried about events, I've used DTSTART to determine if something is
in range and rather then figuring out what attributes to retrieve I've
just put "*". So here we go:

BEGIN:VQUERY

EXPAND:TRUE

QUERY:SELECT * FROM VEVENT WHERE

DTSTART < upper_range

AND DTSTART > lower range

AND DTSTAMP > last_time_asked

END:VQUERY

If the Sync CUA then filters out any objects whose SEQUENCE seems to
indicate that it is actually older then what the Sync CUA last knew about
(as per Doug's suggestion) then the results that remain should be all the
changes within the range. By enumerating through the results a Sync CUA
can determine everything that is new by the objects whose UID it doesn't
recognize, everything that has changed by the objects whose UID it does
recognize, and finally everything that has been deleted from the objects
within the changed list who now are marked with METHOD:DELETE (as long as
CUAs are forced to use this method to delete objects).

What about instances of a recurring object?

The EXPAND:TRUE line in the query should ensure that all new and changed
instances get returned. Correct?

The problem is still deleted instances. I don't think EXPAND:TRUE will
make sure that the Sync CUA gets returned instances marked with METHOD
DELETE within the range. If it does then we're covered but I somehow
don't think that's the way it is (comments from the CAP-QL
gang?).

Instead another query such as this is needed I think:

BEGIN:VQUERY

EXPAND:TRUE

QUERY:SELECT * FROM VEVENT WHERE

EXDATE < upper_range

AND EXDATE > lower range

AND DTSTAMP > last_time_asked

END:VQUERY

This would give a Sync CUA any newly changed recurring objects with
exceptions within the range, I think? It will only work however if CUAs
always add an EXDATE when an instance from a pre-existing recurrence set
is deleted. If they just redo the recurrence set completely then it
becomes impossible.

If my logic holds true (and that remains to be seen. I need the expertise
of the CAP-QL contributors) then text for the following should be
added:

1. CUAs must use METHOD:DELETE to mark something as deleted.

2. CUAs must use EXDATE to mark an instance within an existing Recurrence
set as deleted rather than redefining the recurrence set.

If we can do this and people agree that my logic makes sense then I think
we can mark off the synchronization requirements for CAP as
covered.

So does my logic make sense?

--

Mark Paterson

Director, Client R&D

Steltor

mailto:[email-masked]

http://www.steltor.com

---

From: Doug Royer <[email-masked]>
Date: Thu, 31 Jan 2002 10:51:40 -0700

Mark Paterson wrote:
> I didn't quite get what you meant about older data appearing newer
> then newer data at first but I suppose some CS implementations where
> all messages ever sent are kept I suppose this is possible but all
> this means is that when we do a query for changes a sync CUA needs to
> be careful to filter out those objects whose SEQUENCE appears older
> then the SEQUENCE which the sync CUA last remembers. It doesn't
> resolve my questions. A sync CUA is still not going to fetch each and
> every object it knows about to check the latest SEQUENCE to determine
> changes. This would be way too heavy on the CS.

I did not mean to imply that. I meant that LAST-MODIFIED only
tells you that it has been modified. It does not mean that
it is more-valid data simply because it has a newer time stamp.

> Let's try pulling back on reigns and refocus on my original questions.
> My claim was that in order for CAP to live up to the Sync requirements
> listed in the CAP Requirements a Sync CUA must be able to get the
> answers to the following 3 questions:
> 
> just worried about events, I've used DTSTART to determine if something
> is in range and rather then figuring out what attributes to retrieve
> I've just put "*". So here we go:
> 
> BEGIN:VQUERY
> EXPAND:TRUE

 (replace DTSTAMP with LAST-MODIFIED and I think it is right,
  I incorrectly used DTSTAMP when I should have said LAST-MODIFIED.
  DTSTAMP is the time the iCalendar object was created for transport
  and is ALWAYS updated [per iTIP?] just before transporting the
  object.)

> indicate that it is actually older then what the Sync CUA last knew
> about (as per Doug's suggestion) then the results that remain should
> be all the changes within the range.

Yes, and by also filtering out from the returned set any
with the same UID and older SEQUENCE numbers in the return set.
As in if there were 5 updates between your sync times, you may get:

   UID:1, and 5 objects, with SEQUENCE numbers 1 2 3 4 5

Where only '5' would be the newest and 1, 2, 3, 4 are discarded
using iTIP rules.

> By enumerating through the
> results a Sync CUA can determine everything that is new by the objects
> whose UID it doesn't recognize, ...

Yes.

> ... everything that has changed by the
> objects whose UID it does recognize, ..

No - changes would have the same UID for the same objects
and has a newer SEQUENCE number.

> .. and finally everything that has
> been deleted from the objects within the changed list who now are
> marked with METHOD:DELETE (as long as CUAs are forced to use this
> method to delete objects). 

iCalendar objects are marked for delete with METHOD:MODIFY.
They are deleted from the CS by <delete/>.

> The EXPAND:TRUE line in the query should ensure that all new and
> changed instances get returned. Correct?

Yes and all existing and unchanged instances.

> don't think that's the way it is (comments from the CAP-QL gang?).

The instances of objects with a RECURRENCE-ID do not exists anyway.
They are an expansion of an existing object that is expanded and
for booked objects only. The ORGANIZER never sends a METHOD:REQUEST
object that contains a RECURRENCE-ID.

And don't forget about METHOD:CANCEL sent by the ORGANIZER,
which can cancel entire objects or specific instances.

So, if the EXPAND:FALSE object is marked METHOD:DELETE, then ALL
of the instances are deleted and that object will NEVER have
a RECURRENCE-ID property (because EXPAND is FALSE).

iTIP allows for updates to instances of objects. When that happens
there can be multiple objects created:

	'One' new object with an updated SEQUENCE

	'More than one' object each with the SAME UID and SEQUENCE.
	And each object specifying one or more instances.
	(see ADD in iTIP) - and all booked.

And if an object is METHOD:CANCEL, then its containing
DTSTART, DTEND/DURATION, RRULE, EXRULE, RDATE, and EXDATE properties
specific instances that have been canceled, for the same
SEQUENCE number and UID objects. (see iTIP).

> exceptions within the range, I think?

No, an object can be modified and instances can be removed by
changing the RRULE and never adding a EXDATE. Or by adding
a RDATE, or by adding or altering a EXRULE, or by DTSTART,
or RECURRANCE-ID, ...

> It will only work however if
> CUAs always add an EXDATE when an instance from a pre-existing
> recurrence set is deleted. If they just redo the recurrence set
> completely then it becomes impossible.

Not impossible. Your first query (with LAST-MODIFIED) will
get all of the objects in that time range. Then the CUA
must compare using SEQUENCE and merging any multiple objects
with the same UID (originally created with ADD), then
it knows the state of the 'UID' object.

> If my logic holds true (and that remains to be seen. I need the
> expertise of the CAP-QL contributors) then text for the following
> should be added:
> 

Entire objects are marked for delete by using METHOD:MODIFY
and changing the object in the store to METHOD:DELETE.

It is NOT used when CANCELing instances of an object.

> 2. CUAs must use EXDATE to mark an instance within an existing
> Recurrence set as deleted rather than redefining the recurrence set.

No. That is not practical.

> If we can do this and people agree that my logic makes sense then I
> think we can mark off the synchronization requirements for CAP as
> covered.

Not yet :-)

> So does my logic make sense?

As noted above.