Subject: (#1) CAP Synchronization proposal.

From: Doug Royer <[email-masked]>
Date: Wed, 13 Feb 2002 21:53:00 -0700

Because of multiple questions raised on this list, the editors
have decided that we need a "Synchronization" section in CAP.
And because it is a in the CAP requirements document that
a CUA be able to synchronize with a CS.

Here is attempt (#1) at the text for that new section:
-------------------------------------------------------------------------
X. "Synchronization between a CUA and CS"

There are two major issues with synchronization (1), iTIP
updates and dealing with out of order or missing iTIP components,
and (2) updating booked entries.

Synchronization is a two sided process. That is you need to update
the CUA with any new object from the CS. And any new objects in the
CUA may need to be deposited into the CS.

Below the term "zombie object" will refer to any out of order ITIP
object that is dead but does not yet know it. As in METHOD:CANCEL,
UID:123, SEQUENCE:10 has been processed, and then you get
METHOD:REQUEST, UID:123, SEQUENCE:9. So the second iTIP object is
lurking around
waiting to be reaped by the CUA.

X.1 "iTIP and synchronization"

There is no difference between the iMIP handling of iTIP objects
and CAP handling of iTIP messages once the iTIP objects are in the
CUA. So this section will describe some CS and CUA implementation
considerations. Such as how to fetch iTIP objects from the CS, when
to delete them from the CS, and when to leave them in the CS for
later processing.

This is not a tutorial on how to process iTIP objects. This is
an outline of the problems that may be unique to an iTIP objects
when a CUA is synchronizing with a CS.

The section titled "Query for all Non-Booked Entries" shows examples
of how to fetch for iTIP objects. It is recommended that the CUA
fetch iTIP objects with EXPAND:FALSE in order to simplify the
synchronization process.

Fetching of all of the data booked, not booked, and marked for
delete, SHOULD BE performed using one VQUERY to reduce the
bandwidth by not having to keep sending commands to the
CUA which can be expensive on slow networks. 

The first step is to fetch all objects where the LAST-MODIFIED
date is greater then the CUAs last synchronization date.

The next step is to pull out any marked METHOD:CREATE (booked).
These may be new or updated objects. Compare them to the
METHOD:CREATE objects in the CUA with the same UID:

    If the CUA does not have that UID - add the new object.

    If the CUA does have that UID:

	compare the SEQUENCE values:

		If the object with the higher SEQUENCE value
		came from the CS drop the one in the CUA
                and have it use the new one from the CS.

		If the object with the higher SEQUENCE value
		is from the CUA, then the CUA must delete the
		older object from the CS and add the newer
		object to the CS.

		If the SEQUENCE numbers are the same then keep
		the one with the greater LAST-MODIFIED date.

			If the one with the older LAST-MODIFIED
			date is from the CS. Then that object
			must be deleted from the CS and it
			must be replaced with the newer one
			from the CUA.

    If the CUA has UIDs that have a LAST-MODIFIED time greater
    than the last synchronization time, and no object came back
    from the query with the same UID, then those new objects
    need to be added or updated in the CS. This will require
    a separate query to determine if they exist at all in the CS
    before deciding if they are new to the CS or need to be
    updated in the CS.

    Now examine the UIDs that you fetched for having METHOD:DELETE.

	The CUA should note the UID and its SEQUENCE value so that
        in later synchronization's the CUA can know that any out of
        order (zombie) iTIP objects can be deleted from the CS and
        otherwise ignored in the CUA. And drop these from the CUA
        and know to later delete them from the CS if the zombie
	objects later arrive.

	At this point the CUA has two choices:

		It can delete those METHOD:DELETE objects from
		the CS with command <delete>.

		Or it can keep them in the CS in case other CUAs
		will be synchronizing with that same calendar and
		will need to know that those UIDs should be deleted
		from the CUAs.

		It is up to the CUA or CU and the CS or CS 
		administrator to decide when the METHOD:DELETE
		object are to be deleted from the CS. And this
		is not specified in CAP.

    Now start processing the iTIP object in the CUA just like
    you would as described in iTIP and iMIP. Then for each of
    those iTIP object that were processed, update the CS with
    the latest version of the object, followed by drop those
    processed iTIP objects from both the CUA and delete them
    from the CS.

    Any iTIP objects that can not be processed yet (example, a
    METHOD:COUNTER, UID:abc, SEQUENCE:1 when you have not yet have
    METHOD:REQUEST, UID:abc in the CUA or CS). These would be left
    in the CS until the CUA or CU and the CS or CS administrator
    decided that they were zombies or just too old to care about
    and then they would be deleted from the CS. And their LAST-MODIFIED
    time MUST BE updated in the CS so that they would continue to
    be fetched for the next synchronization until they were used
    or deleted.

The CUA MUST update the LAST-MODIFIED times of any object that
it updates or adds to the CS. This needs to be done by the 
CUA and not the CS in order for there not to be a race condition
that is caused by the CUA object always being older than
the same object in the CS.

Some optimizations can be done. For example the CUA may
just use <modify> and not <delete> followed by <create> to
update the CS. And many of the updates, deletes, and creates
could be combined and processed at the end, or as they are
determined, which ever works best for the implementations.

---

From: Mark Paterson <[email-masked]>
Date: Fri, 15 Feb 2002 13:44:00 -0500

At 09:53 PM 2/13/2002 -0700, Doug Royer wrote:

Because of multiple questions
raised on this list, the editors

have decided that we need a "Synchronization" section in
CAP.

I think that the idea of this section is good but it should concentrate
simply on showing that it is possible to query for all changes since the
last time such a query was done. It should not get into assuming how a
sync solution should work. That is for the sync solution implementors of
the world to decide and beyond the scope of CAP. We need to be careful
not to lecture as to how a sync solution should work and I believe that
this version does tread into this area. For example what if a sync
solution implementor decides to only sync booked entries? This is
perfectly valid but this section makes it sound like you MUST worry about
unprocessed ITIP objects. You may want your sync solution to process
them, you might not. It is not for CAP to decide. If it does then it is
really no different then any other CUA.

I had started my own proposal and I will try and make sure it gets posted
before the end of the day (I know that today is a cutoff day for
proposals). It actually covers a lot of the same stuff but focus's more
on how you get changes as opposed to how you do sync.

I have more comments interspersed below...

There are hundreds of major issues with sync and they are all beyond the
scope of CAP. With respect to CAP however the main decision you need to
make when you query for changes is however, whether you only care about
booked entries or whether you also are worried about unprocessed ITIP
objects.

Synchronization is a two sided
process. That is you need to update

Synchronization can be a three sided, four sided, n sided process. Being
clear on what we mean by Synchronization is important but we should be
careful about how we define it.

Below the term "zombie
object" will refer to any out of order ITIP

lurking around

Only concern if you plan to sync more than just booked entries.

There is no difference between the
iMIP handling of iTIP objects

later processing.

When to leave them in the CS for later processing??? If it's been
processed then that's that. Where in CAP does it describe when a CUA
should decide to leave them and when should it scrap them? Perhaps there
is such a heuristic defined somewhere but I don't remember it.

This is not a tutorial on how to
process iTIP objects. This is

The section titled "Query for all Non-Booked Entries" shows
examples

synchronization process.

Perhaps but you just downloaded essentially an entire CU's calendar it's
going to be a heavy operation no matter what.

The first step is to fetch all
objects where the LAST-MODIFIED

If the CUA does not have that UID - add the new
object.

compare
the SEQUENCE values:

If
the object with the higher SEQUENCE value

came
from the CS drop the one in the CUA

If
the object with the higher SEQUENCE value

is
from the CUA, then the CUA must delete the

older
object from the CS and add the newer

object
to the CS.

We need to be much more careful with out wording. Sync solution's have
their own policies for conflict resolution when a change has occurred
both client side and server side. We are trying to convey that the server
sequence is out of date so it isn't meaningful and no change has really
occurred on the server so the client change can take precedence but we
are not interfering with any policy the sync solution itself may have.
This isn't really a sync issue though. Any CUA processing ITIP objects
needs to worry about this. This isn't unique to a Sync CUA.

If
the SEQUENCE numbers are the same then keep

the
one with the greater LAST-MODIFIED date.

If
the one with the older LAST-MODIFIED

date
is from the CS. Then that object

must
be deleted from the CS and it

must
be replaced with the newer one

from
the CUA.

Absolutely not.  If they have both been changed since the last sync
and the sequences show that they are both up-to-date then the sync
solution's conflict resolution policy will decide and it is none of CAP's
business.

If the CUA has
UIDs that have a LAST-MODIFIED time greater

than the last synchronization time, and no object came
back

from the query with the same UID, then those new
objects

need to be added or updated in the CS. This will
require

a separate query to determine if they exist at all in
the CS

before deciding if they are new to the CS or need to
be

updated in the CS.

How a sync solution determines if a new object exists or not is none of
CAP's business. A typical sync solution keeps some sort of database
linking device side ID's with server side ones so entries on the device
not yet in the database are new. There is no query required.  Some
vendors may decide they do want to make a query with their solution but
that is out of scope.

Now examine the
UIDs that you fetched for having METHOD:DELETE.

The CUA
should note the UID and its SEQUENCE value so that

in later synchronization's the
CUA can know that any out of

order (zombie) iTIP objects
can be deleted from the CS and

otherwise ignored in the CUA.
And drop these from the CUA

and know to later delete them
from the CS if the zombie

objects
later arrive.

At this
point the CUA has two choices:

It
can delete those METHOD:DELETE objects from

the
CS with command <delete>.

Do you mean the ITIP object or the booked entry? The ITIP object can be
deleted and the booked entry modified and marked METHOD:DELETE.

Or
it can keep them in the CS in case other CUAs

will
be synchronizing with that same calendar and

will
need to know that those UIDs should be deleted

from
the CUAs.

The ITIP object has been processed and therefore should be zapped. The
booked entry is now marked method DELETE so other CUAs can see this.

It
is up to the CUA or CU and the CS or CS

administrator
to decide when the METHOD:DELETE

object
are to be deleted from the CS. And this

is
not specified in CAP.

It should be. A CUA which processes an ITP cancel should mark the booked
entry as METHOD:DELETE.

Now start
processing the iTIP object in the CUA just like

you would as described in iTIP and iMIP. Then for each
of

those iTIP object that were processed, update the CS
with

the latest version of the object, followed by drop
those

processed iTIP objects from both the CUA and delete
them

from the CS.

Any iTIP objects that can not be processed yet
(example, a

METHOD:COUNTER, UID:abc, SEQUENCE:1 when you have not
yet have

METHOD:REQUEST, UID:abc in the CUA or CS). These would
be left

in the CS until the CUA or CU and the CS or CS
administrator

decided that they were zombies or just too old to care
about

and then they would be deleted from the CS. And their
LAST-MODIFIED

time MUST BE updated in the CS so that they would
continue to

be fetched for the next synchronization until they
were used

or deleted.

Out of scope. All Sync solution implementors know they have to do
something to ensure that the changes they initiated don't come back the
next time they ask for changes. How they do  so is implentation
specific. There are hundreds of other things you should be aware of when
doing a sync but just like this one it is out of scope for CAP.

Some optimizations can be done. For
example the CUA may

just use <modify> and not <delete> followed by <create>
to

--

Mark Paterson

Director, Client R&D

Steltor

mailto:[email-masked]

http://www.steltor.com