Subject: [VCARDDAV] PIDs proposal

From: Simon Perreault <[email-masked]>
Date: Tue, 3 Jun 2008 09:06:09 -0400

Hello all,

Here's a proposal for -02:

- Rename UID parameter to PID.
- Change PID value type to "a small integer."
- Adapt examples so that they match the above.

As far as the vCard format is concerned, this is pretty much all we need to 
specify. But if you only read that document and you want to implement a 
synchronization engine, you will be absolutely lost as to how to make use of 
the PID parameter.

So we will need either to expand the Synchronization section or write a 
separate informational document. What's missing is the mechanisms:

- How UA's should handle PID values when
  - Creating a new property.
  - Synchronizing with another party.
- The situations that give rise to conflicts in PID values.
- How conflicts in PID values are detected and handled.
- A rationale for why PIDs are "small integers" and why it is sufficient.

I would gladly add that to the Synchronization section, but right now I have 
no idea how one could write a good synchronization engine with only small 
integers.
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: Mark Paterson <[email-masked]>
Date: Wed, 04 Jun 2008 11:02:52 -0400

Simon Perreault wrote:
> separate informational document. 
Or have the specifications that call for its use describe its usage for 
their particular context (e.g the OMA DS Enabler Specifications)

Mark
> What's missing is the mechanisms:
>

-- 

Oracle Email Signature Logo
Mark Paterson | Product Manager, UC & Mobile Services | 514.905.8649
Oracle Server Technologies - Collaboration
600 Blvd. de Maissoneuve West, Suite 1900, Montreal, Quebec, H3A 3J2
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: "Darryl Champagne" <[email-masked]>
Date: Sat, 21 Jun 2008 16:32:32 -0400

I basically agree with Mark here, vCard specs don't need to explain how to
write a sync engine, there are other protocols that describe that, including
examples, overviews of what is needed to sync, issues, and so on (and there
is at least one book as well).  

It is also important for the people on this list to realize that there are
real world scenarios where there is no single correct answer - sync engines
may do different things based upon the target audience, or many other
factors.  For example, many sync engines will refuse to delete a majority of
items, even if that is exactly what was done on one device, because it is
typically caused by user error (such as allowing one's children to play with
the device), rather than really being what was desired.  Similarly, many
cases where a property has been added to multiple devices between syncs are
because the user attempted to enter the same data in multiple places, so
duplication is rarely desired, even if that is what is technically the most
correct choice.

Overall, a brief overview seems sufficient.

Specifically:

Revise the example in section 6, from the unlikely possibility of multiple
FN's:

     FN;UID="urn:uuid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6":Jane Doe

To a more typical case of multiple Tels:

     TEL;PID=3;TYPE=home:tel:+33-01-23-45-67

I do want to mention that the use of URI's with their embedded colons is
slightly confusing with the use of colon's to signify the start of the data
- ":tel:".  It's not a real issue, it just seems less elegant.

Of course, update the ABNF, from uidparam to pidparam.

Section 6.4 is updated as described below, including deleting the phrase ",
and MUST be present if synchronization is to be used."

The result would be something like:

6.4 PID

The "pid" parameter is used to identify a specific property among multiple
instances.  It plays a role analogous to the UID property (Section 7.7.7) on
a per-property instead of per-vCard basis. It MUST NOT appear more than once
in a given property.  It MUST NOT appear on properties that only may have
one instance per vCard.  Note that the "pid" parameter is not globally
unique, so it is possible for duplicate values to be created.

Also, note that I think several properties in addition to UID should have
prohibition against having multiple instances, such as the REV property (or
I am just missing it) - and is it really settled regarding multiple entries
in a single vCard?  By specifying the restriction generically, we can
eliminate the specific prohibition against having UID/PID on the UID param.

.

Update the end of section 7.7.7 UID from:

This property MUST NOT appear more than once in a given vCard, and MUST be
present if synchronization is to be used.  See Section 8.

The UID parameter MUST NOT be applied to this property.

To just:

This property MUST NOT appear more than once in a given vCard.

or perhaps:

This property MUST NOT appear more than once in a given vCard.  This
property is well-suited for synchronization.  See Section 8. 

Note that we must not require UID, as some devices that expect to be
synchronized will not be able to preserve it.

In section 8.1, revise from:

Two vCards for which the UID properties (Section 7.7.7) are equivalent MUST
be considered representations of the same object. Equivalence is determined
as specified in [RFC3986], Section 6.  A missing UID property MUST NOT be
considered equivalent to any other. Hence, the UID property MUST be present
if synchronization is to be used.

Pairs of properties, one from each of a pair of vCards representing the same
object, having equivalent names and UID parameter values MUST be considered
representations of the same object.  A missing UID parameter MUST NOT be
considered equivalent to any other.  Hence, the UID parameter MUST be
present if synchronization is to be used.

It is expected that a synchronization engine will exploit such property
associations by e.g. overwriting the destination property value and
parameters with the source destination property value and parameters.  The
particular merging strategy is out of scope for this document.

To something like:

Two vCards for which the UID properties (Section 7.7.7) are equivalent MUST
be considered representations of the same object. Equivalence is determined
as specified in [RFC3986], Section 6.  

vCards without a UID property may be matched to vCards with a UID property
where a synchronization engine determines there is sufficient similarity to
assume equivalence.  The particular strategy and criteria used is out of
scope for this document.

Updates to vCards with multiple instances of particular properties may use
the PID associated with each property to aid in determining what values have
changed.  Since PIDs are not globally unique, they can only be used as
guidelines to synchronization engine logic.  Such logic is out of scope for
this document.

For section 8.2, update from:

   Two vCards are to by synchronized:

...

Etc.

...

to:

   Two vCards are to be synchronized:

     BEGIN:VCARD

     VERSION:4.0

     UID:urn:uuid:77a01597-0603-40f3-8138-36deca8618da

     FN:Jane Doe

     TEL;PID=1;TYPE=home:tel:+33-01-23-45-67

     TEL;PID=3;TYPE=home:tel:+1-800-555-1234

     EMAIL;PID=1:[email-masked]

     IMPP;TYPE=personal,pref:xmpp:[email-masked]

     END:VCARD

     BEGIN:VCARD

     VERSION:4.0

     FN:Jane Doe

     TEL;PID=1;TYPE=home:tel:+33-01-23-45-67

     TEL;PID=4;TYPE=home:tel:+1-800-555-5678

     EMAIL;PID=1:[email-masked]

     IMPP;TYPE=personal,pref:xmpp:[email-masked]

     END:VCARD

   Assuming a synchronization engine is presented with the two vCards, it
may decide based upon logic out of scope of this document, that both vCards
are representations of the same object, and create a merged vCard such as:

     BEGIN:VCARD

     VERSION:4.0

     UID:urn:uuid:77a01597-0603-40f3-8138-36deca8618da

     FN:Jane Doe

     TEL;PID=1;TYPE=home:tel:+33-01-23-45-67

     TEL;PID=3;TYPE=home:tel:+1-800-555-1234

     TEL;PID=4;TYPE=home:tel:+1-800-555-5678

     EMAIL;PID=1:[email-masked]

     IMPP;TYPE=personal,pref:xmpp:[email-masked]

     END:VCARD

If the synchronization engine then is presented with an updated vCard such
as:

     BEGIN:VCARD

     VERSION:4.0

     UID:urn:uuid:77a01597-0603-40f3-8138-36deca8618da

     FN:Jane Doe

     TEL;PID=1;TYPE=home:tel:+33-01-23-45-67

     TEL;PID=4;TYPE=home:tel:+1-800-555-5678

     EMAIL;PID=1:[email-masked]

     IMPP;TYPE=personal,pref:xmpp:[email-masked]

     END:VCARD

It may use the PIDs on each property to determine that the 2nd phone number
in the sequence has been deleted.  Note that there may be data beyond what
is available within a vCard, such as a SpeedDial number, that is specific to
each individual property instance, which is why providing a correlation
between versions is significant.

This is just a suggestion, meant to show an example of merging a vCard
without a UID with one that has one, and also showing a specific instance of
a property being modified.  A lot of this could be worded better, but this
is just a quick first pass to show the main ideas.

I hope this helps.

dgc

  _____  

From: [email-masked] [mailto:[email-masked]] On Behalf
Of Mark Paterson
Sent: Wednesday, June 04, 2008 11:03 AM
To: Simon Perreault
Cc: [email-masked]
Subject: Re: [VCARDDAV] PIDs proposal

Simon Perreault wrote: 

Hello all,

 

the PID parameter.

Mark

 

 

--

---

From: Aki Niemi <[email-masked]>
Date: Mon, 23 Jun 2008 10:57:53 +0300

> synchronization engine, you will be absolutely lost as to how to make use of 

That doesn't sound right. We can't just stick a new "Foo" property in,
but have no idea how it will be used or if it even works? Especially as
pervasive a change as this is compared to the previous version of vCard.

> no idea how one could write a good synchronization engine with only small 

I'd propose that the PID stays out vCard-bis until someone writes that
engine spec. Based on the email messages alone on this subject, I am at
least unconvinced that the PID solves any real protocol issue. At most,
it seems like a hack to get around either client bugs, or deficiencies
in sync protocols.

Cheers,
Aki

---

From: Simon Perreault <[email-masked]>
Date: Mon, 23 Jun 2008 07:44:44 -0400

On Monday 23 June 2008 03:57:53 Aki Niemi wrote:
> > I would gladly add that to the Synchronization section, but right now I
> > have no idea how one could write a good synchronization engine with only
> > small integers.
>
> engine spec.

Note that I don't think of that as an engine spec. It's only the 
PID "mechanisms", i.e. what a PID does/doesn't provide, what is its intended 
usage, etc. An engine spec would me much more complex and out of scope.

> Based on the email messages alone on this subject, I am at 

Glad to finally see I'm not alone.

-- 
Please try Numb, a STUN/TURN server implementation.
Free access at http://numb.viagenie.ca/.
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: "Chris Bryant" <[email-masked]>
Date: Mon, 23 Jun 2008 09:30:57 -0400

> it seems like a hack to get around either client bugs, or deficiencies

+1

USA.NET 
You Run Your Business.  We'll Run Your Email.

This message is for the sole use of the intended recipient(s) and may contain confidential and/or privileged information of USA.NET, Inc.  Any unauthorized review, use, copying, disclosure, or distribution is prohibited.  If you are not the intended recipient, please immediately contact the sender by reply email and delete all copies of the original message.

---

From: "Darryl Champagne" <[email-masked]>
Date: Mon, 23 Jun 2008 17:50:42 -0400

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Simon Perreault
> Sent: Monday, June 23, 2008 7:45 AM
> To: [email-masked]

There are some very clear examples in previous emails, where it matters
during sync to know which particular property was changed - because going
from a list of X values to a new list of Y values conveys no information as
to which item was actually changed or deleted.  What about this was not
understood?  

The only reasonable way for OMA-DS or other sync protocols to be able to
identify specific fields as they move forward in time would be to either add
some huge kludge to number or otherwise reference each line inside a vCard
on our own, or just throw away vCard, and give in to the requests to define
our own data format (like ActiveSync did).

Writing a sync engine is something that other protocols spend many pages
discussing, and which there have even been books written about.  Even the
simplest example of creating a real conflict with small integer pids is
going to be several pages long (involving 4 devices, with changes occurring
on at least two at specific points of time between syncs, as shown in
Javier's email of 5/30) - yet it is relatively obvious that small integers
can have conflicts, so why does this have to be demonstrated?

Given how little information is documented about how to split up name and
address components, and similar issues, this level of detail seems
ridiculous.

Perhaps half of the items above could be discussed in a reasonable amount of
space (3 pages?), but answering every question asked is tens of pages.  I am
extremely hesitant to waste the time on something that will just be removed
again as being too wordy.  Does anyone really need the current set of
examples in the Sync section to be tens of pages?

I provided a specific set of changes to virtually every part of the document
affected by PIDs (in fact, everything but the abnf).  What SPECIFICALLY is
desired beyond that?  At this point, no one has discussed even a specific
sentence of what was provided.  I provided the specific text to implement
what was previously agreed upon - e.g. Simon's email of 6/3, including some
updates to the sync section.

If specific liaisons from the OMA data synchronization group about what is
desired, along with a number of specific examples in emails does not let
people understand the need, then I am not sure what could - you need to talk
to people who have implemented sync engines (servers - not clients).
Otherwise it seems like we are at the point that it has been a couple of
months since everything was discussed, and people have forgotten about it,
and we are starting everything over, which is incredibly frustrating and
wasteful.  If people did not agree to the last round of emails about PIDs,
why wait until now?

dgc

---

From: Aki Niemi <[email-masked]>
Date: Tue, 24 Jun 2008 11:12:47 +0300

> to which item was actually changed or deleted.  What about this was not

First of all, you insist on a per-property ID, yet most of the
discussion is about changes in TEL properties. Surely the PID is
completely superfluous on singleton properties like N or FN, no?

Secondly, the only way to detect deletions with the PID scheme reliably
is to store the entire history in the object, with something like NULL
property values. That's just wrong. I don't want to clutter the data
format with an edit history information. 

I am convinced that if this edit history is really necessary (and I'm
not), that information can be split into a separate diff format; not
textual diff, but on a higher level: "TEL property with value 1234567
was changed to have parameter TYPE=WORK", and so on.

> our own data format (like ActiveSync did).

I can see you're firmly set on the PID solution being the one and only
right solution. However, not everyone is that convinced. And the OMA-DS
is of course free to specify their standards in a way that makes most
sense.

> address components, and similar issues, this level of detail seems

All that is really required is a clear and concise problem statement,
i.e., what is the root of the problem here. I've seen buggy clients,
unreasonable data storage requirements for sync servers, etc mentioned,
but it isn't clear what part of that problem space actually belongs in
the data format work. Clients can be bugfixed, servers and sync
protocols optimized, and so on.

> updates to the sync section.

I think what happened was: You sent a patch for a major pervasive code
change. It compiles, but I have no idea what you're code is trying to
achieve. Therefore, I question the change.

> to people who have implemented sync engines (servers - not clients).

Traditionally, the IETF works by individual contribution. It doesn't
really matter what reads on the badge under a person's name, nor to
which other SDOs they attend.

And usually, writing an internet-draft is the best way to communicate
ideas that are too complex to represent efficiently over email. It also
allows having a single proposal on the table, on which everyone can then
focus their discussion.

> wasteful.  If people did not agree to the last round of emails about PIDs,

Well, the only changes I've seen since then are that the parameter name
was changed and the type size decreased from UUID to int. Those are
purely cosmetic changes, and have done nothing to address the
fundamentals.

Cheers,
Aki

---

From: "Darryl Champagne" <[email-masked]>
Date: Tue, 24 Jun 2008 07:08:41 -0400

> From: Aki Niemi [mailto:[email-masked]]
> Sent: Tuesday, June 24, 2008 4:13 AM
> To: ext Darryl Champagne
> Cc: 'Simon Perreault'; [email-masked]

If you would look at the text I wrote (has anyone?) - I specifically said
that:  "It MUST NOT appear on properties that only may have one instance per
vCard.".  This is a total non-issue.

> format with an edit history information.

Sync Engines can already detect deletions successfully.  That is a total
non-issue.  The primary thing they cannot do well with the vCard format is
to know which item of multiples was deleted.

> was changed to have parameter TYPE=WORK", and so on.

And there was already a thread on how much additional work this would
require for any client or server to support this, and that it would be
impossible to recover from a number of common error situations if it were
done, and thus why it was not feasible.

> is of course free to specify their standards in a way that makes most

I would love to see something else - but things like the per property UID -
what is currently there - are not it.  The edit history proposal was
interesting, but seemed way too verbose, and seemed exactly functionally
equivalent to PIDs.  I am set of not getting anything worse that what was in
the original DS presentation as a suggestion.

> protocols optimized, and so on.

Well, I thought this had already been done, but I guess I will try again.

> achieve. Therefore, I question the change.

Except I sent a patch to specifically implement exactly what was listed by
Simon in his 6/3 email.  I didn't see any progress on it, so I proposed
something.  No more and no less than what was already listed as agreed.
There were no complaints when Simon said he would do that 

" Rename UID parameter to PID. 
- Change PID value type to "a small integer." 
- Adapt examples so that they match the above. "

All I did was supply the text to actually do that, along with the cleanups
that had been suggested in other emails.  Why were there no objections then?

If I was supplying something totally new - then fine, but it isn't.

> which other SDOs they attend.

True.  But it helps to have people knowledgeable on the subject.  If the
proposals from a group of experts on sync are ignored, and replaced with
random guesses by people with different domain knowledge (most of which will
cause every contact to be duplicated regularly), not much is gained.

If people do not understand the implications of sync - mapping between
different formats, the loss of data, the recovery issues, the issues that
the vast majority of real world contact books do not store vCards, yet can
be synchronized with vCard, and what sync engines can do easily (like detect
deletions), and what they cannot, then it is not very useful discussion.

> purely cosmetic changes, and have done nothing to address the

Define the fundamentals.  I have already done threads on the actual issues
with collisions, and the real world odds of collisions, and people keep
forgetting that the way it works today is that every change is the
equivalent of a PID collision.  If people need yet another thread on the
chance of a collision, and the impact, then so be it, but it still seems a
waste.

It especially seems a waste to write stuff, and have no sign of anyone
actually reading it.

---

From: Aki Niemi <[email-masked]>
Date: Tue, 24 Jun 2008 15:23:45 +0300

> vCard.".  This is a total non-issue.

I had missed that, sorry.

> to know which item of multiples was deleted.

They can detect deletions, but not additions or modifications? How is
that possible?

<snip />

> deletions), and what they cannot, then it is not very useful discussion.

And this is exactly what bothers me. It's clear there are sync solutions
deployed today using vCard that arguably work without any per property
identifiers.

How is it that this fundamental change is required at this point?
Assuming clients worked as they should, i.e., didn't lose information
when converting vCard into their internal data format and back, what
exactly is the problem with vCard?

I think there needs to be some fundamental flaw in the data format to
justify PIDs. Bugs in client implementations are not such a flaw for
mainly two reasons: 

      * We've got no guarantees that they will ever implement PIDs in
        the correct way either. In fact, you've said changes generally
        propagate to phones slowly (which might or might not be true ;),
        which actually seems contradictory to proposing extensive
        changes to the data format.
      * The devices on which this new version ends up being implemented
        are a few years away anyway, which makes it futile to optimize
        on current hardware limitations.

> chance of a collision, and the impact, then so be it, but it still seems a

Fundamentals, as in why the heck this new parameter when sync today
seems to be working quite well without it, modulo the client bugs? 

And assuming there was a good justification, the ordered list approach
seems much cleaner than the PID proposal, but it's hard to evaluate
their merits when the problem isn't clear.

> It especially seems a waste to write stuff, and have no sign of anyone

Don't assume I don't read your posts. I've read them, and still don't
understand your point. There's a big difference, and apparently the
situation won't improve by me re-reading your posts (which I also did).

Cheers,
Aki

---

From: Helge Hess <[email-masked]>
Date: Tue, 24 Jun 2008 14:48:42 +0200

Hi,

I agree that PIDs would be quite useful. W/o them you can't do exact  
mapping between vCards and many backends. Which is often good enough,  
but optional PIDs would improve the situation.
I also think that vCard items should be considered ordered. IMHO a lot  
of software already relies on that to do proper mapping.

Example for PIDs: RDBMS based groupware server, which has a table for  
contacts and additional tables for an arbitrary numbers of phone  
numbers / adresses. User N might have permissions to change one phone  
number of the contact, but NOT to another. He can, however read the  
second one (and hence export it to vCard).

   GET /123.vcf

   BEGIN:VCARD
   ...
   TEL:12345		phone record 1
   TEL:54321		phone record 2
   END:VCARD

If we do a PUT with one TEL modified, how can the server know which  
backend record he needs to patch?
I think most syncs imply an ordering, but this does not really help,  
since phone records might get deleted too.

Summary: a way to identify ADRs and TELs reliably would be quite  
helpful. Though personally I don't see how a simple number is  
sufficient. I guess the PID should be an arbitrary string which MUST  
be unique inside the vCard object (eg could be a primary key).

Thanks,
   Helge
-- 
Helge Hess
http://www.helgehess.eu/

---

From: "Darryl Champagne" <[email-masked]>
Date: Tue, 24 Jun 2008 09:35:35 -0400

> Sent: Tuesday, June 24, 2008 8:24 AM
> To: ext Darryl Champagne
> Subject: RE: [VCARDDAV] PIDs proposal
> 

I didn't say they couldn't detect additions or modifications, I said they
cannot identify which particular item of multiples.  I mentioned deletes
because that is what your example was, which I should not have done, sorry.
When an item is deleted and others changed, it is difficult to identify
which one was deleted.  If nothing changes other than a single delete, then
yes, most people can figure it out.  If more than one modification is made,
it can be difficult to figure out.  If modifications and additions are made,
it can be difficult to figure out.

Rephrase to which item of multiples was added, deleted, or modified (for
complex cases).

> deployed today using vCard that arguably work without any per property

And I have worked on 3 of the existing ones over the past 12 years -
Nokia/Intellisync, OpenWave, and Funambol/Sync4J, and Mark has worked on the
Oracle, and several of the other people from DS have worked on a variety of
other ones.

And every one of them out there can easily be broken by doing round-robin
sync, exactly as in the examples that product collisions.   By performing a
series of syncs in the right directions, one can almost always cause every
item in the contact book to be duplicated, or if not that, at least
overwrite a bunch of data.  And the reason for this is the data format
(along with the issue that Microsoft ActiveSync does everything it's own
way, so if you use one product for a couple pairs, and ActiveSync for
another that causes a loop, you start replicating everything).

They all work, mostly, for simple cases of two devices (sometimes even
three) that each talk to a single server.  And to do that, they all have a
large amount of ugly heuristics, and they all have long lists of subtle
failure modes.  They all tend to work because peoples typical use cases are
actually pretty simple - they tend to follow patterns, and we can workaround
people who only do simple things - add new contacts on their PC, and only
add phone numbers for new contacts on their phone, never edit on a different
device, and so on.  It's the people who work on sync clients and servers
that want the change (OMA-DS) - shouldn't that mean something?

> exactly is the problem with vCard?

And here is yet another very false assumption.  

Virtually every client out there - Outlook, Lotus Notes, Windows Mobile
devices, Palm Devices, RIM devices, Nokia phones, SonyEricson phones, and so
on - receives the data as a vCard, and then stores only those fields that
can be mapped into their local data format.  They ALL lose the data they do
not understand, such as UIDs, or more phone numbers than they support.  They
ALL store Sync information separate from the Address Book (which only
includes change log and mapping information, not any of the vCard info), and
have problems when the sync information is lost, yet the address book is
not.  This does regularly happen - when devices are reset, clients upgraded,
or when servers do not see communications from a device for a month or so,
they flush out the device specific data.

If you look at SyncML/OMA-DS, you will see that every device is supposed to
define its device info, including a list of which fields of a
vCard/vCal/iCal they actually support, specifically so the server doesn't
bother to waste the bandwidth to send fields that the clients will throw
out.

Your idea of what clients should do is not even remotely close to what the
product managers of those clients want them to do.  It's great to have a
clear idea of the best way to do things, but if you are trying to impose
changes that are completely ignored, perhaps something else needs to change.

You work for Nokia - how about you try to get just one of their sync clients
to be loss-less? I still haven't seen one that was (I haven't tested some of
the most recent high end ones, though).  

> justify PIDs. Bugs in client implementations are not such a flaw for

If it's a bug that ALL clients except for a very few exceptions cannot store
fields they do not understand (some servers will handle it correctly), and
there is no indication that any of them will change that any time in the
next 3-5 years, then fine - call it a bug, and we need a hack to work around
it.  But we still need it.

>         on current hardware limitations.

The advantage of PIDs is that the clients do not have to do much. If the
transient sync data is lost, then by sending values without PIDs, servers
can do a reasonable recovery.  But in all other cases, adding the overhead
of a single integer allows a reasonable set of problems to be addressed at
very little cost.  If 50% of clients can handle a single extra integer in 3
years, I would think it a moderately big improvement.

And given that most clients out there still use vCal for calendar data
exchange - including more than half of the Nokia devices - then yes - it is
very hard to get changes done.

Try logging what a Nokia phone sends out for sync, or try to sync it until
it is full...

I can get changes into one of our clients that a user has to download.  I
can't get changes into most built-in clients (like Nokia's).

And just for reference - our product line avoids vCard whenever possible.
Our client to our server defaults to using our data format - just like a lot
of other products.  SyncML/OMA-DS does not mean that vCard is actually used.

> > > > months since everything was discussed, and people have forgotten
> about
> > > it,

This is clearly said by someone who does not actually implement or test
sync.  Sync today is considered in the industry to be moderately bad, with
many problems, and cases that cannot be handled, and there is no interest at
all in even trying to fix round-robin sync issues given the underlying data
formats, and IOP issues with each companies workarounds.  

> situation won't improve by me re-reading your posts (which I also did).

If you think that most clients should loss-lessly store vCard data, and
return it back unchanged, you are not dealing with anything related to
today's mobile or PDA world, and it is no wonder our communications miss
each other.

dgc

---

From: Aki Niemi <[email-masked]>
Date: Tue, 24 Jun 2008 17:42:49 +0300

ti, 2008-06-24 kello 09:35 -0400, ext Darryl Champagne kirjoitti:
> not understand, such as UIDs, or more phone numbers than they support.

But you expect them all to store and maintain PID values correctly?

Cheers,
Aki

---

From: Aki Niemi <[email-masked]>
Date: Tue, 24 Jun 2008 17:47:44 +0300

> be unique inside the vCard object (eg could be a primary key).

Stupid question: assuming ordering of the TEL properties, why isn't the
property value unique enough key?

Cheers,
Aki

---

From: "Darryl Champagne" <[email-masked]>
Date: Tue, 24 Jun 2008 11:08:41 -0400

No, I do not expect them all to - it has always been listed as optional -
not required for sync.

I hope that some reasonable percentage might.  I also will be able to ensure
that all of our downloadable clients will.

dgc

> Sent: Tuesday, June 24, 2008 10:43 AM
> To: ext Darryl Champagne

---

From: "Darryl Champagne" <[email-masked]>
Date: Tue, 24 Jun 2008 11:16:09 -0400

> Behalf Of Aki Niemi
> Sent: Tuesday, June 24, 2008 10:48 AM
> To: ext Helge Hess
> Cc: [email-masked]

Ordering addresses some of the cases.

However, several examples were given - especially for the deletes combined
with changes, where it is not sufficient.

E.g.

Tel1=1, speedDial=1 (and other private data)
Tel2=2, speedDial=2 (and other private data)
Tel3=3, speedDial=3 (and other private data)

Delete Tel2
Change Tel3 to have the value 2.

You want to know that Tel2 was deleted, and that the value 2 now goes into
the Tel3 speeddial position.

Again - speeddial is an easy to recognize example - there is plenty of other
things that are affected in the same way, such as user interface options.

---

From: "Darryl Champagne" <[email-masked]>
Date: Tue, 24 Jun 2008 11:43:30 -0400

> Behalf Of Helge Hess
> Sent: Tuesday, June 24, 2008 8:49 AM
> To: [email-masked]

Definitely.  I saw one earlier suggestion to add something to explicitly
state that ordering was meaningful.  That might be more accurate when
devices recreate a vCard from their locally stored data - they usually
generate it in a consistent order, but it might not match what they were
originally given.

> be unique inside the vCard object (eg could be a primary key).

I'm not sure what level of uniqueness you mean - If you are just referring
to unique within this particular vcard at this particular point in time -
yes, that is a reasonable restriction.  I was being slightly less strict, in
saying that it is unique for the particular type of data - I don't care if
there is a PID=3 on both a Tel and an Email.

The main trade-off I see is extra work/storage vs. actual value.  During a
sync, the sync engine knows if it is receiving new information or not, and
if it has locally changed information (it might not know which parts have
changed, however).  For many typical cases, sync engines can already make a
reasonable guess at what has actually occurred.  It is only the complicated
cases, such as changing one entry to hold a value that was originally in
another entry that really cause problems.  Even for those changes, it is
only the really complicated scenarios, such as 4 devices making round robin
changes, with two of them ending up with a colliding PID that a small
integer is not enough.

So to me - if there was a better solution, great - but we seem to have a
situation where:

90-95% of the cases usually work already, just by server making intelligent
guesses (which does require a fair bit of work for the developer).

Of the 5-10% that might have a problem, the server usually has either a 50%
or 33% change of a random guess being right even if nothing is done.  Thus
end users see a 95+% success rate already.

If small integer PIDs are added, then I would expect 99%+ of those 5-10%
cases to now be addressed.  In many environments, they would not be possible
at all (since few users really type concurrently on multiple devices, let
alone trying to modify the same data entry on each, so if the syncs are
quick enough, it can't happen).  In the remaining <1% of the 5-10%, a PID
collision means the server has to just make a best guess, and have a 33% or
50% chance of being lucky.  

It seems like a huge improvement, at very minimal cost.  It also seems to be
well down the slope of diminishing returns - I can probably break it if I
know what is going on and try to, but it is unlikely to happen for real.
Since it's not a random chance - it's not each user hitting it once every X
events, but rather a user hitting a long sequence of things exactly right in
very complicated scenarios to perhaps see a change missed (and operator
error was probably way more likely), it seems questionable how much
refinement is really needed (unless it doesn't impose significant additional
cost).

---

From: Helge Hess <[email-masked]>
Date: Tue, 24 Jun 2008 17:50:06 +0200

> property value unique enough key?

a) there can be multiple TEL properties with the same number

b) the property value changes and the vCard does not contain
    the old value? how can the server detect which value was
    changed?

c) what if a client deleted one value and changed another. Ordering
    is b0rked then.
       TEL;PID=1:12345      TEL:12345
       TEL;PID=2:23456  =>  TEL:83723 (is it PID 2 or 3?,
       TEL;PID=3:34567                 which one got deleted)

d) remember that ordering cannot be stable on the server either (phone
    records might get deleted, which would then introduce 'holes'
    in the ordering, etc)

e) again: TEL/ADR objects on the server might be self-contained
    resources with own access limitations, auditing, etc,
    so its important to find the proper record.

If its really helps with understanding the issue, I could try to come  
up with a more complete scenario. But isn't the issue somewhat  
obvious? :-)

Ordering works OK in many many scenarios, but optional PID addressing  
would make consistent updates much easier. Keeping ADRs separate from  
the core contact information is very common in contact databases.

Greets,
   Helge

---

From: Helge Hess <[email-masked]>
Date: Tue, 24 Jun 2008 18:01:57 +0200

> there is a PID=3 on both a Tel and an Email.

Yes, sure. A PID should be unique inside a single vCard for each type.  
Thinking about it, this might not even be necessary. If PIDs are the  
same, regular ordering semantics would be used (in fact the situation  
"no PID" could be specified as PID defaults to "").
In fact the 'WORK' and 'HOME' types are actually used like PIDs in  
implementations, they are just not fine grained enough.

> The main trade-off I see is extra work/storage vs. actual value.

To be honest I can't follow you completely, I also came late into the  
discussion anyways. The question is who assigns the PIDs. If the  
implementors use small ints to please very small devices, its OK, but  
why would the standard add such a limit? That does not sound sensible  
to me.

A sync server like Funambol might also want to rewrite PIDs delivered  
by client/server like systems if its talking to small devices. But  
thats really a device specific optimization which should not be in the  
standard.

Greets,
   Helge

---

From: Kepeng Li <[email-masked]>
Date: Tue, 24 Jun 2008 09:17:46 +0200

-1

-----邮件原件-----
发件人: [email-masked] [mailto:[email-masked]] 代表
Chris Bryant
发送时间: 2008年6月23日 15:31
收件人: [email-masked]
主题: Re: [VCARDDAV] PIDs proposal

+1

USA.NET 

This message is for the sole use of the intended recipient(s) and may
contain confidential and/or privileged information of USA.NET, Inc.  Any
unauthorized review, use, copying, disclosure, or distribution is
prohibited.  If you are not the intended recipient, please immediately
contact the sender by reply email and delete all copies of the original
message.

---

From: "Darryl Champagne" <[email-masked]>
Date: Tue, 24 Jun 2008 18:27:32 -0400

> Sent: Tuesday, June 24, 2008 12:02 PM
> To: [email-masked]

I actually do not have a problem with it being a string, and the standard
not having such a limit - the issue is just that the minute we start talking
about something not specified as short, then people keep jumping in with the
"it must be globally unique to avoid collisions" theme, which has gotten
very tiresome, and defeats the purpose.  

A lot of this deals with human nature - Something that is always long would
be hard to justify supporting in a small device - I would expect an
automatic rejection to bother with supporting it.  Since that the majority
of devices still use vCal instead of iCal, I would want it to be a painless
transition - make this minor change to improve sync.  Note that there wasn't
as much resistance between vCard versions - devices kept vCal so they didn't
have to touch any recurrence logic, but they didn't see any real big
difference between vCard versions (at least for the fields they cared
about), so at least some used 3.0.  

If PIDs are easy enough, then I can see developers just doing the right
thing - spend a day, and now support it.  But if supporting PIDs ever showed
up as an issue - the QA department starts reporting that as of a certain
build, the maximum number of contacts dropped significantly - then it would
be forced to be ripped back out.

If one looks closely at my proposed text (6/21), you will actually see that
I did not limit the length - just stressed that it is not globally unique,
and only used very simple examples - PID=1, PID=3.  I actually did not even
require it to be locally unique, just implied that for it to be useful it
would be.  I could possibly see that in confusing cases - an implementation
received colliding PIDs, and it just sends the colliding information to
someone else, hoping that they would figure it out.  Not something I would
advocate - a SHOULD be locally unique is probably better.

Really, I see only using the simple examples as the biggest issue.

> thats really a device specific optimization which should not be in the

Yes, I would expect that to be done if for some reason large values were
being used by someone else, but it would interfere with round-robin sync if
the device we sent small PIDs to ever synced directly with whoever gave us
the large values.  The danger is that if that other sync ever happened, you
would now duplicate every value - have a phone number for the large PID and
a second copy for the small PID.  That's a big problem for contact sync -
most of the time it works, but hit a subtle combination of things, and you
just duplicated everything, which really ticks the user off.

---

From: "Javier Godoy" <[email-masked]>
Date: Tue, 24 Jun 2008 20:23:33 -0300

Sorry I'm joining this discussion quite late

On Tue, 24 Jun 2008 18:01:57 +0200 Helge Hess wrote:

> The question is who assigns the PIDs. If the  implementors use small ints
> to please very small devices, its OK, but  why would the standard add such
> a limit?

I think it is because of two reasons (in addition to those already commented 
by Darryl):

- Long PIDs impose a significative overhead (about 50% of PID and 50% of 
data, if not worse). If some devices does only allow few instances of a 
property type in each vCard, probably they will not support arbitrarly long 
PIDs and that will be a serious interoperability problem. There is no 
recommended length, because short PIDs are

- We are talking about information about people: it does not changes 
frequently, then a small value space may be suitable and collisions should 
not be too frequent. Furthermore, there are some situations which do not 
benefit from longer PIDs: if you add the same phone number in two devices, 
they would be assigned with different PIDs even though your intention is 
that both are the same property... other algorithms are required in this 
case for inferring that two properties with different PIDs (or a property 
with PID versus other property without PID) are actually the same one.

Best Regards

Javier