Subject: [VCARDDAV] Core vCard requirement for Synchronization

From: "Darryl Champagne" <[email-masked]>
Date: Thu, 26 Jun 2008 12:15:12 -0400

Hi all,
Since there seems to be recurring discussion related to this (and I do not
like/am not good at, repeating myself, or figuring out what background
information is not available to other people), and it is a fundamental
point, I would like an explicit, preferably agreed upon, statement of
synchronization needs related to vCard.

--
Proposal: That vCard extensions for synchronization need to accommodate
situations where the vCards are stored on systems that cannot accurately
return the original vCard they are given.
--

This requirement is based upon the current market situation, along with what
is expected in the medium-term (3-5 year) future - that a significant
presence in the Synchronization market will need to work systems that do not
return the full, original, correctly ordered vCard that they were first
given.

--
In specific Detail, as justification, and background information for people
not already aware:

The following devices/software systems (among many others), which have
address books and other data that are frequently synchronized, which have
significant market presence, and all share common characteristics:
- Microsoft Outlook/Exchange
- Lotus Notes/Domino
- Windows Mobile/CE Devices
- Palm OS devices
- RIM devices
- Nokia devices(*)
- Sony Ericcson devices
- Motorola phones
- LG phones
- SyncEvolution
- SugarCRM
- Mobile Phone SIM cards
(and many others, without any intention of leaving other significant market
presences out)

(*) Note: There may be newer models not yet reviewed with different
characteristics, however certainly the numerical majority fit.

Note: There also may be specific versions of any of these that this author
is not aware of, which may have different properties.  Some information is
second-hand.  This is not a claim to know every detail of every system.

These shared characteristics include:
- They store address book information in an internal format.
- The internal format does not support the addition of arbitrary new objects
in a reusable fashion.  Note that some software systems may allow for
additional data, but it would not be generally identifiable as the original
vCard.
- Have Sync clients available, that are independent of the
application/device

Because these systems do not store and manipulate the original vCard
internally, sync clients need to store contact information in the native
data format.  Any additional information related to what original vCard was
sent thus ends up stored as private data, not accessible to other
applications.  Note that storing the original vCard in additional to the
native data can cause a significant increase in storage requirements, so
there is a strong dis-incentive towards doing so.

The vast majority of these systems can synchronize via multiple mechanisms -
e.g. a phone has a built-in sync client, and several companies provide sync
clients that may be downloaded.  Thus private data that was stored by one
sync application is not available or usable for other sync applications.

Thus it is common practice for:
- Systems to return vCards without the information they do not understand,
since they did not store data they cannot use.
- Systems to return vCards that might not preserve the original order of
data, because the recreate the outgoing vCard based upon the native data
format.

Additionally it is possible, although not common for:
- Systems to be synchronized by a different client from one sync to the
next, thus having no access or understanding of previous private data.
- Systems to be synchronized by a client that has been upgraded or reset
from one sync to the next, thus losing prior private data.
- Systems to remove data that may be recovered, when low on resources. For
example, a sync server may delete all data other than the actual address
book relative to a particular client if it has not seen a sync from that
client in some period of time (such as a month).  Similarly, a restricted
device may need to free up memory.

These issues also cause systems to return vCards without previous
information, or order of information.

Note that some of these systems can synchronize via non-vCard data - e.g.
Microsoft ActiveSync between a PC running Outlook, and a Windows Mobile
Device.  Sync engines that then communicate with these systems must be able
to identify data on both that should be considered common, even if device
limitations mean that the data is currently slightly different.

--

If people feel that vCard updates should not have to deal with any of those
systems, or other systems that do not store and manipulate vCards directly,
or that vCard updates should only work with systems that update to store the
entire vCard, I'd rather hear a -1 on that now, than as future objections.

Thanks,
dgc
--
Darryl G. Champagne
Technology Evangelist
Funambol
AIM/Y!/Skype: dgc03052
ph: +1 (603) 424-0323     Mob: +1 (603) 440-3621 (+ SMS)
[email-masked]          http://www.funambol.com 

_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: Mark Paterson <[email-masked]>
Date: Thu, 26 Jun 2008 12:25:09 -0400

+1, this has to be a requirement.

-- 

Oracle Email Signature Logo
Mark Paterson | Product Manager, UC & Mobile Services | 514.905.8649
Oracle Server Technologies - Collaboration
600 Blvd. de Maissoneuve West, Suite 1900, Montreal, Quebec, H3A 3J2
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: <[email-masked]>
Date: Fri, 27 Jun 2008 11:35:03 +0300

+1
It's about time to put things into perspective. 

Best regards: Zoltán Ördögh
E-mail: zoltan dot ordogh at nokia dot com
Phone: +358 50 386 0566

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of ext Darryl Champagne
Sent: 26 June, 2008 19:15
To: [email-masked]
Subject: [VCARDDAV] Core vCard requirement for Synchronization

Hi all,
Since there seems to be recurring discussion related to this (and I do not like/am not good at, repeating myself, or figuring out what background information is not available to other people), and it is a fundamental point, I would like an explicit, preferably agreed upon, statement of synchronization needs related to vCard.

--
Proposal: That vCard extensions for synchronization need to accommodate situations where the vCards are stored on systems that cannot accurately return the original vCard they are given.
--

This requirement is based upon the current market situation, along with what is expected in the medium-term (3-5 year) future - that a significant presence in the Synchronization market will need to work systems that do not return the full, original, correctly ordered vCard that they were first given.

--
In specific Detail, as justification, and background information for people not already aware:

The following devices/software systems (among many others), which have address books and other data that are frequently synchronized, which have significant market presence, and all share common characteristics:
- Microsoft Outlook/Exchange
- Lotus Notes/Domino
- Windows Mobile/CE Devices
- Palm OS devices
- RIM devices
- Nokia devices(*)
- Sony Ericcson devices
- Motorola phones
- LG phones
- SyncEvolution
- SugarCRM
- Mobile Phone SIM cards
(and many others, without any intention of leaving other significant market presences out)

(*) Note: There may be newer models not yet reviewed with different characteristics, however certainly the numerical majority fit.

Note: There also may be specific versions of any of these that this author is not aware of, which may have different properties.  Some information is second-hand.  This is not a claim to know every detail of every system.

These shared characteristics include:
- They store address book information in an internal format.
- The internal format does not support the addition of arbitrary new objects in a reusable fashion.  Note that some software systems may allow for additional data, but it would not be generally identifiable as the original vCard.
- Have Sync clients available, that are independent of the application/device

Because these systems do not store and manipulate the original vCard internally, sync clients need to store contact information in the native data format.  Any additional information related to what original vCard was sent thus ends up stored as private data, not accessible to other applications.  Note that storing the original vCard in additional to the native data can cause a significant increase in storage requirements, so there is a strong dis-incentive towards doing so.

The vast majority of these systems can synchronize via multiple mechanisms - e.g. a phone has a built-in sync client, and several companies provide sync clients that may be downloaded.  Thus private data that was stored by one sync application is not available or usable for other sync applications.

Thus it is common practice for:
- Systems to return vCards without the information they do not understand, since they did not store data they cannot use.
- Systems to return vCards that might not preserve the original order of data, because the recreate the outgoing vCard based upon the native data format.

Additionally it is possible, although not common for:
- Systems to be synchronized by a different client from one sync to the next, thus having no access or understanding of previous private data.
- Systems to be synchronized by a client that has been upgraded or reset from one sync to the next, thus losing prior private data.
- Systems to remove data that may be recovered, when low on resources. For example, a sync server may delete all data other than the actual address book relative to a particular client if it has not seen a sync from that client in some period of time (such as a month).  Similarly, a restricted device may need to free up memory.

These issues also cause systems to return vCards without previous information, or order of information.

Note that some of these systems can synchronize via non-vCard data - e.g.
Microsoft ActiveSync between a PC running Outlook, and a Windows Mobile Device.  Sync engines that then communicate with these systems must be able to identify data on both that should be considered common, even if device limitations mean that the data is currently slightly different.

--

If people feel that vCard updates should not have to deal with any of those systems, or other systems that do not store and manipulate vCards directly, or that vCard updates should only work with systems that update to store the entire vCard, I'd rather hear a -1 on that now, than as future objections.

Thanks,
dgc

---

From: Kepeng Li <[email-masked]>
Date: Fri, 27 Jun 2008 07:44:55 +0200

+1

  _____  

发件人: [email-masked] [mailto:[email-masked]] 代表
Mark Paterson
发送时间: 2008年6月26日 18:25
收件人: Darryl Champagne
抄送: [email-masked]
主题: Re: [VCARDDAV] Core vCard requirement for Synchronization

Darryl Champagne wrote: 

Hi all,

 

 

 

 

--

Thanks,
dgc

 

--

---

From: Simon Perreault <[email-masked]>
Date: Fri, 27 Jun 2008 10:07:57 -0400

> return the original vCard they are given.

+1, and I don't even see that as a bug. It's perfectly fine to use vCard as a 
temporary interchange format rather than as a storage format.

So far we've discussed a few ideas:

- UUID
- Small integer
- Opaque string

All those are compatible with your proposal, are they not?

-- 
Please try Numb, a STUN/TURN server implementation.
Free access at http://numb.viagenie.ca/.
_______________________________________________
VCARDDAV mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/vcarddav

---

From: Marc Blanchet <[email-masked]>
Date: Fri, 27 Jun 2008 10:16:28 -0400

> temporary interchange format rather than as a storage format.

in fact, most implementations are not storing vcards in the native  
vcard format. Therefore, we can assume that vcard is (just) an  
interchange format. If it is also used as storage, it is a  
"coincidence", a nice property but the storage requirement should not  
be taken into account into the design of the interchange format.

my 2 cents.

Marc.
-----
IPv6 book: Migrating to IPv6, Wiley, http://www.ipv6book.ca

---

From: "Darryl Champagne" <[email-masked]>
Date: Fri, 27 Jun 2008 11:57:17 -0400

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Simon Perreault
> Sent: Friday, June 27, 2008 10:08 AM
> To: [email-masked]
> Subject: Re: [VCARDDAV] Core vCard requirement for Synchronization
> 

Do we really need to repeat this, yet again?
Theoretically (*), Yes, Yes, Yes.
Practically (*), No, Yes, Yes if small.

(*) And none of them are compatible based upon the prior normative text,
only with the most recent proposal, not yet published.

And the key to "small" being the storage overhead needs to be small relative
to the specific property's data.

What is not compatible with my proposal are various third party statements
like "Assuming clients worked as they should, i.e., didn't lose information
when converting vCard into their internal data format and back".  

I wished it to be explicitly noted that we need to support clients that
cannot return a vCard without information loss relative to what they were
originally given.

This came up a month ago, and it came up again recently.  When it comes up
again, I don't want to have to repeat myself.

Collision probabilities also came up a month ago, and recently, which may
needs its own thread, rather than being embedded in other discussions, to
prevent it from coming up again in another month (or at least being able to
refer to a single email, rather than having to repeat the same stuff over
again), but that's another issue.

dgc

---

From: Simon Perreault <[email-masked]>
Date: Fri, 27 Jun 2008 12:12:35 -0400

On Friday 27 June 2008 11:57:17 you wrote:
> > > situations where the vCards are stored on systems that cannot
> > > accurately return the original vCard they are given.
> >

Ok, now I don't agree.

The reason UUID have been rejected is that the makers of those small devices 
*just won't implement it* because it's too big. Small integers, on the other 
hand, will probably get implemented because they're small.

It's as simple as that, and it has nothing to do with the fact that devices 
don't return the vCard exactly as it was given to them. Even a standard PC 
doesn't return the vCard exactly as it was given to it.

---

From: Mark Paterson <[email-masked]>
Date: Fri, 27 Jun 2008 14:33:37 -0400

> hand, will probably get implemented because they're small.
Actually I think it is more that they have historically have only 
supported vCard 2.1 I see no reason to change. I don't think the size of 
UUID has had anything to do with vendors not supporting UUID which is in 
vCard 3.0. Vcard 3.0 just doesn't do anything for them to feel the need 
to switch anything..

Mark

--

---

From: Simon Perreault <[email-masked]>
Date: Fri, 27 Jun 2008 14:36:09 -0400

> vCard 3.0. Vcard 3.0 just doesn't do anything for them to feel the need

I was talking about UUID the PID parameter value type, not UID the property.

---

From: Mark Paterson <[email-masked]>
Date: Fri, 27 Jun 2008 15:51:33 -0400

Yep sorry but now you've confused me. You commented on why you think 
vendors haven't implemented something that isn't in the spec yet?

--

---

From: Simon Perreault <[email-masked]>
Date: Fri, 27 Jun 2008 15:52:58 -0400

> vendors haven't implemented something that isn't in the spec yet?

I did use the future tense:

---

From: Mark Paterson <[email-masked]>
Date: Fri, 27 Jun 2008 16:08:12 -0400

Device Manufacturers will move to vCard 4.0 if they see a compelling 
market reason to do so. If they are pushed by operators such as Orange 
Vodafone, etc.. to provide a better sync client that doesn't end up in 
duplicated phone numbers and a vCard 4.0 solution (whether it be OMA DS 
or CardDAV based) provides a way to solve this issue they will do it. 
The size of a UUID vs a PID isn't really going to be what makes them 
decide I suspect. Having said that I prefer not to see big huge UUIDs 
every where in vCARD. I'd like to see implementors using UIDs (although 
making them a MUST is not reasonable I suspect)  and smart use of PIDs 
but let's be careful about presuming we know why device manufacturers 
may or may not do something.

Mark

--

---

From: Simon Perreault <[email-masked]>
Date: Fri, 27 Jun 2008 16:27:19 -0400

> may or may not do something.

This is absolutely *not* what I was talking about. Let me try again:

- We have been examining 3 value types for the PID parameter: UUIDs, small 
integers, and opaque strings.

- Darryl states that UUIDs are (-1) because many devices cannot reproduce the 
original vCard as it was given to them.

- I say that's not the real reason, since even PCs cannot reproduce the 
original vCard as it was given to them. The real reason is that UUIDs are too 
big for the devices in question and won't be implemented.

- My point: the fact that many devices cannot reproduce the original vCard as 
it was given to them *has no impact whatsoever* because it has never been a 
prerequisite.

---

From: "Darryl Champagne" <[email-masked]>
Date: Fri, 27 Jun 2008 18:13:33 -0400

I really don't appreciate this quoting out of context nonsense.  Say what
you want as your opinion, but *do* *not* *misrepresent* *mine*.

I was very explicit about the size burden being the issue of a UUID being
attached to every property - "In this environment the requirements to store
additional data need to be minimized" and "Imposing a requirement for a UID
adds additional data of comparable size to the data being stored, which is
not practical for limited devices" - when I started the thread "[VCARDDAV]
UID/PID problems" on 5/29, because of lack of action.  Saying that I only
claimed that it was because clients could not store the vCard is completely
misleading, since that was just the start of an explanation as to WHY the
space mattered.  Saying that you are right and I am wrong because they are
too big - when that is what I said long before you, the person who put them
in - is flat out deceptive.  And in actual fact, the first person to
actually say that was Cyrus, on 1/22, unless you count the OMA-DS
contribution which suggested small numbers, which was from well before that.

The current wording is at least reasonable, although not ideal.  Clearly the
suggestions I make will not be incorporated without pages of text providing
absolute proof, so I need not bother to try and refine anything.  Stop the
rest of the nonsense unless there is a real issue.  If the point doesn't
involve a change to the spec, why is it on the mailing list?

I don't really care if you agree or not, or if you think you are the
ultimate guru of sync, vCard, or programming in general, or can't finish a
thread unless you are the only one right or whatever.  I care when it wastes
my time having to explain the absolute basics of sync, over and over, which
I am very tired of (and in general, I don't mind questions made in
ignorance, it's claiming expertise that isn't there that really bugs me).

I care if vCard 4.0 is going to be something that stands a chance of being
adopted, and I care about trying to get enough improvements that I can
justify to my boss the time spent on this, which is currently considered
mostly a waste (and frankly, anyone looking at the last 8 or 9 emails would
probably agree).

I have also worked with Mark for enough years to respect his opinions
(whether I agree or not :-; ).  I completely agree with him in that device
manufacturers have seen no reason to bother to change away from vCard 2.1.
Clearly the availability of UID (the property) was not a compelling reason
for them to switch to vCard 3.0, which makes me doubtful that it would help
push vCard 4.0 very much (sorry to disagree slightly with Cyrus).  The
availability of a PID might help, but I considered having every single PID
be a full uuid value as a strong dis-incentive to adoption, and that is what
I stated, which I think is pretty close to Mark's statement "prefer not to
see big huge UUIDs every where in vCARD " - with his context below (and
Cyrus had a similar statement back in January).

I also fully agree with Cyrus that "4.0 really does need to show some clear
benefits to be adopted".  I wish PIDs and UIDs would be considered part of
strong arguments, but I have been disappointed before - right now I am just
hoping that we can show a moderate benefit with little cost, to perhaps get
some adoption.

If my views are misrepresented, or further untruths stated I will respond,
or if there are actual issues with the spec, but right now this whole thing
has just been a total waste of time, and I hope to be done with it.  Anyone
who wants to know who understands what about sync need only review these
threads from the beginning, rather than looking at the final
misrepresentations.  And if this means I look mean, well, better that than
having to re-explain every little thing in pages of text from the most basic
point month after month to get simple little text changes in, like getting
rid of a totally inappropriate MUST NOT (which of course, still isn't out of
a posted draft).  If there is some point to be made that impacts the spec,
then make it, otherwise just stop.

dgc

> Sent: Friday, June 27, 2008 4:27 PM
> To: [email-masked]