Subject: Re: Can we fix BYSETPOS?

From: Alan Davies <[email-masked]>
Date: Fri, 08 Jun 2001 16:49:16 -0400

At 07:21 PM 18/05/2001 -0400, you wrote:
>One further comment on why BYSETPOS is horribly broken.
>
>RECUR:FREQ=YEARLY;INTERVAL=100000000;BYSECOND=0,1,2,3,4,5,6,7,8,9,10,11,12,13,
>         14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,
>         37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59;
>         BYSETPOS=-1
>
>is a legal RFC 2445 recurrence.  It expands to approximately three
>quadrillion events before BYSETPOS is applied.  Good luck...

[...]

>Yes, this is clearly pathological.  But how well can it be prevented?  And
>what should a calendaring program do upon receiving it?

Why should we prevent this from happening? Many useful languages and data
formats can express absurdities/contradictions. Unless arbitrary restrictions
are placed all over iCalendar, it'll always be possible to create an
'absurd' object.

Without arbitrary restrictions, each implementor has to decide what they
consider to be absurd. If you send me an object containing the
"BYSECOND=0,1,2,3,4,..." recurrence rule, you'd get the same response
that you would if you sent me three quadrillion RDATEs. Something along
the lines of:

REQUEST-STATUS:3.1;Ridiculous property value;RECUR:[...]

or maybe

REQUEST-STATUS:3.2;This system is not yet fully Y100M compliant;RECUR:[...]

Also, removing the BYSETPOS=-1 from the rule doesn't make it any less absurd,
I don't think many CUAs can handle events in the year 100M. The rule is
only absurd in our problem domain though; with the universe estimated
to be 12 billion years old, so an Astronomer might want to express information
about events far in the future. How else could you see the opening times
of the "Restaurant at the End of the Universe" in a SKiCAL object?

Of course, an iCalendar 'date-fullyear' can have only 4 digits (a restriction
that I don't like- and I'm sure Astronomers and Historians wouldn't be too
pleased about this either). Perhaps your RRULE could have been considered
invalid because it would be impossible to express the recurrence as an RDATE.
(perhaps this should be noted in the iCalendar spec, if the restriction 
remains)

--Alan

---

From: Jonathan Lennox <[email-masked]>
Date: Mon, 11 Jun 2001 10:06:59 -0400 (EDT)

> REQUEST-STATUS:3.2;This system is not yet fully Y100M compliant;RECUR:[...]

As we've discussed before on these lists, we have two choices when it comes
to dealing with absurd statuses:

1. Let each implementation decide what it isn't going to handle.
2. Let the standard limit what implementations have to support.

The problem with choice 1 is that it effectively abandons interoperability.
If I generate a calendar object, I have *no idea* whether any given server
is going to be able to deal with it correctly, or is going to reject it.

Remember, RFC 2445 is a data format.  You're not always running in an
environment where you're going to get back a REQUEST-STATUS to try to
negotiate a sensible representation.  Consider an "Export as iCalendar"
option, for instance.

Thus, the track I've been advocating -- and I think a lot of the other
people here have agreed with me -- is that we instead have to limit the
things the specification allows.

-- 
Jonathan Lennox
[email-masked]

---

From: Alan Davies <[email-masked]>
Date: Mon, 11 Jun 2001 11:54:47 -0400

>things the specification allows.

If this were true then there would be absolute limits to everything
within iCalendar- maximum number of RDATES, ATTENDEES, max components in
an object, maximum size of an object, etc.

If a (valid) iCalender object was created that contained, say, several
years worth of all employees' agendas for a large corporation, it
could feasably be 100's of Gigabytes in size. You'd still have *no
idea* whether or not anyone would be able to import this object,
or without pared down RRULEs.

(I don't think there's anything wrong with simplifying RRULEs, I just
don't think that preventing absurdity is a valid or applicable reason)

--Alan

---

From: "Shannon J. Clark" <[email-masked]>
Date: Mon, 11 Jun 2001 11:53:50 -0500

To follow up on this thought.

The current standard is highly vague in many areas - which leads us to see
at present, and to predict in the future, issues around interoperability and
varying interpretations of certain types of complex (and perhaps even
not-so-complex) structures - especially around RRULES.

Indeed it is the case that in use cases that involve direct interaction
between C&S systems these cases could be negotiated by way of messages like
"too complex for me to deal with" or "non-sensical rule detected" - however
iCalendar is intended as a DATA standard - which means that it should be
usable as an interchange structure ABSENT messages and communications. It
should be usable as a format to STORE calendaring data for interchange
BETWEEN tools. This should be doable in a NON-INTERACTIVE manner - i.e.
store to fixed media and move to another machine absent communications
between the two Calendaring and Scheduling systems.

Perhaps this is not a primary case that many application developers have
been considering - certainly the issues that common tools like MS Outlook
have with multi-event iCalendar files shows that the primary use case has
been perceived as a message encapsulating ONE object - but it seems like a
highly valuable and useful use case is an ability to STORE an entire
calendar structure to a persistent media in a format that other tools can
read and interpret in a useful and consistent manner.

Shannon

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Jonathan Lennox
Sent: Monday, June 11, 2001 9:07 AM
To: Alan Davies
Cc: [email-masked]
Subject: Re: Can we fix BYSETPOS?

--
Jonathan Lennox
[email-masked]

---

From: "Shannon J. Clark" <[email-masked]>
Date: Mon, 11 Jun 2001 12:39:29 -0500

To build on this thought...

There is a distinction that we should keep in mind when thinking about and
suggesting limitations and/or restrictions built into the standard.

Certainly the standard should be able to stretch from handling a
representation of ONE event taking a few K of space to store to handling
millions of user's information for many years (your multi-gigabyte example).

IF the large scale is built by adding individual entries this should be
feasible - there are significent programming and database issues from the
implementer's perspective - some of which do suggest certain modifications
to the standard to help accommodate this situation - but the standard should
be able to handle high use situations as well as low-use cases. (for
example, in a large scale implementation with many calendars in the calendar
store extracting vTimezone information into a baseline set and avoiding
excess storage and search requirements in the various vCalendars may make a
lot of sense, this is fine as long as the interfaces to the calendar store
correctly generate full iCalendar operations in interoperability cases -
i.e. when exporting a given vCalendar the vTimezones required by that
particular vCalendar are added back)

That said, this is a case of scaling up, not a case of avoiding absurdity.
Avoiding absurdity is looking at a couple of main questions.

One - Does the structure suggested represent something even potentially
useful? Especially important is does it somehow define or encapsulate
information NOT captured in alternative representations (see the difference
between storing a START and END time vs. storing a START time and a
duration - they result initially in displaying the same event, but they
encapsulate different information - by storing (and thus fixing a duration)
behavior MIGHT differ should the start-time be changed. This is probably a
case that should be clarified in the standard.)

Two - Is it possible for User Interfaces to BOTH enter this case AND display
this case in a consistent manner. Translation - is interpreting this case
possible or is it ALWAYS vague?

Three - If there are MULTIPLE possible interpretations - which suggests that
different vendor's tools will make differing interpretations - we break
interoperability. This means the standard should IDENTIFY these cases and
DEFINE correct behavior (see issues such as whether ONE DAY == 24 HOURS, or
RRULES defining a reoccurring event on a non-existent day or time)

Basically the standard should define what minimum set of features and
functions are required by a calendaring information storage system -
optional features should truly be optional and required features truly
required. The major issues come up where there is a great deal of room for
interpretation - a standard should avoid interpretation issues as much as
possible.

Shannon

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Alan Davies
Sent: Monday, June 11, 2001 10:55 AM
To: Jonathan Lennox
Cc: [email-masked]
Subject: Re: Can we fix BYSETPOS?

--Alan