Subject: Re: Can we fix BYSETPOS?

From: [email-masked]
Date: Thu, 26 Jul 2001 15:55:28 -0400

Jonathan Lennox wrote on 20-Jun-2001:
>We really need clear, formal semantics defined for all of the RRULE 
cases.
>Ideally, a pseudo-code algorithm should be given for it.

W/o waiting to read all the massive backlog I see I would simply ask if 
Toms postings helped clarify this better from my example?  He was going to 
clarify my pseudo-algorigthm  from my earlier examples.  If not, then we 
can revisit this but I expect that his postings should be useful in 
providing some basis for the text you desire.

Bruce
===========================================================================
Bruce Kahn                                INet: [email-masked]
Iris Associates                          Phone: 978.392.5335
Westford, MA, USA 01886                    FAX: and nothing but the FAX...
Standard disclaimers apply, even where prohibited by law...

---

From: [email-masked]
Date: Thu, 26 Jul 2001 18:17:32 -0400

Jonathan Lennox wrote on 11-Jun-01:
>This also means that you need a clear distinction between "original-X" 
and
>"extended-X."  Having them shade vaguely into each other is a bad idea.

Thats why we defined the ABNF to use iana-token and x-tokens so that  an 
older parser would at least be able to parse the formalized extensions 
(iana-token) and the experimental or custom ones vendors add on for 
assorted reasons (x-token). 

That is also why we have VERSION that allows for minver and maxver.  An 
older "2.0" client may not fully grok the new "2.5" objects intent but if 
that object says that minver is 2.0 then the 2.0 client should be able to 
take some reasonable action on that object. 

Any revisions or extensions to 2445 should (will??) be done is such a way 
as to minimize the possible disjoin.

>>                                        But the logic that "If my engine 

>> does not like Feature X of a protocol for some reason so it wont be 
able 
>> to interop with others that do" is simply a myth if the protocol plans 
>> ahead and provides ways for conveying this back (hence some of the 
>> REQUEST-STATUS values in iTIP).
[Snip]
>Are you saying I shouldn't be able to e-mail you an iCalendar object, or 
to
>put it on my web page?

Absolutely NOT.  I fully expect to be able to receive iCalendar objects 
via eMail, Web site, FTP, etc.  What I am saying is that claiming 
interoperability wont happen just because any particular engine cannot 
deal with a 'feature' of iCaledar is just a myth.  If that were a trueism 
then MIME extensions would never have been possible.  However MIME lays 
out expected behaviour for 'unknown' types and subtypes so that MUAs can 
still 'interoperate' just fine.

>Oh, absolutely.  I reached this point of view by worring about entries
>generated by actively hostile adversaries, who want to launch a
>denial-of-service attack on my server by feeding me input that makes me 
fall
>over.  If you're not worried about this case, things are not nearly so 
much
>of a problem.  (Though I would say that current experience in the 
Internet
>seems to indicate that you *do* have to worry about this.)

Well Id hope that you would have considered putting in some kinds of 
filters to prevent just anyone from randomly sending iCalendar objects 
that your CPL engine would just blindly accept and process.  These could 
be anything from requireing authentication of the user to ECLs to quotas 
on the CPL that someone can send (assuming you want to prevent internal 
'bombings' from leaving employees, etc).

I have yet to be convinced that the data format is the place to put such 
kinds of controls/filters/security (pending CAP).  I would suggest that 
these kinds of concerns/constraints are not iCalendar issues currently 
(again, pending CAP) and that the responsible parties should be dealing 
with it in an appropriate manner.

>> For the CPL case the CUA / CS has some restrictions on what it will be 
>> able to accept (ie: no sub-day recurrences) but is that an "absurd" 
case 
>> like "repeating every second of every day of the year" or a more 
realistic 
>> case like "The walking tour of Boston begins at 9:00AM and again at 
1:00PM 
>> EST".  The CPL worlds requirements are such that it cannot accept 
>> (currently) some kinds of real world examples (not absurdities) and 
that 
>> is fine; its part of their model of things.  In the future there will 
be 
>> other models that do not have those restrictions (ie: SKiCAL). 
>
>My original model of CPL was to define a number of restrictions on what a
>CPL script had to accept.  This was where the IESG got all up in arms --
>"you're breaking interoperability with iCalendar!"  Okay, I figured,
>interoperability with iCalendar on (moderately) complex or pathological
>cases is important, so we need a good definition of what's allowed and
>what's too complex or pathological.  But now I'm hearing that CalSch
>*doesn't* want to define this, that each implementation is allowed to 
pick
>its own definition of complex and pathological events.

There is a fine line between having the standard define what is 'absurd' 
and leaving it to each implementation to decide that.  Given that each 
implementation has its own underlying architecture as well as usage model 
there is some leeway between what is patently absurd for all C&S and what 
some apps may consider absurd.  I do think that for the C&S world CalSched 
can define its own set of rules as to what we consider truely absurd while 
not precluding reuse or expansion in the future.  In the case of 
recurrence grammars we have attempted to create a powerful and flexible 
grammar that can be reused.  From what Ive heard so far I do not think its 
way off the mark; just some clarifications seem to be in order (along 
w/some better examples).

Your restrictions were not necessarily ones that a C&S implementation 
would have used since it has a different focus.  Your reuse of our format 
would have mandated changes that would prevent some existing CUAs from 
working w/other CUAs in addition to any CPL server.  The WG had already 
considered that differences of architecture/implementation would happen 
and we created the REQUEST-STATUS property to help deal with it.  Instead 
of ipTel considering that in its design from what I read initially, it 
appeared as if ipTel expected all implemenations to always be at the same 
'level' of functionality.  No apparent design was put into dealing with 
older CPL servers and newer clients (or visa versa) so there was no way to 
communicate this diversity.  This is an overly optimistic view of the 
world...

As systems get faster, CPL servers will presumably be able to do more 
processing of RRULEs in the same O(1) time so I would hope that the CPL 
draft would take that into account.  So far Ive yet to hear why the CPL 
server cannot simply reject the iCalendar that it deems unable to be 
processed in O(1) and accept others.  Even w/o a BYSETPOS subcomponent, it 
is still possible for me to generate ~31M instances using time BYxxx 
subcomponents.  Unless you propose removing them entirely then the CPL 
server needs to be able to deal w/them and presumably reject the iCalendar 
object that contains any BYHOUR, BYMINUTE or BYSECOND.  So why cant the 
same prefiltering be done by a CPL server for BYSETPOS instead of removing 
it entirely?

This may be belaboring a point that has already been resolved and if so, 
great.  However, if not then we can revisit BYSETPOS once we have some 
clearer points of discussion identified.

>   Compliant iCalendar CS or CUAs should not generate event descriptions
>   beyond the basic subset unless they know that all receivers of the 
event
>   can understand them correctly.

You can nevery really claim this with MUAs or CUAs.  I may suspect that, 
say, Steve Mansour is using the iPlanet CUA since hes its PM and that my 
friends at Microsoft are using Exchange/Outlook but I have absolutely 0 
idea what Jonathans CUA/MUA is.  Its not possible to tell that short of 
looking at the PRODID of an iCalendar you send me (and even then...).

This is why we built VERSION to have a "minver;maxver" format so that a newer CUA could send an iCalendar 
object to any ATTENDEE and their CUA could detect if it can even deal with 
the iCalendar or not.  If the recieving CUA is below the minver then it 
cannot really deal with it and it should 'fail gracefully' (thats a 
technical term).  At one point we had an example of that in one of our 
drafts but its been 3+ years since I even looked for it.  Perhaps Eric or 
Steve can confirm it and post the citation (no time to search if I want to 
get thru this backlog).

Bruce

From:   [email-masked]
Date:   06/11/2001 07:10:33 PM AST
Subject:        Re: Can we fix BYSETPOS?

On Monday, June 11 2001, "[email-masked]" wrote to "Jonathan Lennox, 
Alan Davies, [email-masked]" saying:

> Jonathan responded with:
> >As we've discussed before on these lists, we have two choices when it 
> >comes to dealing with absurd statuses:
> >
> >1. Let each implementation decide what it isn't going to handle.
> >2. Let the standard limit what implementations have to support.
> >
> >The problem with choice 1 is that it effectively abandons 
> >interoperability.
> 
> Well unless you expect _every_ implementation to implement every
> feature/option the exact same way then this is not really a problem.  We
> would then have no need for the COMPATIBILITY in IMAP4 or the EHLO in 
SMTP
> (or... ) since every implementation implements only a fixed set of
> features and we would never have a migration case since the set never
> grows w/o _every_ implemenation concurrently rolling it out...

iCalendar COS isn't like IMAP4 or SMTP.  iCalendar COS is like RFC822 or
HTML.  It's a data format.

iTIP is like IMAP4 or SMTP.  It's a transaction protocol.

When RFC822 was extended with MIME you got "If you can see this, your mail
reader doesn't understand MIME" blocks, and large unreadable blobs of
base64-encoded text.  When HTML was extended with frames, you got 
<noframes>
sections that were supposed to contain equivalent content, but actually
contained "Netscape Now!" buttons.

Or, in other words, extensibility in data formats is almost always 
painful.

Now, mind you, sometimes you need to have your data formats be extensible.
However, when you do this, experience seems to have shown that the right
thing to do is to have the thing say, up front, "I'm an extended-X.  I may
look like an original-X, but I'm not, so if you don't know what an
extended-X is, give up now."

This also means that you need a clear distinction between "original-X" and
"extended-X."  Having them shade vaguely into each other is a bad idea.

> Ok, the sarcasm was a bit heavy there but I got stuck in a 2.5 hour mtg 
> today for no reason so Im a bit punchy.  But the logic that "If my 
engine 
> does not like Feature X of a protocol for some reason so it wont be able 

> to interop with others that do" is simply a myth if the protocol plans 
> ahead and provides ways for conveying this back (hence some of the 
> REQUEST-STATUS values in iTIP).

If there's a protocol which is specific to your application, yes.

> Before we accept that most CUAs are going to actually generate and use 
> some of these "absurd" entries I'd contest the fact that they would to 
> begin with.  The existing C&S CUAs from CalConnect do a decent job of 
not 
> actually generating some of the overly complex, nasty or "absurd" 
entries 
> so is it realistic to think that the current or immediately pending crop 

> of CUAs are going to generate them?  Probably not.  Even if someone 
redid 
> an OpenSource C&S app to generate them; its not likely to be the SOP we 
> will find in day to day usage.

Oh, absolutely.  I reached this point of view by worring about entries
generated by actively hostile adversaries, who want to launch a
denial-of-service attack on my server by feeding me input that makes me 
fall
over.  If you're not worried about this case, things are not nearly so 
much
of a problem.  (Though I would say that current experience in the Internet
seems to indicate that you *do* have to worry about this.)

> For the CPL case the CUA / CS has some restrictions on what it will be 
> able to accept (ie: no sub-day recurrences) but is that an "absurd" case 

> like "repeating every second of every day of the year" or a more 
realistic 
> case like "The walking tour of Boston begins at 9:00AM and again at 
1:00PM 
> EST".  The CPL worlds requirements are such that it cannot accept 
> (currently) some kinds of real world examples (not absurdities) and that 

> is fine; its part of their model of things.  In the future there will be 

> other models that do not have those restrictions (ie: SKiCAL). 

My original model of CPL was to define a number of restrictions on what a
CPL script had to accept.  This was where the IESG got all up in arms --
"you're breaking interoperability with iCalendar!"  Okay, I figured,
interoperability with iCalendar on (moderately) complex or pathological
cases is important, so we need a good definition of what's allowed and
what's too complex or pathological.  But now I'm hearing that CalSch
*doesn't* want to define this, that each implementation is allowed to pick
its own definition of complex and pathological events.

So I find myself in the situation that I'm told I have to be fully
interoperable with a number of implementations which aren't fully
interoperable with each other.  This is ... challenging.

Is this group willing to say, definitively:

   Every iCalendar CS or CUA implementation is allowed to pick its own
   definition, aribitrarily, of what event descriptions are too complex 
for
   it to handle, and refuse to handle such event descriptions.  Such
   implementations are still fully compliant implementations of RFC 2445 
and
   its successors.

If so, I'll just resurrect the last CPL draft with some minor editorial
changes, tell the IESG about the CalSch working group consensus, and be on
my way to Proposed Standard.  However, I suspect the IESG would have
*serious* concerns with this statement.  (And I don't think the group 
would
be willing to say it, anyway.)

As I mentioned in my earlier mail, I think the following statement would 
be
much more generally acceptable:

   Every compliant iCalendar CS or CUA must be able to correctly 
understand
   a certain basic subset of event descriptions, defined by the
   specification.  They may also understand more general event
   descriptions.
   Compliant iCalendar CS or CUAs should not generate event descriptions
   beyond the basic subset unless they know that all receivers of the 
event
   can understand them correctly.
   Profiles of iCalendar can declare that a larger subset of iCalendar is
   required for that profile.

> Again this last bit assums that some to-be-written CUA will actually 
> generate internally (and use??) an absurd case and I challenge that 
> assumption.  Sure, it is possible to build a CUA that is purposely nasty 

> but is that a realistic norm or a "what about" kind of exception to the 
> rule??

Some of both.  If I'm writing a powerful CUA, I want to know what sorts of
things I should *avoid* generating to prevent a CS from choking.  (For
instance, I know of calendaring applications that refuse to deal with 
events
with more than 500 or so repetitions.  I don't think these applications
currently make any claim to iCalendar support -- they're just vCalendar, 
or
something proprietary, I think -- but it's the sort of thing you have to
worry about.) 

-- 
Jonathan Lennox
[email-masked]