Subject: MIME encapsulation of multipart responses

From: Andrea Campi <[email-masked]>
Date: Tue, 27 Aug 2002 18:19:42 +0200

Hi all,

I had a look at the archives and, while there was some discussion
about the (optional) use of multipart/* mime types, I didn't see
any agreement.

I'm asking this because the draft is still ambigous on how to send
multiple replies, in case multiple TARGETs or multiple METHODs are
involved. For instance, in section 9.1.4:

The CS would sends the REPLY in separate MIME objects, one per
   TARGET.

 S: Content-Type: text/calendar
      S:
      S: BEGIN:VCALENDAR

[...]

   And would send the second part of the reply later:

 S: Content-Type: text/calendar
      S:
      S: BEGIN:VCALENDAR

[...]

multipart/* is obviously not used, but then I can't see any mean
of determining where one body ends and the other one begins - I
can only guess they can be in different ANS BEEP frames?

Besides, the messages from Doug and other people left another
opportunity open, that is, one body with more iCalendar containers:

BEGIN:VCALENDAR
..
END:VCALENDAR
BEGIN:VCALENDAR
..
END:VCALENDAR

Thoughts?

Bye,
	Andrea
-- 
Andrea Campi                              mailto:[email-masked]
I.NET S.p.A.                              http://www.inet.it
Direzione Tecnica - R&D			  phone: +39 02 32863 ext 1
v. Darwin, 85 - I-20019			  fax: +39 02 32863 ext 7705
Settimo Milanese (MI), Italy

---

From: John Stracke <[email-masked]>
Date: Tue, 27 Aug 2002 12:47:07 -0400

>opportunity open, that is, one body with more iCalendar containers:
That would not be legal text/calendar, any more than

<foo>...</foo>
<bar>...</bar>

is legal XML--in both cases, you've got a forest when the spec says you 
have to have a tree.

-- 
/================================================================\
|John Stracke      [email-masked]                       |
|Principal Engineer|http://www.centivinc.com                     |
|Centiv            |My opinions are my own.                      |
|================================================================|
|"I said no camels! That's 5 camels! Can't you count?" -- Indiana|
|Jones & TLC                                                     |
\================================================================/

---

From: Andrea Campi <[email-masked]>
Date: Wed, 28 Aug 2002 10:21:17 +0200

> is legal XML--in both cases, you've got a forest when the spec says you 

I might be wrong, and please forgive me if I am, but:

Content-Type: text/calendar

...

Content-Type: text/calendar

...

doesn't look any more valid to me.

This would only leave multipart/related, then.

Bye,
	Andrea

---

From: Mark Davidson <[email-masked]>
Date: Wed, 28 Aug 2002 10:16:12 +0100

> This would only leave multipart/related, then.

Each responce is in a separate ANS (see BEEP) reply followed by a NUL reply.

So if there is only a single text/calendar object as a reply it is sent as a RPY.

If there are multiple text/calendar objects then each is send as an ANS.

On the receiving end if you get a RPY you know this is the only response, if you get
an ANS then you continue getting responces until you get a NUL.

Mark

---

From: Andrea Campi <[email-masked]>
Date: Wed, 28 Aug 2002 11:30:47 +0200

> an ANS then you continue getting responces until you get a NUL.

This is really ok with me, but it's not stated that way in the draft.
There's just a very brief mention of ANS wrt multiple targets, not
multiple methods. Even then, the examples for CREATE only say:
" The CS would sends the REPLY in separate MIME objects, one per
   TARGET. "
But there is no mention that they are in separate ANS.

Bye,
	Andrea

---

From: Mark Davidson <[email-masked]>
Date: Wed, 28 Aug 2002 11:14:23 +0100

> But there is no mention that they are in separate ANS.

The draft kind of says something in section 11 (BEEP and CAP). But it would be better if it was more 
explicit. I might be wrong in how I am interpreting the draft, but it seemed to be the simplest 
answer when using BEEP.

  Mark

---

From: Doug Royer <[email-masked]>
Date: Wed, 28 Aug 2002 14:16:11 -0600

In 2445:

> 4.4 iCalendar Object
>
>   The Calendaring and Scheduling Core Object is a collection of
>   calendaring and scheduling information. Typically, this information
>   will consist of a single iCalendar object. However, multiple
>   iCalendar objects can be sequentially grouped together. The first
>   line and last line of the iCalendar object MUST contain a pair of
>   iCalendar object delimiter strings. The syntax for an iCalendar
>   object is as follows:
>
>     icalobject = 1*("BEGIN" ":" "VCALENDAR" CRLF
>                  icalbody
>                  "END" ":" "VCALENDAR" CRLF)

So, 'multiple iCalendar objects ... grouped together." Tells
me that:

	BEGIN:VCALENDAR
	...
	END:VCALENDAR
	BEGIN:VCALENDAR
	..
	END:VCALENDAR

Is valid.

---

From: Doug Royer <[email-masked]>
Date: Wed, 28 Aug 2002 14:33:09 -0600

Mark Davidson wrote:
> On the receiving end if you get a RPY you know this is the only
> response, if you get an ANS then you continue getting responces
> until you get a NUL.

I think we want to allow:

	(1) Multiple BEGIN/END VCALENDAR in one MIME body.
	    All must be the same METHOD (per iCal or iMIP?)
            And I think 2445 "4.4 iCalendar Object" says
	    we can.

	(2) Allow multipart/related, where each 'part' can
	    have separate METHODs from the other 'parts'. Each
	    MIME object (part) may have multiple BEGIN/END
	    VCALENDAR

This is so we can transport an entire VAGENDA or VCALSTORE in
one multipart/related.

In CAP, I think we need to allow for any ANS or RPY to have:

	(1) A single MIME object with a single icalobject.

	(2) A single MIME object with multiple icalobjects's.
            (All same METHOD).

	(3) A multipart/related MIME object, with each part
	    being (1) or (2). And as each 'part' has a MIME
	    header, each related object can have a unique METHOD.

In some cases (T1 for example), everything in one blob will
be faster with less CPU overhead (does it matter?)

On small wireless devices, one large blob may give slower
response if retransmission's and noise are an issue. As smaller
packets tend to get through better.

Is there a BEEP way that the Initiator can request ANS vs RPY?
(I don't think so). I think that using ANS vs RPY is a transmission
issue or a and I don't think that it is a technical gain one way
or the other. If the Initiator  has very little memory, it may want
ANS with single mime objects with single icalobjects.

---

From: Doug Royer <[email-masked]>
Date: Wed, 28 Aug 2002 14:43:53 -0600

Mark Davidson wrote:
> 
> The draft kind of says something in section 11 (BEEP and CAP). But
> it would be better if it was more explicit. I might be wrong in how
> I am interpreting the draft, but it seemed to be the simplest
> answer when using BEEP.

I could flip a coin and be happy with the answer. I do not see
that it matters to CAP. (I know we have to decide).

If I do a 'get me the entire VAGENDA', which do I want:

	(a) One massive blob in a multipart/related MIME object.
   or
	(b) A massive amount of small blobs each in a single
            MIME object.
   or
	(c) A mixture of (a) and (b) with multiple icalobjects
	    in each (b) of the same METHOD.

Before we can decide on the ANS vs RPY. Are there any
of the above we can eliminate as valid (outside of BEEP)?
Are there more?

---

From: John Stracke <[email-masked]>
Date: Wed, 28 Aug 2002 16:51:17 -0400

Doug Royer wrote:

>>  iCalendar objects can be sequentially grouped together.
>>
Ah, bleep.  I knew I should've checked the spec.

(This'd be something to test for at CalConnect, BTW; I know I've written 
at least one parser that assumed there was only one iCalendar object per 
text/calendar, and I bet I'm not the only one.)

-- 
/===========================================\
|John Stracke      [email-masked]  |
|Principal Engineer|http://www.centivinc.com|
|Centiv            |My opinions are my own. |
|===========================================|
|This space intentionally not left blank.   |
\===========================================/

---

From: John Stracke <[email-masked]>
Date: Wed, 28 Aug 2002 17:06:11 -0400

>	(b) A massive amount of small blobs each in a single
I would recommend (b), on the grounds that it makes it easier to 
transport the iCalendar objects together outside of the BEEP and MIME 
wrappers--for example, if, N years from now, somebody wants to put CAP 
over a different transport.

---

From: John Stracke <[email-masked]>
Date: Wed, 28 Aug 2002 17:08:41 -0400

>	    MIME object (part) may have multiple BEGIN/END
I don't think we want to allow multiple choices--There's More Than One 
Way To Do It is a curse, not a blessing.

And why /related? What's the root object?

>packets tend to get through better.
I don't think this makes a difference--at and below the TCP level, the 
difference between packaging in MIME and packaging in iCalendar will be 
invisible.

---

From: Doug Royer <[email-masked]>
Date: Wed, 28 Aug 2002 15:54:47 -0600

> Way To Do It is a curse, not a blessing.

I think we do for the reasons below...

> And why /related? What's the root object?

What is an iCalendar 'root object'?
No such item is needed. You can reconstruct a VAGENDA from
booked items out of (original) order.

> difference between packaging in MIME and packaging in iCalendar will be

As a HAM radio operator I can tell you if sending 1k of data;
if you get air-wave noise 10 times, you can get this
for the same 1k data blob:

	Sending 256 byte packets; you send 10 of
	the 256 byte packets 10 times.
	(2,560 retry bytes + 1,024 non-retry bytes sent)

	Sending 1k byte packets; you send 10 of
	the 1k byte packet 10 times.
	(10,240 retry bytes + 1,024 non-retry bytes sent)

One large blob is almost always slower over the radio (and cell).
I am saying that it can be an issue.

On faster and more reliable transports, the overhead of small
packets starts to be greater than the transmission time of
larger packets.

I think we need to allow for both one large blob and
many small blobs. Let the CS decide. A CS implementation
could know or be configured to know its 'average' user requirements
and adjust the method. 

AND BECAUSE: All of the above are valid iCalendar objects.

The one multipart/related huge blob can arrive now - per iCal.
Correct?

Multiple single icalobject MIME objects can arrive now - per iCal.
Correct?

---

From: John Stracke <[email-masked]>
Date: Wed, 28 Aug 2002 18:34:20 -0400

>What is an iCalendar 'root object'?
There isn't one.  But multipart/related has a single bodypart, the root, 
which coordinates the others--as in the case of an HTML email message 
which contains both the HTML page (the root) and the images it uses. 
 Maybe you mean multipart/mixed?

>for the same 1k data blob:
Yeah, I *understand* that.  But that's for the IP layer to worry 
about--it should set the MTU to match the link layer, so that TCP can do 
path MTU discovery.  At the application layer, we just shovel bytes down 
the TCP pipe; formatting them into smaller or larger chunks won't make 
any difference in how they get broken into TCP segments, nor how the TCP 
segments get broken into IP packets.

>could know or be configured to know its 'average' user requirements
That's both unnecessary (IP handles it) and undesirable (an application 
admin should not 

>The one multipart/related huge blob can arrive now - per iCal.
Uh...no.  The term "multipart/related" does not appear in RFC-2445.  Nor 
is it a requirement for basic MIME processing.

>Multiple single icalobject MIME objects can arrive now - per iCal.
Yes.

-- 
/===============================================================\
|John Stracke      [email-masked]                      |
|Principal Engineer|http://www.centivinc.com                    |
|Centiv            |My opinions are my own.                     |
|===============================================================|
|Both candidates are better than a megalomaniac mutant lab mouse|
|bent on world domination...but it's pretty close.              |
\===============================================================/

---

From: Doug Royer <[email-masked]>
Date: Wed, 28 Aug 2002 17:41:56 -0600

> Maybe you mean multipart/mixed?

It has been a while so maybe it is multipart/mixed that I mean.

Is multipart/related when you have the first part that can
contain CID's or MID's?

If so, in 2445:

	4.2.1 Alternate Text Representation

	...

     DESCRIPTION;ALTREP="CID:<[email-masked]>":Project
       XYZ Review Meeting will include the following agenda items: (a)
       Market Overview, (b) Finances, (c) Project Management

	...

So multipart/<something> must be valid.

> is it a requirement for basic MIME processing.

Nor is it (or multipart/<anything>) excluded - so it is valid.
And I agree that it is not 'required', but iCalendar is a MIME
object, and a MIME object can be multipart/<whatever>  - correct?

I remember this list talking about multipart/<whatever> pre CAP
and just after the first cal-connect. Some implementations
used it, others disallowed it. I think the consensus was that
we can not mandate it, and can not disallow it. It was a MAY.

It is not that hard to implement multilpart/<whatever>.

---

From: John Stracke <[email-masked]>
Date: Thu, 29 Aug 2002 09:17:11 -0400

Yes.

>So multipart/<something> must be valid.
Not necessarily.  A cid: URL can point to a body-part outside the 
current message (see RFC-2392, section 1, paragraph 3).

Besides, examples aren't normative.

>Nor is it (or multipart/<anything>) excluded - so it is valid.
Oh, OK, so all CUAs have to understand multipart/I-just-made-it-up?

>object, and a MIME object can be multipart/<whatever>  - correct?
Yes, but a CUA is not required to understand the special semantics of 
multipart/related.

>we can not mandate it, and can not disallow it. It was a MAY.
I believe the context was that some iMIP implementations were refusing a 
multipart/<whatever> that contained a text/calendar body part.  This was 
proven to be unacceptable because RFC-2049 says that any MIME-compliant 
MUA MUST recognize multipart/mixed ("show the user all the body parts 
and let them do what they will with it") and, if they don't understand 
multipart/foo, MUST treat it as multipart/mixed.

The situation here is different, because RFC-2049 can't be taken to 
apply to CAP--or else we'd be requiring CUAs to accept text/plain from 
the CS.  Besides, before we were trying to look at existing standards 
and figure out what they said; here we're writing a new standard.  We 
need to agree on what the Right Thing is and write the spec to require it.

So, the question remains: multipart/mixed or multiple VCALENDARs in a 
text/calendar? The only advantage I can see to multipart/mixed is that, 
if you're parsing the incoming text/calendar as it comes in, and see 
something that means you have to discard it (say, if you did a query 
that returned more than you really wanted, because you couldn't express 
what you wanted in the query language), then multipart/mixed offers an 
easy way to skip past the rest of the body part--just look for the 
multipart boundary.

---

From: Mark Davidson <[email-masked]>
Date: Thu, 29 Aug 2002 16:30:13 +0100

> easy way to skip past the rest of the body part--just look for the 

Using multiparts seems to be an extra layer of complexity that is not needed.
Currently CAP parsers only need to understand text/calendar and I'd rather keep
it as simple as possible.

Mark Davidson

---

From: John Stracke <[email-masked]>
Date: Thu, 29 Aug 2002 12:30:34 -0400

Mark Davidson wrote:

>
>> then multipart/mixed offers an easy way to skip past the rest of the 
>> body part--just look for the multipart boundary.
>

I think I'd agree, but I wouldn't be too opposed if somebody came up 
with another benefit for multipart/mixed--after all, most implementors 
are going to use off-the-shelf MIME libraries, which will already have 
multipart.

---

From: Doug Royer <[email-masked]>
Date: Thu, 29 Aug 2002 10:58:22 -0600

> current message (see RFC-2392, section 1, paragraph 3).

No, re-read that RFC. A 'MID' can point out side
of a message. A 'CID' points to a specific body part
in the same message.

> Oh, OK, so all CUAs have to understand multipart/I-just-made-it-up?

No, but it seems clear to me that the examples in iCal (normative
or not) should not be excluded from working in CAP.

> text/calendar?

I don't see that it has to be an OR. Multiple VCALENDARs in
a single MIME body part are valid. And there are already
iMIP implementations that can do multipart/mixed. (and thanks
for the history).

And don't forget xCal. The current plan is to allow sending multipart
mixed with xCal and text/calendar (and plain text).

---

From: Andrea Campi <[email-masked]>
Date: Fri, 30 Aug 2002 12:02:51 +0200

> it as simple as possible.

Another thought just occurred to me. The same reasoning that allows
multipart/* in answers would allow it in commands. THIS would be a
mixed blessing: an initiator could send a multipart/* with multiple
CREATE, but then it would have a bad time interpreting the answer
(think what happens if some commands fail and some succeeds).

IMHO this is too complicated, and I've yet to see any real advantage.
I'd say if we have no really important reason to go this way, we should
KISS.

The only advantage to multipart/* that was mentioned in this thread
is the opportunity to skip answers we're not interested in, but this
could be accomplished even better using one-to-many ANS. I also
disagree on the point that the choice between a single large blob or
many small ones is irrelevant, or network performance related only -
a multithread CUA would work better with many small ones, as it would
be able to parse each of them independently and refresh its views.

Bye,
	Andrea

---

From: John Stracke <[email-masked]>
Date: Fri, 30 Aug 2002 08:43:34 -0400

>be able to parse each of them independently and refresh its views.
I don't think that's true; with multipart/mixed, you still need a 
single-threaded MIME parser to split out the various iCalendar objects 
in any event, so your multiple threads are going to be blocking on that 
main parser.  Plus, then you'll effectively be parsing the same data 
more than once (the first phase will be simpler, is all).  With a single 
multiobject text/calendar, you can just run a single pass over the data; 
parse it as it comes in, and construct an internal representation of 
each object.  When you hit an END:VCALENDAR, you can take that internal 
representation and pass it off to another thread, if you want.

|A mime is a wonderful thing to waste.      |

---

From: John Stracke <[email-masked]>
Date: Fri, 30 Aug 2002 08:57:34 -0400

>of a message. A 'CID' points to a specific body part
Nope.  You're thinking of RFC-1873, which defines the content-ID access 
type for message/external-body.  That one refers only to body parts in 
the same message; but RFC-2392 explicitly states that cid: can cross 
messages.  The paragraph I referenced says:

The "mid" (Message-ID) and "cid" (Content-ID) URL schemes provide
identifiers for messages and their body parts.  The "mid" scheme uses
(a part of) the message-id of an email message to refer to a specific
message.  The "cid" scheme refers to a specific body part of a
message; its use is generally limited to references to other body
parts in the same message as the referring body part.  The "mid"
scheme may also refer to a specific body part within a designated
message, by including the content-ID's address.

And, further down:

The Content-ID of a MIME body part is
required to be globally unique.

And:

Conforming
implementations can choose to, but are not required to, take
advantage of the content-id's uniqueness and interpret a "cid" URL to
refer to any body part within the message store.

>or not) should not be excluded from working in CAP.
The term "multipart" does not appear in RFC-2445.  The use of "cid:" in 
an iCalendar object does not imply that the object is contained in a 
multipart message.

>I don't see that it has to be an OR.
>
Of course it does.  When there's more than one equivalent way to do 
something, a standard should pick one.  Otherwise, everybody has to 
implement both in order to interoperate.

>And don't forget xCal.
See, case in point!

> The current plan is to allow sending multipart
That's terrible.  You'd spend bandwidth and cycles and gain absolutely 
no benefit in expressiveness or interoperability.

---

From: Andrea Campi <[email-masked]>
Date: Fri, 30 Aug 2002 15:54:32 +0200

> main parser.  Plus, then you'll effectively be parsing the same data 

Sorry, my fault for not being clear.
When I said many small ones, I meant multiple ANS each containing a
single text/calendar (with one or a few iCalendar objects). A
multipart/mixed would still qualify as a single large one from the
point of view of a multithreaded client, for the exact reason you
mentioned.

So, we're basically agreeing, you just explained it better ;-)

Bye,
	Andrea

---

From: Doug Royer <[email-masked]>
Date: Fri, 30 Aug 2002 11:18:00 -0600

> (think what happens if some commands fail and some succeeds).

Each would need to have a CMD and ID, so they would be uniquely
identifiable just as if they had been sent separately.

> I'd say if we have no really important reason to go this way, we should

We would be electing not to allow all valid 2445 mime objects.

How about a capability property to say that the endpoint can handle
multipart/<valid>? And if not provided, then no.
Such as:

	MULTIPART: related, mixed, ...

> could be accomplished even better using one-to-many ANS.

Why would that be better? I see it as just different.

> I also disagree on the point that the choice between a single large
> blob or many small ones is irrelevant, or network performance related
> only - a multithread CUA would work better with many small ones, as
> it would be able to parse each of them independently and refresh
> its views.

How about a capability property:

	MULTIPLE-VCAL: TRUE / FALSE

If FALSE, then the endpoint only allows for a single
VCALENDAR per MIME object, if TRUE multiple per MIME object
with the same METHOD are allowed. Then state that if MULTIPLE-VCAL
is FALSE, then use one-to-many ANS.

And state that a CS MUST be able to handle a CUA that has
no MULTIPART and MULTIPLE-VCAL:FALSE.

---

From: Doug Royer <[email-masked]>
Date: Fri, 30 Aug 2002 11:26:09 -0600

John Stracke wrote:
> Nope. ...
> ....Conforming

Your right. I was confused. A MID can refer to outside
the store, I got that mixed up - thanks.

> an iCalendar object does not imply that the object is contained in a

I does in 2445:

   Example:

--> The "ALTREP" property parameter value might point to a "text/html"
    content portion.

If it is not talking about another body part, what is it?

> implement both in order to interoperate.

No, a standard must pick one as a must. And make sure that
things interoperate. But disallowing some valid 2445 MIME objects
in CAP is the wrong thing to do.

---

From: John Stracke <[email-masked]>
Date: Fri, 30 Aug 2002 13:50:56 -0400

>We would be electing not to allow all valid 2445 mime objects.
2445 says nothing about multipart.

[...]

>	MULTIPLE-VCAL: TRUE / FALSE
No, this is illegitimate, because iCalendar permits multiple objects per 
text/calendar.

-- 
/=====================================================\
|John Stracke      [email-masked]            |
|Principal Engineer|http://www.centivinc.com          |
|Centiv            |My opinions are my own.           |
|=====================================================|
|"No, no, that's *not* a boat, that's Queen Victoria."|
\=====================================================/

---

From: John Stracke <[email-masked]>
Date: Fri, 30 Aug 2002 13:57:42 -0400

>If it is not talking about another body part, what is it?
It's another body part *in another message*.

>things interoperate.
A standard MUST provide at least MTI option.  A standard MAY provide 
extra options; but, if those options are equivalent in semantics, it 
SHOULD NOT.

We already have multiple VCALENDARs in a single text/calendar; packaging 
them instead as multiple text/calendars in a multipart/mixed will not 
add any new semantics.  Therefore, we SHOULD NOT permit multipart/mixed 
in CAP.

>But disallowing some valid 2445 MIME objects
MIME multipart is *not* valid in 2445.