Subject: Re: [Sip] New revision of draft-ietf-sip-body-handling-00.txt

From: Gonzalo Camarillo <[email-masked]>
Date: Mon, 03 Sep 2007 10:48:54 +0300

Hi Paul,

thanks for your comments. Yes, this is the open issue we need to close 
in order to move forward. We have really two issues:

1) In Chicago, people talked about multipart/related and the possibility 
of restricting references between body parts to bodies within a 
multipart/related. The current revision of the draft specifies that so 
that we can discuss whether or not it is what we want to do.

Another way of restricting which bodies can reference other bodies would 
be to only allow forward references. That is, a body can only reference 
another body if the latter body appears after the former. Do we have 
examples where a body would like to reference another one that is not 
part of the same multipart/related?

2) Then, we have the issue of what happens when the receiver does not 
understand the reference. Right now the draft says that extensions 
should take care of this case. However, we could say something more 
about that. For example, multipart/related addresses this issue saying 
that if the receiver understands multipart/related, it ignores the 
disposition types of the body parts within it. However, if the received 
does not understand multipart/related, it will treat it as 
multipart/mixed and, thus, will process the body parts based on their 
disposition types.

Defining a disposition type that means "don't process me unless you find 
a reference to me" would be a good way to have receivers ignore body 
parts. It works better than an option tag because the sender does not 
need to resend its request. It would be good to get more opinions on this...

Cheers,

Gonzalo

Paul Kyzivat wrote:
> To clarify futher my last point below...
> 
> What I am saying is that IMO body parts must always be processed
> according to their type and disposition, whether there are references to
> them or not. If there is a reference, and it is understood, then it can
> modify or supplement the processing, but if the reference is not
> recognized or understood then the other processing still occurs.
> 
> If we want to support the possibility of a reference to a body part,
> where there is to be no processing of the body part when the reference
> is not understood, then we need a content-disposition that says "don't
> do anything with this unless you process a reference to it."
> 
> An alternative is to simply assume there are no such cases, and warn
> people that use references to select a C-D for the referenced part that
> will have an appropriate result if the reference is not understood.
> 
> For instance, we have both the Alert-Info header and the "alert" C-D. If
> you use an A-I with a CID reference, then you can use "alert" as the C-D
> of that part. In that case, the A-I header is redundant unless it
> contains something that modifies the processing.
> 
> An example where the problem I am concerned about may exist is in
> draft-ietf-sip-location-conveyance-08. It has an example of a
> Geolocation header referencing a body part. There is no C-D header in
> the example, so by default it is "render;handling=required". And its C-T
> is application/pidf+xml. If a recipient of this message didn't
> understand the Geolocation header it would reach the conclusion that it
> should render the pidf. Its debatable whether that is the desired
> conclusion - I think not. IMO the intent is that if the Geolocation
> header isn't processed then the pidf should be ignored. So, IMO this
> body part ought to contain:
>     Content-Disposition: by-reference;handling=optional
> 
> (I don't care what name we use for the disposition, just that we have one.)
> 
>         Thanks,
>         Paul
> 
> Paul Kyzivat wrote:
>  > Gonzalo,
>  >
>  > Obviously we allow a "forward" reference from the sip message itself
>  > (which from a mime perspective are just a bunch of mime headers.)
>  >
>  > I think that establishes a precedent that needs to be continued. An
>  > example of why can be seen in sipfrag:
>  >
>  > If we had a valid sip response that had a reference into one of its body
>  > parts, then if that was turned into a sipfrag and stuffed into the body
>  > of a NOTIFY, then it must still be valid.
>  >
>  > I just did a quick scan of the new draft, so the following is a
>  > tentative comment, until I can read it more carefully:
>  >
>  > I think the draft is now saying that the decision about whether to
>  > process a body part according to a reference to it, or process
>  > independently, is decided based on whether a reference is found. We had
>  > this discussion before, and I think we agreed that wouldn't work well,
>  > and that instead we would have a special C-D type for body parts that
>  > are disposed of based on a reference.
>  >
>  > I don't think the introduction of issues related to multipart/related
>  > changes this. The obvious cases are when there is a single
>  > multipart/mixed containing some body parts. Some of those may be
>  > referenced from CID URIs in sip headers, and others (e.g. SDP) may stand
>  > on their own. It is entirely possible that a body part might be
>  > referenced from some extension header that the recipient doesn't
>  > understand. In that case it may erroneously decide to process the body
>  > part on its own, when it should instead have ignored it because the
>  > header isn't being processed.
>  >
>  >     Thanks,
>  >     Paul
>  >
>  > Gonzalo Camarillo wrote:
>  >> Folks,
>  >>
>  >> I have just submitted a new revision of the body handling draft. Until
>  >> it appears on the archives, you can fetch it from:
>  >>
>  >> http://users.piuha.net/gonzalo/temp/draft-ietf-sip-body-handling-00.txt
>  >>
>  >> Per our discussions in Chicago, the draft now states that only body
>  >> parts within the same 'multipart/related' can reference each other
>  >> using cid URIs.
>  >>
>  >> If this is too restrictive, we could also allow using forward
>  >> references in any context (with no 'multipart/related' at all).
>  >>
>  >> Comments?
>  >>
>  >> Thanks,
>  >>
>  >> Gonzalo
>  >>
>  >>
>  >>
>  >> _______________________________________________
>  >> Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
>  >> This list is for NEW development of the core SIP Protocol
>  >> Use [email-masked] for questions on current sip
>  >> Use [email-masked] for new developments on the application of sip
>  >>
>  >
>  >
>  > _______________________________________________
>  > Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
>  > This list is for NEW development of the core SIP Protocol
>  > Use [email-masked] for questions on current sip
>  > Use [email-masked] for new developments on the application of sip
>  >
> 

_______________________________________________
Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [email-masked] for questions on current sip
Use [email-masked] for new developments on the application of sip

---

From: Paul Kyzivat <[email-masked]>
Date: Mon, 03 Sep 2007 11:56:07 -0400

Eric - we need your input here as our mime guru.

	Paul

> that we can discuss whether or not it is what we want to do.

It seems that multipart/related has some role to play. I am not entirely 
clear what that is. I think it clearly isn't enough by itself.

> part of the same multipart/related?

Well, we clearly have a need for references from the sip message itself 
to body parts that it contains. I guess that is a restricted form of 
forward reference. Perhaps it, plus the usage within multipart/related, 
is sufficient.

This reference into a contained body part does need to apply even when 
the reference itself is in a body part. An example of this is to take a 
sip message that uses this form and put the whole thing into a sipfrag.

> need to resend its request. It would be good to get more opinions on 

I do think we need input from a mime guru. Eric: please guide us on how 
to do this in a mime-compatible way.

	Paul

---

From: Eric Burger <[email-masked]>
Date: Tue, 04 Sep 2007 10:56:15 -0700

I agree 100% with Paul.  However, one thing I would like to hear opinions on
is the following:

Do we way body parts are always processed, no matter what?

It would be convenient to say body parts get processed as needed.  If there
is a related part that never gets referenced, then a conforming processor
might not process that part, as an optimization.

However, one thing we need to be mindful of are side effects.  MIME body
parts can make external references.  Those external references (to
http/https URI's) may hit application servers.  Those application servers
may do something based on the receipt of the URI.  It could be as simple as
recording the fact the body part was read (like a web beacon), or it could
be as complex as changing application state.

What do people think of this?  Do we say that SIP engines are opportunistic,
and the side effect MUST happen if the body gets used, buy the side effect
MAY happen if the body does not get used?  That would be my vote, as the
alternative, that all side effects (external references get resolved) happen
puts a pretty big burden on User Agents.

>>> references in any context (with no 'multipart/related' at all).

Notice:  This email message, together with any attachments, may contain information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated entities,  that may be confidential,  proprietary,  copyrighted  and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.

---

From: Eric Burger <[email-masked]>
Date: Tue, 04 Sep 2007 10:56:15 -0700

---

From: Paul Kyzivat <[email-masked]>
Date: Tue, 04 Sep 2007 17:39:13 -0400

> might not process that part, as an optimization.

This gets into semantics...

For one thing, I guess we should talk about "handling" rather than 
"processing" - this is tied to the C-D "handling=" parameter.

But the key thing is what does it *mean* to say you have handled 
something? Its one thing to understand the meaning of the type and 
disposition. What you actually *do* once that is understood is something 
else entirely.

I would expect that somewhere there would be a normative specification 
for each C-D of what it means to *handle* body parts with that 
disposition. This is where we get into trouble with "render", where we 
need a different definition for SIP than is used for email.

I presume that if we had such normative definitions, then that would 
determine what sort of optimizations, such as lazy processing, are allowed.

For example, 3261 defines (barely) the C-D "alert". It says this 
"should" (lower case) be rendered. Together with some words elsewhere 
about Alert-Info that warn of security issues, its seems clear that the 
UA is given discretion about actually rendering this. (But it is not a 
stellar example of clarity.) I would interpret this to mean that you can 
consider the "alert" body part to have been "handled" if you have indeed 
understood it and then *either* rendered it or made an explicit not to 
render it.

> be as complex as changing application state.

We have similar issues with headers that contain URL references, that 
might or might not be resolved. E.g. Call-Info.

Again, I don't see any general answer better than the specification of 
the particular header or C-D.

Based on this discussion I am beginning to conclude that we might as 
well treat each body part almost as if it were a header whose name is 
its C-D.

> puts a pretty big burden on User Agents.

This is further complicated by intermediate nodes. They are *permitted* 
to look at body parts. If, as a result, an intermediate node 
dereferences a reference within a body part, you will potentially have 
another side effect.

I also agree that we should give the UA latitude in what processing it 
does. For example, even though a request has a geopriv header with a 
body part containing location data, and has specified 
"handling=required", the UAS may have no use for the location. If it 
doesn't intend to use it then why should it process it?

	Paul

---

From: Gonzalo Camarillo <[email-masked]>
Date: Wed, 05 Sep 2007 13:10:28 +0300

Hi,

to be more concrete in our discussions, we need to make decisions on the 
following two issues:

1) Where in a SIP message is it allowed to reference a body part?

a) in header fields and body parts within the same multipart/related as 
the referenced body part
b) in header fields and body parts that appear before the referenced 
body part
c) in header fields and any body part

We discussed that doing c) would require to parse SIP messages twice to 
look for potential references. Therefore, we decided to do a) or b). The 
current draft specifies a), we doing b) might make sense as well (it 
would be less restrictive).

2) If a receiver does not understand a reference to a body part:

a) it should process (handle) it solely based on its disposition type 
(as a fall-back mechanism). If the sender does not want this to happen, 
it can use an option tag.

b) the body part will have a disposition type of "only-by-reference". 
This way, since the receiver did not understand the reference, it knows 
it should not process the body.

The current draft specifies a), but b) would be an interesting 
optimization (no need to use option tags and reissue requests). However, 
we would need to get the opinion of a MIME person on b).

Cheers,

Gonzalo

Paul Kyzivat wrote:
>> Notice:  This email message, together with any attachments, may 
>> contain information  of  BEA Systems,  Inc.,  its subsidiaries  and  
>> affiliated entities,  that may be confidential,  proprietary,  
>> copyrighted  and/or legally privileged, and is intended solely for the 
>> use of the individual or entity named in this message. If you are not 
>> the intended recipient, and have received this message in error, 
>> please immediately return this by email and then delete it.
>>

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 05 Sep 2007 09:38:32 -0400

Gonzalo Camarillo wrote:
> look for potential references.

It is only necessary to parse twice if you can't decide what to do with 
the body part without finding the reference first.

If the rule is that you *always* process a body part based on its type 
and disposition, and that references provide *additional* behavior, then 
you can do an initial parsing and processing without regard to 
references. If the processing in that pass happens to yield references 
that cannot yet be resolved, then the processing of those needs to be 
deferred.

> Therefore, we decided to do a) or b). The 

We have plenty of counterexamples that (a) is insufficient. Any case 
where a header field in the sip message references a body part (e.g. 
GeoLocation) is a counterexample. So I think we must exclude (a) from 
consideration.

This can be solved by (b). But it can also be solved by a weaker 
condition (call it (d)) - allowing references from one part to any parts 
contained within it. This would include a direct reference to *the* 
immediate body, or if the body is a multipart then to any immediate 
child of the multipart. Its  unclear to me whether reference to a more 
deeply nested part is needed, but to be safe lets say that should also 
be permitted.

But (b) or this weaker condition (d) doesn't cover the cross references 
among related parts covered by (a). I don't know where that usage comes 
from, but we probably need to allow it, since if it is needed for mail 
then it is probably needed for MESSAGE too.

So, I think the minimal sufficient requirement is for (a)+(b), (a)+(d), 
or (c).

I'm inclined to say we should just go with (c), plus the specification 
that body parts are *always* processed according to their type and 
disposition, and that references simply add additional processing on top 
of that. In turn, *that* leads to the need for a C-D that means "do 
nothing" for those cases when the reference defines the only intended usage.

> we would need to get the opinion of a MIME person on b).

Eric???

>> disposition. What you actually *do* once that is understood is 
>> something else entirely.
>>
>> about Alert-Info that warn of security issues, its seems clear that 
>> the UA is given discretion about actually rendering this. (But it is 
>> not a stellar example of clarity.) I would interpret this to mean that 
>> you can consider the "alert" body part to have been "handled" if you 
>> have indeed understood it and then *either* rendered it or made an 
>> explicit not to render it.
>>
>> This is further complicated by intermediate nodes. They are 
>> *permitted* to look at body parts. If, as a result, an intermediate 
>> node dereferences a reference within a body part, you will potentially 
>> have another side effect.
>>
>>> copyrighted  and/or legally privileged, and is intended solely for 
>>> the use of the individual or entity named in this message. If you are 
>>> not the intended recipient, and have received this message in error,

---

From: Adam Roach <[email-masked]>
Date: Wed, 05 Sep 2007 10:49:13 -0500

> be as complex as changing application state.

We can't count on such side-effects being activated -- whether external 
references are acted upon is a policy choice that the client can make 
unilaterally. For example, to prevent beacon-tracking of emails, many 
email clients allow suppression of embedded remote images.

> puts a pretty big burden on User Agents.

I think that such a MUST is overly-restrictive. For security and privacy 
reasons, I believe it is important for clients to be able to make 
policy-related decisions regarding external references, even in the case 
that the body containing such references is otherwise used by the client.

/a

---

From: Adam Roach <[email-masked]>
Date: Wed, 05 Sep 2007 10:52:51 -0500

On 9/5/07 5:10 AM, Gonzalo Camarillo wrote:
>
> to be more concrete in our discussions, we need to make decisions on 
> the following two issues:
>
> a) in header fields and body parts within the same multipart/related 
> as the referenced body part
> We discussed that doing c) would require to parse SIP messages twice 
> to look for potential references. Therefore, we decided to do a) or 
> b). The current draft specifies a), we doing b) might make sense as 
> well (it would be less restrictive).

I think a) is correct. If you have a body-part referencing another 
body-part, they are (by definition) related. Specifying b) does nothing 
but encourage extensions to specify illegal MIME handling (e.g., using 
"multipart/mixed" when the body parts are actually related).

> (as a fall-back mechanism). If the sender does not want this to 
> happen, it can use an option tag.
>
> This way, since the receiver did not understand the reference, it 
> knows it should not process the body.
>
> optimization (no need to use option tags and reissue requests). 
> However, we would need to get the opinion of a MIME person on b).

I like the properties of b) -- it gets you the same place as a), but 
without the occasional round-trip. If the MIME folks think that such a 
Content-Disposition is kosher, I think it's worth specifying it.

/a

---

From: Adam Roach <[email-masked]>
Date: Wed, 05 Sep 2007 10:57:01 -0500

On 9/5/07 8:38 AM, Paul Kyzivat wrote:
>> to look for potential references.
>
> It is only necessary to parse twice if you can't decide what to do 
> with the body part without finding the reference first.
>
> and disposition, and that references provide *additional* behavior, 
> then you can do an initial parsing and processing without regard to 
>> Therefore, we decided to do a) or b). The current draft specifies a), 
>> we doing b) might make sense as well (it would be less restrictive).
>

I think you're misreading a).  What Gonzalo proposes in a) is "IN HEADER 
FIELDS and body parts within the same multipart/related as the 
referenced body part"

Perhaps there's some ambiguity in the grammatical construction here. If 
you read this as "(in header fields and body parts) within the same 
multipart/related as the referenced body part" then you're right: this 
won't work. It also makes little sense. I think that the grouping was 
meant as: "(in header fields) and (body parts within the same 
multipart/related as the referenced body part)".

That seems to satisfy all the existing use cases. It also prevents the 
accidental specification of extensions that erroneously try to specify 
related body parts without using multipart/related.

/a

---

From: Paul Kyzivat <[email-masked]>
Date: Wed, 05 Sep 2007 13:19:51 -0400

Adam Roach wrote:

> won't work.

That indeed is the way I was reading it. At the least it needs to be 
clarified

> It also makes little sense. I think that the grouping was 

I'm still not sure that covers all the valid cases, though again it may 
be a matter of parsing the words.

Lets take a concrete example:

1) A usage for location-convenance (trimmed for readability):

    INVITE sips:[email-masked] SIP/2.0
    To: Bob <sips:[email-masked]>
    From: Alice <sips:[email-masked]>;tag=9fxced76sl
    Call-ID: [email-masked]
    Geolocation: <cid:[email-masked]>
      ;inserted-by=[email-masked] ;recipient=endpoint
    Supported: geolocation
    Accept: application/sdp, application/pidf+xml
    CSeq: 31862 INVITE
    Contact: <sip:[email-masked]>
    Content-Type: multipart/mixed; boundary=boundary1
    Content-Length: ...

    --boundary1

    Content-Type: application/sdp

    ...SDP goes here

    --boundary1

    Content-Type: application/pidf+xml
    Content-ID: [email-masked]

    <?xml version="1.0" encoding="UTF-8"?>
    ... bunch of presence stuff
    --boundary1--

2) Alice sends a REFER to Bob. As a result, Bob sends an
    INVITE to Charlie. Charlie's response contains an
    Alert-Info header with a CID URL. Bob wants Alice to
    know about this, so he sends the entire response in
    the NOTIFY for the refer event package:

    NOTIFY sip:alice.atlanta.example.com SIP/2.0
    To: <sip:[email-masked]>;tag=193402342
    From: <sip:[email-masked]>;tag=4992881234
    Event: refer
    Subscription-State: active;expires=600
    Content-Type: message/sipfrag;version=2.0
    Content-Length: ...

    SIP/2.0 200 OK
    To: <sip:[email-masked]>;tag=876402342
    From: <sip:[email-masked]>;tag=4992881235
    Alert-Info: <CID:[email-masked]>
    Call-ID: [email-masked]
    CSeq: 182 INVITE
    Content-Type: multipart/mixed; boundary=boundary1
    Content-Length: ...

    --boundary1

    Content-Type: application/sdp

    ...SDP goes here

    --boundary1

    Content-Type: audio/mpeg
    Content-ID: [email-masked]

    ... bunch of mpeg stuff here
    --boundary1--

I think your interpretation covers (1). But in (2) it is a header in the 
*body* of the notify that contains a reference to one of the parts of 
the multipart body of that body. I don't think the existing words cover 
that case.

	Paul

---

From: Gonzalo Camarillo <[email-masked]>
Date: Wed, 05 Sep 2007 21:08:46 +0300

Hi,

yes, the sentence should be read as Adam points out. After sending the 
email I noticed that adding a comma may have helped...

Thanks for the clarification,

Gonzalo

---

From: [email-masked]
Date: Thu, 27 Sep 2007 14:38:14 -0400

From: Gonzalo Camarillo <[email-masked]>

Reading this draft I was struck by:

   UAs MUST use the binary transfer encoding for binary payloads in SIP.

Firstly, this specification is orthogonal to everything else in the
I-D.  Nonetheless, it is an important point.

Secondly, what about non-binary payloads, etc.?  Don't you really
mean:

   UAs MUST use only the "binary" content transfer encoding in SIP.

Dale