Subject: [rohc] FW: [Sipping] SIGCOMP and large binary content SIP messages

From: "Lars-Erik Jonsson (EAB)" <[email-masked]>
Date: Wed, 12 Mar 2003 18:55:04 +0100

Forwarded from the SIPPING list....

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: den 12 mars 2003 17:26
To: [email-masked]; [email-masked];
[email-masked]; [email-masked]
Subject: RE: [Sipping] SIGCOMP and large binary content SIP messages

Yes, that is also what I had in mind for SIP. But my point is that
a SigComp receiver now needs to have some knowledge about application
message format and do parsing to determine the end of an uncompressed
message. This is something new (or at least unspecified) in RFC 3320 
and needs to be spelled out in a new (standard track) RFC. The document
will be short, but I think it's needed for interoperability.

I'll write up an I-D and submit it after IETF 56. But I'd like to know 
if folks in SIPPING think it should be done here or in ROHC.

Zhigang

> -----Original Message-----
> From: ext Adam Roach [mailto:[email-masked]]
> Sent: March 11, 2003 9:03 PM
> To: Liu Zhigang.C (NRC/Dallas); Adam Roach; Isomaki Markus
> (NRC/Helsinki); [email-masked]; [email-masked]
> Subject: RE: [Sipping] SIGCOMP and large binary content SIP messages
> 
> 
> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]]
> > 
> > Having binary content in SIP messages may have two impacts on 
> > SigComp: 
> > 
> > a) multiplexing of uncompressed messages with SigComp 
> messages on the
> > same port.
> ...
> > For a), it is only an issue for SigComp/TCP, not 
> SigComp/UDP (where one
> > SigComp message maps to one UDP packet). Adam's point 1 and 
> 2 are related
> > to this. As to point 1, I think the SigComp receiver still 
> needs to know 
> > (somehow) the end of an uncompressed message, or at least 
> the end of the 
> > binary part of each SIP messages. Otherwise, a bit pattern 11111 in 
> > the binary part will be mistaken by SigComp as the 
> beginning of a SigComp
> > message. As to point 2, I would add that the delimiter 
> 0xFFFF can appear
> > at the both the beginning and the end of a SigComp message. So, the
> > receiver needs to have the same parsing logic just mentioned for bit
> pattern
> > 11111.
> 
> This isn't any more of an issue than it is without SigComp. SIP
> messages provide their own framing, and SigComp messages do the
> same. They can be trivially mixed on the same stream:
> 
> 1. Peek at the first byte waiting in the stream.
> 
> 2. If it is 0xf8 or higher, the next message in the stream is
>    a SigComp message. The message continues until a 0xFFFF is
>    encountered in the stream.
> 
> 3. Otherwise, the next message in the stream is a SIP message.
>    Read the stream until a CR/LF/CR/LF is encountered. Parse
>    the headers, and then read the number of bytes specified
>    in the "Content-Length" header.
> 
> 4. Return to step 1.
> 
> /a
> 
_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP
_______________________________________________
Rohc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rohc

---

From: [email-masked]
Date: Thu, 13 Mar 2003 09:54:31 -0600

I agree. How to handling binary content is a generic issue for any
application messages. What I have in mind is to give generic 
descriptions/requirements (mainly on the SigComp receiver parsing part),
then touch on SIP (perhaps RTSP too) as a particular example.

Unless someone objects, I'll submit it in ROHC.

Zhigang

> From: ext Gonzalo Camarillo [mailto:[email-masked]]
> Sent: March 13, 2003 5:36 AM
> To: Liu Zhigang.C (NRC/Dallas)
> Cc: [email-masked]; Isomaki Markus (NRC/Helsinki);
> Subject: Re: [Sipping] SIGCOMP and large binary content SIP messages
> 
> 
> Zhigang,
> 
> you can submit that draft to ROHC. SIPPING can review it afterwards.
> IMO, the competence needed to generate this draft can be found more
> easily in ROHC than in SIPPING.
> 
> Besides, you could use SigComp to compress RTSP messages, for instance
> And RTSP is not handle by SIPPING either.
> 
> Gonzalo
> 
> [email-masked] wrote:
> > 
> -- 
> Gonzalo Camarillo         Phone :  +358  9 299 33 71
> Oy L M Ericsson Ab        Mobile:  +358 40 702 35 35
> Telecom R&D               Fax   :  +358  9 299 30 52
> FIN-02420 Jorvas          Email :  [email-masked]
> Finland                   http://www.hut.fi/~gonzalo
>

---

From: [email-masked]
Date: Tue, 25 Mar 2003 11:59:30 -0600

Hi Carsten and Lars-Erik,

I've volunteered to submit a personal draft on this issue. From
ROHC WG chairs' point of view, do you think this is the way to proceed?

Of course, I understood during the IETF56 that Carsten had some
doubts on whether this is indeed an issue. My email below explained
the issue a bit. I don't know if it's clear. Either way, we can continue
discuss this in ROHC mailing list
http://www1.ietf.org/mail-archive/working-groups/sipping/current/msg04115.html

BR,
Zhigang

> From: ext [email-masked] [mailto:[email-masked]]
> Sent: March 13, 2003 9:55 AM
> To: [email-masked]
> [email-masked]; [email-masked]; [email-masked]
> Subject: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP
> messages
> 
> 
> > From: ext Gonzalo Camarillo 
> [mailto:[email-masked]]
> > > Sipping mailing list  
https://www1.ietf.org/mailman/listinfo/sipping

---

From: [email-masked]
Date: Wed, 26 Mar 2003 18:13:06 -0600

Lars-Erik,

> Actually, the proposal from the SFO meeting was to create a new
> WG draft to define what we think should be defined for the case
> of "SigComp for SIP", as RFC 3486 does not seem to be sufficient.

Yes, this is the document Carsten proposed to "bind" SigComp and SIP. 
If I understand correctly, it will address SIP specific issues: larger
DMS, SigComp compartment mapping, etc. I volunteered to be the editor 
of this document. This is clear to me and not my question here. 

My question is the handling of binary content in application messages. 
In my view, this is a generic issue although it started with SIP. My 
thought was to have a *separate* document that addresses this issue (and 
perhaps other generic SigComp "bugs" if we find out). So, my question is:

1) Does it make sense to have two separate documents, as I explained above?

2) If the answer is yes, how do we proceed for the second one (i.e. on binary
content)? I discussed the issue on SIPPING list and volunteered to write 
a draft. I guess from your reply that it's OK for me to submit a personal
draft to ROHC WG.

BR,
Zhigang

---

From: [email-masked]
Date: Wed, 26 Mar 2003 19:00:32 -0600

Richard,

I agree on most of your comments. What I want to clarify and ask
for input is the way to organize documents. Based on comments
from Gonzalo and my own view, we should have *two* documents:

Document-1. Not specific to SIP. Deal with how to apply SigComp
to application protocol whose messages may contain binary (i.e.
non UTF-8 encoded) data. It should contain at least two parts:
a) multiplexing between uncompressed and SigComp messages on
the same TCP port, and b) performance implication. Part a) should
be normative and b) should be just recommendations.

Document-2. SIP Specific. Deal with issues on how to apply SigComp 
to SIP, e.g. default DMS, compartment mapping/management, how to
deal with binary data in SIP messages, etc.

If we agree on the two documents approach, I think Document-1 should
be ROHC WG document as it is not SIP specific. As to document-2, I
tend to agree with you that it should be done in SIPPING WG. But
I really don't have strong feeling, as long as it is done. 

During the ROHC meeting in IETF-56, I've volunteered to be the editor
of document-2. I see you've made many good points here. So, here is 
my proposal. I'll write a skeleton draft with TOC, fill in some sections
that looks obvious to me (e.g. DMS, binary content, and perhaps 
compartment mapping) and then send to you so you can change and fill with 
other issues you pointed out below. I'll try to do that in the next 2~3 weeks.
Hopefully, people will agree which WG to home the document by then. :-)

BR,
Zhigang

> From: ext Price, Richard [mailto:[email-masked]]
> Sent: March 26, 2003 7:58 AM
> To: Liu Zhigang.C (NRC/Dallas); [email-masked];
> [email-masked]
> Cc: [email-masked]; [email-masked]
> Subject: RE: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP
> messages
> 
> 
> Hi,
> 
> I think that the issue with SigComp and large binary
> content SIP messages is part of a more general issue -
> namely that before using SigComp to compress a particular
> application we need to standardise the following:
> 
> 1. Mechanism for SigComp discovery
> 2. Resources offered by SigComp
> 3. How to delimit messages
> 4. When to save state
> 
> Currently the SIPPING WG has provided RFCs for 1 and 2,
> but not for 3 and 4. It would be useful to get a draft
> out for these ASAP, as it would clarify several issues on
> how to get SigComp working with SIP (including the issue
> with large binary content SIP messages).
> 
> I think that any WG draft should be a product of the
> SIPPING WG, because the proposed solution for delimiting
> SigComp messages will need to take into account the
> behaviour of SIP itself (or we won't be able to multiplex
> SigComp and SIP messages on the same port). However, it
> would be good to get the ROHC WG involved in generating
> the draft as well!
> 
> Regards,
> 
> Richard
> 
> P.S. For anyone who's interested, the mechanisms that
> particular applications need to provide before they can
> use SigComp are as follows...
> 
> ------------------------------------------------------
> 
> 1. Mechanism for SigComp discovery
> 
>    - How does the application know when to use SigComp?
> 
> For SIP this issue is already sorted (RFC 3486).
> 
> ------------------------------------------------------
> 
> 2. Resources offered by SigComp
> 
>    - Does the application need to offer more than the
>      minimum amount of resources?
> 
>    - Does the application need any extra state (e.g.
>      a static dictionary)?
> 
> For SIP we've already got a static dictionary (RFC 3485).
> However, we haven't currently discussed how much memory
> should be provided in a SigComp implementation for SIP.
> If we don't specify a value then SigComp defaults to the
> minimum possible (2K), which is rather tight for some of
> the more generously sized SIP messages!
> 
> ------------------------------------------------------
> 
> 3. How to delimit messages
> 
>    - How does the application distinguish between
>      SigComp messages and uncompressed messages?
> 
>    - Is the application allowed to use "continuous mode"?
> 
> This is the issue that's been raised for large binary
> content SIP messages. If we want to send a mix of
> uncompressed and SigComp messages within the same TCP
> connection, how do we delimit the end of one message and
> the start of another?
> 
> A related issue is how to handle very large SIP messages
> (> 64K). SigComp is able to compress large SIP messages
> by using "continuous mode", where the message boundaries
> are ignored completely and the application data is just
> transmitted as a stream. However, this mode of operation
> raises some additional security issues, so it's up to the
> SIPPING WG to decide whether or not to use it in SigComp
> for SIP.
> 
> ------------------------------------------------------
> 
> 4. When to save state
> 
>    - How much security is needed in order to save state?
> 
>    - Which messages belong to each state "compartment"?
> 
>    - How long should the application keep state?
> 
> This is quite an important issue that doesn't seem to
> have been discussed in the SIPPING WG. It's up to the
> application whether or not to let SigComp save state,
> and for interoperability it's important to have at
> least a general idea of how long to keep saved state,
> when to reject state due to insufficient security etc.
> 
> ------------------------------------------------------
> 
> > Sent: Tuesday, March 25, 2003 6:00 PM
> > To: [email-masked]; [email-masked]
> > Cc: [email-masked]; [email-masked]
> > Subject: RE: [rohc] RE: [Sipping] SIGCOMP and large binary 
> content SIP
> > messages
> > 
> > 
> > the issue a bit. I don't know if it's clear. Either way, we 
> > can continue discuss this in ROHC mailing list
> > 
> > > > > > > a) multiplexing of uncompressed messages with 
> SigComp messages on the
> > > > > > > same port.
> > > > > > ...
> > > > > > > SigComp message maps to one UDP packet). Adam's 
> point 1 and 2 are related
> > > > > > > to this. As to point 1, I think the SigComp 
> receiver still needs to know
> > > > > > > (somehow) the end of an uncompressed message, or 
> at least the end of the
> > > > > > > the binary part will be mistaken by SigComp as 
> the beginning of a SigComp
> > > > > > > message. As to point 2, I would add that the 
> delimiter 0xFFFF can appear
> > > > > > > receiver needs to have the same parsing logic 
> just mentioned for bit pattern
> > > > > > > 11111.
> > > > > >

---

From: [email-masked]
Date: Thu, 27 Mar 2003 19:50:29 -0600

Lars-Erik,

IMO, binary content in application message is a general issue for
SigComp multiplexing. It's not specific to SIP. I already explained 
this in rohc and sipping mailing list. But here is a simple example. 

Let's say endpoint A already knows endpoint B supports SigComp on TCP 
port number 5000. A sends, on port 5000, uncompressed message 1 followed 
by message 2 compressed using SigComp. From RFC 3320, message 2 starts
with bit pattern 11111. So, if message 1 has no binary content, endpoint
B can easily detect the beginning of message 2 (i.e. end of message 1)
by detecting 11111. However, if message 1 contains binary content,
that means the (uncompressed) binary part may contain byte 11111xxx.
This "mimicking" will make endpoint B mistakenly think a new SigComp
message starts at that byte. Obviously, things will be messed up
from that point.

This is what I referred to as the multiplexing issue. I hope I explained
it better this time for you. 

Anyway, I'll submit a personal draft to help the discussion, although the 
above example should make it obvious already.

Zhigang

> From: ext Lars-Erik Jonsson (EAB)
> [mailto:[email-masked]]
> Sent: March 27, 2003 2:18 AM
> Cc: [email-masked]
> > My question is the handling of binary content in application 
> > messages. In my view, this is a generic issue although it started
> > with SIP. My thought was to have a *separate* document that
> > addresses this issue (and perhaps other generic SigComp "bugs" if
> > we find out).
> 
> First of all, it is not yet clear to me that this is a general issue,
> at least I have not been able to see consensus on that. Secondly, if
> we have general unclear issues with SigComp, those would go into the
> Implementer's guide at this point.
> 
> Currently, I therefore think we can and should avoid having even more
> documents.  However, you can always provide input by writing a draft,
> that is up to you.
> 
> Regarding the SigComp&SIP document, it does not matter much
> in which WG it is "hosted", as long as it is written and well 
> reviewed 
> by people from both sides.
> 
> 
> Rgds,
> /L-E

---

From: [email-masked]
Date: Fri, 28 Mar 2003 08:24:09 +0100

Hi Zhigang,

 I don't think it is a problem with SigComp multiplexing, but more like a SIP internal issue. What if there is a TCP stream of SIP messages and the Content-Length header is missing or false? They will screw up the same way with or without SigComp. During normal functioning both SIP and SigComp provide their own flow separation mechanism and they work just fine.
 As for other applications the requirement is that the messages must not start with 11111xxx and the end of the messages must be detected. If that is assured binary content is not a problem anymore. Besides, whatever pattern you choose to distinguish SigComp it might appear in application messages <evil> for example if we attach a Sigcomp message as application/octet stream </evil>. Only the possibility for screwing up will be lower.

Best Regards,
Robert

> Sent: March 28,2003 2:50
> To: [email-masked]

---

From: [email-masked]
Date: Fri, 28 Mar 2003 09:36:09 +0200

SIP has this header called content-length that tells how long the body of a sip message is regardless of its binary of UTF-8 contents.

The SIP parser will read the content-length value and then take that many octets from whatever follows the sip headers as body. If the body has binary with pattern 11111, the parser doesn't care and doesn't even look in the body.

Any protocol that wants to make use of sigcomp should fix its own problems.

Regards,
Hisham

> Sent: Friday, March 28, 2003 3:50 AM