Subject: RE: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP messages

From: [email-masked]
Date: Tue, 1 Apr 2003 07:44:03 -0600

Robert,

>  I don't think it is a problem with SigComp multiplexing, but 
> more like a SIP internal issue. What if there is a TCP stream 
> of SIP messages and the Content-Length header is missing or 
> false? They will screw up the same way with or without 
> SigComp. 

Since TCP is a reliable transport, this only happens for a
mis-behaved SIP sender. But that is not the "multiplexing" problem 
I mentioned below, which happens even "during normal functioning" 
(see below).

> During normal functioning both SIP and SigComp 
> provide their own flow separation mechanism and they work just fine.

I don't think so. That is what I intended to show by the example,
i.e. the issue to "multiplex uncompressed application messages and 
SigComp messages on the same port". One solution, as I mentioned
before, is to require sigcomp receiver has the ability to parse
uncompressed application messages and detect the end. 

(Of course, I saw Carsten's email and I need understand his
point of view, which says there is no such "multiplexing" and
it is not possible and necessary.)

BR,
Zhigang
_______________________________________________
Rohc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rohc

---

From: [email-masked]
Date: Wed, 2 Apr 2003 18:31:48 -0600

Carsten and all,

The reason I raised the "binary multiplexing", as you found out, 
is the following text in RFC 3320. I took the sentence literally, 
i.e., uncompressed application messages and SigComp messages can 
be transmitted on the same (destination UDP or TCP) port, one 
followed by another. If that understanding is incorrect, I'm wondering
what else it means or is intended to mean.

  "... for applications which use this 
  encoding (or ASCII encoding) it is possible to multiplex uncompressed
  application messages and SigComp messages on the same port."

In addition, the above text works - in my mind - at least for case
00, 01, and 10 listed below. Note I assume we are talking about application
messages that start with UTF-8 encoding (my interpretation of "use" in
above text). Also, I try to visualize the issue in two dimensions: encoding
of application messages and transport below SigComp.

					message-based	stream-based
					transport		transport
					(e.g. UDP)		(e.g. TCP)

  contain no binary data	case 00		case 01

  contain binary data		case 10		case 11

Here is why/how I see it works for case 00-10:

- For case 00 and 10, RFC 3320 says "In case of a message-based
transport such as UDP, a SigComp message corresponds to exactly one 
datagram." So, it is trivial for the receiver to tell if a UDP packet
carries uncompressed message or sigcomp message by inspecting 
the first byte.

- For case 01, the receiver can detect the beginning of a SigComp
message by bit pattern "11111" and the end by END-MESSAGE instruction.
Of course, the SigComp message delimiter also works for this purpose. 

As to case 11, I think we can also make it work, if sigcomp receiver
can detect the end of an uncompressed message. Obviously, that means 
the receiver needs to have some knowledge about the application 
message format. This is why I separate case 11 from the rest (which
are application independent). 

Now come to the question if the multiplexing is necessary. I think it
is useful in cases where a sigcomp sender does not want to compress 
every application message even though the other end has indicated its 
support of sigcomp. The reason could be CPU load of the compressor, 
message uncompressible, etc.

I could be wrong by thinking the multiplexing works. But if it indeed
works, the next question is whether RFC 3320 already covers some of 
this or we add extension to RFC 3320.

BR,
Zhigang

> -----Original Message-----
> From: ext Carsten Bormann [mailto:[email-masked]]
> Sent: March 28, 2003 9:24 AM
> To: [email-masked]
> Cc: Carsten Bormann
> Subject: Re: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP
> messages
> 
> 
> ROHCers,
> 
> in San Francisco, I promised to come back to the "binary multiplexing"
> issue after the IETF.
> 
> It has taken me a while to even understand what is being talked about:
> There is an assumption in this whole thread that I don't know where it
> came from.
> 
> We designed SigComp messages so that they always start with a byte
> that cannot be used as a coding symbol in UTF-8 (see
> draft-yergeau-rfc2279bis-04.txt for the updated version of RFC2279
> that defines the contemporary version of UTF-8).  The assumption was
> that most signalling protocols would use UTF-8 headers, so this would
> allow some self-describing-ness of SigComp messages.
> 
> RFC3320 says:
> 
>     All SigComp messages contain a prefix (the five 
> most-significant bits
>     of the first byte are set to one) that does not occur in UTF-8
>     encoded text messages [RFC-2279], so for applications 
> which use this
>     encoding (or ASCII encoding) it is possible to multiplex 
> uncompressed
>     application messages and SigComp messages on the same port.
>     Applications can still reserve a new port specifically for SigComp
>     however (e.g., as part of the discovery mechanism).
> 
> I now see that this might be phrased a bit misleading
> ("multiplexing").  But clearly, the scheme discussed is about using
> the same port for two different *transports*: the raw transport and
> the SigComp-compressed transport.  Nowhere does it say that you would
> multiplex messages from these two *on the same connection*.  I don't
> believe this is possible or even necessary.
> 
> Record Marking (4.2.2) is used to delimit SigComp messages on stream
> transports (TCP, TLS).  The assumption, of course, is that the
> connection carrying these messages already has been disambiguated to
> run SigComp instead of the raw transport protocol.
> 
> This is the end of my main argument for this message.  If you disagree
> with it, please discuss it and not the interesting speculation below.
> 
> <div class="speculation">
> 
> It is somewhat conceivable to occasionally intersperse uncompressed
> messages in the SigComp record marking transport, but it remains clear
> that the record marking layer is in effect at all times:
> 
>     For a stream-based transport, the dispatcher delimits messages by
>     parsing the compressed data stream for instances of 0xFF 
> and taking
>     the following actions:
> 
> So, we are talking about a situation where the data flows to a SigComp
> dispatcher, not to an entity that still needs to find out whether we
> are running SigComp or not.
> 
> Could SigComp record marking be used to carry both compressed and
> uncompressed messages on one connection?  (Remember that this is not
> part of the standard, but of course could be part of an extension to
> RFC 3320.)
> 
> In principle, yes, but there is a problem (see below).  In any case,
> though, Content-Length headers or any other application message
> delimiting protocol would not enter the picture at all -- the SigComp
> record marking layer remains in effect throughout a SigComp
> connection.
> 
> Now the problem: If you don't have any other way to find out (e.g.,
> different port numbers for compressed and uncompressed), it becomes
> hard to decide whether such a compressed/uncompressed multiplexed
> stream is SigComp or not.  If it starts with a non-UTF-8 coding
> symbol, you know it's SigComp.  But if the first message you want to
> send on the connection happens to be uncompressed, you need another
> way of indicating that a SigComp transport is being used.  Of course,
> you could simply send 0xFFFF and trust that the receiver will discard
> empty messages.  So to make this multiplexing business work, you will
> have to add a mandate that says empty messages (or at least one 0xFFFF
> at the beginning of the connection) are discarded.
> 
> Of course, none of this applys to any hypothetical SigComp application
> that does not use UTF-8 headers in their messages -- the multiplexing
> would not work this way at all, period.  So you would devise another
> way of multiplexing, which I'm too lazy to do right now.
> 
> </div>
> 
> Oh, and we still have to do the work on the SIP binding for SigComp,
> which is completely unrelated to this whole thread.  There is no known
> problem with SigComp-compressing large binary layloads in SIP messages
> (except, maybe, that there won't be very good compression schemes for
> data that already have been compressed).
> 
> Gruesse, Carsten
>

---

From: [email-masked]
Date: Thu, 3 Apr 2003 11:29:23 -0600

Mark,
 
The problem I raised is how "to multiplex uncompressed application messages and SigComp messages on the same port" when application messages start with UTF-8 encoded text but contain binary data. For the problem itself, I don't assume where the demultiplexing is done (that goes to the solution space, see below). My point is that it needs to be done somewhere at the receiving endpoint (which by definition, is "one instance of an application, a SigComp layer, and a transport layer").
 
Now, the solutions:
 
- One is to let decompressor dispatcher do the demultiplexing (I won't repeat how it can be done), which means all bytes received on a port must go through SigComp layer first. This follows the SigComp architecture in RFC 3320. It makes the layering more clean, i.e., (de)compression and (de)multiplexing is handled completely by SigComp layer. The drawback is a bit redundancy in the sense that the decompressor dispatcher performs some parsing that the application layer will later do again. 
 
- The other one is to let the application do the demultiplexing as you described below. That means all received data must go through application first.
 
In my view, either one will work and the choice is a local implementation issue (although the latter is not described in RFC 3320). The difference boils down to where do put the demultiplexing function. It's a matter of preference of SW coding. However, what is NOT an implementation issue is that - looking at transport layer - a receiving endpoint must expect it may receive uncompressed messages interspersed with SigComp messages on the same port and demultiplex them using either of the above approaches (or any other working solutions). This needs to be spelled out in a document.
 
BR,
Zhigang

-----Original Message-----
From: ext Mark Watson [mailto:[email-masked]]
Sent: April 03, 2003 8:24 AM
To: Liu Zhigang.C (NRC/Dallas); Sugar Robert (NET/Budapest)
Cc: [email-masked]; [email-masked]
Subject: RE: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP messages

Zhigang, 

If I understand the solution described by Robert correctly, then the problem you describe below does not arise. Your problem seems to assume that demultiplexing of compressed and uncompressed messages will be done before the SIP parser is invoked, based on the 11111xxx delimiter.

My understanding is that messages are passed directly to the SIP parser. This software must be aware that SIGCOMP may be used, and checks the first octet of the input stream. If this is 11111xxx then it uses the sigcomp delimeter OxFFFF to find the end of the compressed message, and then sends it to the decompressor. If the first octets is not 11111xxx then it just starts parsing as normal.

The only assumption is that the SIP parser has its own way of recognising the end of an uncompressed message. 

Regards...Mark 

> From: [email-masked] [ mailto:[email-masked]] 
> Sent: 01 April 2003 14:44 
> To: [email-masked] 
> Cc: [email-masked]; [email-masked] 
> Subject: RE: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP 
> messages 
> 
> 
> Robert, 
> 
> Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping 
> This list is for NEW development of the application of SIP 
> Use [email-masked] for questions on current sip 
> Use [email-masked] for new developments of core SIP 
>

---

From: [email-masked]
Date: Thu, 3 Apr 2003 14:19:31 -0600

> The mechanism in
> SigComp is not there to allow multiplexing of SigComp
> messages and raw messages within the same connection, 
> but only to identify whether the connection uses
> SigComp or not. 

If by "multiplex uncompressed application messages and
SigComp messages on the same port" RFC 3320 does not
mean what I thought it meant, I'd propose to add 
mutltiplexing (in the true meaning) capability as an
extension to RFC 3320. 

For details and arguments, please see my previous email:
http://www1.ietf.org/mail-archive/working-groups/rohc/current/msg01423.html

BR,
Zhigang

---

From: [email-masked]
Date: Thu, 3 Apr 2003 15:56:54 -0600

Richard,

> Regarding point a), I don't see how this can be done without
> talking about specific protocols. 

I agree if you mean the exact logic (see below). But what I had in 
mind for a) is a generic or abstract logic as follows:

0. Endpoint B informs endpoint A that it supports SigComp AND
multiplexing of uncompressed message and SigComp message on
a TCP connection. This should be part of the SigComp discovery.

1. Endpoint B reads in one byte. 

2. If the first 5 MSB of the byte is 11111. this is the beginning
of a SigComp message or a SigComp message delimiter (0xFFFF). 
Endpoint B enters "SigComp parsing mode" and processes byte stream
according to SigComp until it encounters END-MESSAGE instruction 
or Sigcomp delimiter. When that happens, go to 1. 

3. Else, this is the beginning of an uncompressed message. Endpoint
B enters "uncompressed parsing mode" and parses the byte stream 
according to the delimiting scheme of the application messages. 
When reaching the end of application message, go to 1. 

Note: it is assumed that uncompressed application messages start
with UTF-8 encoded character. Otherwise, the logic does not work.

Note: the sigcomp can deliver the bytes to application layer either as 
they are arriving or at the end of the application message.

Note: this abstract demultiplexing logic can be either done at
SigComp layer (decompressor dispatcher) or at application layer.
That is a choice of local implementation.

Now, back to your comment. I certainly agree with you that detecting 
end of application message in Step 3 depends on a particular application 
protocol. But other than that, the rest is pretty generic. We can 
consider the above as the abstract logic that will be "instantiated" 
when binding SigComp to a particular application.

> Regarding point b), it might be good to have some recommendations
> on how to implement an efficient compressor for messages with
> binary content. 

Yes.

> A suitable home for these would be the SigComp
> User Guide, which already talks about the various different
> compressors that can be used in conjunction with SigComp.

OK. I don't mind where it ends up as long as the issue finds
a home. I can provide some text.

BR,
Zhigang