Subject: [rohc] Comments on sigcomp-extended-02

From: "Price, Richard" <[email-masked]>
Date: Mon, 4 Mar 2002 17:29:35 -0000

Hi Hans,

I've attached some comments on sigcomp-extended-02, available from
http://www.dmn.tzi.org/ietf/rohc/draft-ietf-rohc-sigcomp-extended-02.txt.

General question: how will the compressor ensure that the decompressor can
handle SigComp messages using the extended operation mechanisms?

So far we have three suggestions for how this can be achieved:

1   Reserve a bit in the base SigComp header to indicate that the extended
    mechanisms are in use. The decompressor can then ignore the part of the
    SigComp message devoted to extended operations if it doesn't understand it.

2   Add optional functionality to basic SigComp, and indicate support for this
    functionality using the announcement mechanism (by upping the UDVM_version).

3   Describe the extended operation mechanisms in UDVM bytecode so that they
    can be advertised using the announcement mechanism (by sending the
    appropriate state identifier) or downloaded to the decompressor as needed.

I'd like to avoid Option 1 if possible, because it wastes quite a lot of
bandwidth (19 bytes per compressed message) in the case that the compressor
supports extended operations but the decompressor does not.

Option 3 is more flexible that Option 2, as it allows the compressor to
download bytecode for the extended operations to the decompressor in the case
that the code is not already available. However, this is only possible for
the extended operation mechanisms that can be implemented as bytecode using
whatever instructions are already available.

Are there currently any extended mechanisms that *can't* currently be
implemented as bytecode? I think that the functionality of the 'r' bit and
the 'b' bit can be provided without upgrading the UDVM (I explain how to do
this below) - is there any other functionality that is required?

Some specific comments follow:

Page 1: "This document is a submission of the IETF ROHC WG. Comments
should be directed to its mailing list, [email-masked]."

Replace [email-masked] with [email-masked].

Page 3: "Shared compression

     Compression relative to messages received by the local associated
     decompressor prior to the current compressed message."

I'd replace the term "local associated decompressor" with "local endpoint",
because for security purposes SigComp doesn't have a well-defined association
between a compressor and a decompressor. In fact, a new decompressor is
invoked for each SigComp message received by the endpoint.

Page 4: Architectural view of feedback.

In Figure 1 it would be very useful to point out that the local decompressor-
compressor interfaces all pass through the state handler. This is
important for security; the state handler determines which compressor is
associated with a particular decompressor so that the feedback data can be
forwarded correctly.

Page 7: "For security reasons as explained in SigComp it is RECOMMENDED
that this particular feedback is piggybacked and not sent standalone."

We shouldn't make the above text prescriptive, because the final decision
on whether standalone messages should be accepted or banned depends on the
transport layer. When running over UDP they are banned, but when running
over a secure transport layer then they are OK.

Page 8: Implicit deletion when creating a new state.

The base SigComp draft contains a very simple algorithm for deleting state
when memory is full. If an endpoint wants to establish new state and there
isn't enough memory, the oldest items of state are deleted to free up enough
room for the new item. See Section 6.2 for further details.

Page 9: "The shared_identifier as described in Section 2, and Section 5.2,
MUST be conveyed when shared compression is applied."

I wouldn't use the term "MUST" in the above phrase because it isn't an
interoperability issue. In fact the phrase is a tautology, since shared
compression is *defined* to be the case in which compression is performed
relative to the shared_identifier.

Some other uses of the term "MUST" could also be deleted throughout the
draft.

Page 10: Example of SigComp message for extended operations.

The basic SigComp message now has a length field to specify the size of the
state_identifier. This goes at the very front of the SigComp message.

Page 12: Format of the extended feedback.

Is it necessary to extend the announcement data by adding support for the
'r' bit? Why not calculate an MD5 hash over the uncompressed message at the
decompressor, and then advertise this hash using one of the id_value fields
already provided? This gives the same functionality but can already be
implemented as UDVM bytecode, without needing an additional announcement
field to be added.

Page 13: Acknowledgement optimization.

I think that in the very latest SigComp draft it might be possible to make
this optimisation without requiring the 'b' bit to be set.

Suppose that the shared_identifier has been used by compressor 2 but the 'b'
bit is set to 0. This is only possible if the announcement data that
indicated support for the shared_identifier was accepted, but the state
creation request for state (s1) failed for some reason. However the current
SigComp draft bans this possibility: if the state creation request fails then
the announcement request MUST fail also.

So I don't see a need for the 'b' bit ever to be set to 0.

Regards,

Richard

---

From: "Hans Hannu (EPL)" <[email-masked]>
Date: Tue, 5 Mar 2002 11:33:51 +0100

Hi Richard,

Thanks for your comments they are highly appreciated!

> handle SigComp messages using the extended operation mechanisms?

After the discussions that have been on the list, I assumed that this should be handled by the appropriated uploaded UDVM byte-code. 

* For explicit acknowledgements (Section 5.1) this is about writing the appropriate "header parser" in UDVM byte code, i.e. copying the acked_identifiers from the "header" to the feedback location, which is then passed to the compressor by the END-MESSAGE instruction.

* For shared compression (Section 5.1) there are two items to be addressed:
1) A compressor that has used shared compression for the current compressed message. Then the shared identifier is carried in the "header" and can be read into the UDVM by the using the STATE-EXECUTE instruction. 
2) For sending the "I was remembered"- bit (the 'r'-bit) to advertise that this decompressed message can be used for shared compression, a similar code as for explicit acknowledgements can be used.

For efficiency reason it is preferable if an endpoint announce its support for explicit acknowledgements and shared compression. However, if an announcement is not done it is still possible to run these two extended mechanisms. An endpoint will soon notice whether the other endpoint supports these two mechanisms. Until it notices that the other endpoint does not make use of the explicit acknowledgements, one or perhaps two message may have carried an acked_identifier, and for shared compression one extra bit (the r-bit), may have been carried in one or perhaps two messages. 

Perhaps Zhigang could answer for the mechanisms in Section 5.5 and Section 5.6?

>     appropriate state identifier) or downloaded to the 

Well, I also like option 3.
 
> implemented as bytecode? 

I'm not sure about Section 5.5 and Section 5.6.

> interoperability issue. 
>In fact the phrase is a tautology, 
> since shared

Agree. A compressor must make sure that a message that it has compressed can be decompressed and therefore it must send along the shared_identifier. This is basics so I'll make the change.
 
> implemented as UDVM bytecode, without needing an additional 

I'm not sure I follow you correctly, hear me out:
A message m is sent from compressor 1 to decompressor 2. Message m is saved as a shared state at endpoint 1, and is indicated to endpoint 2 using the 'r-bit'.
1) Is it decompressor 2 that calculates the MD5 hash over the uncompressed message? If yes there are two alternatives
1.1) If yes, then we need to add an additional instruction to the UDVM, which calculates an MD5 hash over a certain range of the UDVM memory, so that this information is available when the UDVM is done, i.e. the information can be placed into the feedback location passed with the END-MESSAGE instruction.
1.2) Or, the feedback information is altered by decompressor 2 after it has been passed from the UDVM, i.e. the decompressor adds the hash for the uncompressed message before the feedback information is passed to the local compressor.

Of course message m must be made available for compressor 2 if it is supposed to use shared compression. But, that is up to the implementation of the SigComp end-point.

I'm not sure if this is a good approach, isn't it just as easy to use the 'r' bit?

> So I don't see a need for the 'b' bit ever to be set to 0.

I agree, as the SigComp drafts bans this then the 'b' bit is never set to 0. For simplicity of the implementation we should also include the shared_identifier along with the 'b' bit in the feedback data.

BR
/Hans H

_______________________________________________
Rohc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rohc

---

From: "Price, Richard" <[email-masked]>
Date: Tue, 5 Mar 2002 13:59:57 -0000

> Thanks for your comments they are highly appreciated!

You're welcome!

> A message m is sent from compressor 1 to decompressor 2. 
> Message m is saved as a shared state at endpoint 1, and is 
> indicated to endpoint 2 using the 'r-bit'.
> 1) Is it decompressor 2 that calculates the MD5 hash over the 
> uncompressed message?

Yes, we need to calculate the MD5 hash at decompressor 2 in order to save
the 9 bytes of unnecessary overhead.

> If yes there are two alternatives
> 1.1) If yes, then we need to add an additional instruction to 
> the UDVM, which calculates an MD5 hash over a certain range 
> of the UDVM memory, so that this information is available 
> when the UDVM is done, i.e. the information can be placed 
> into the feedback location passed with the END-MESSAGE instruction.

This is my proposal. If we have an "MD5" instruction in the UDVM then we
can avoid the need for an 'r' bit because we can calculate the exact
hash that the 'r' bit would acknowledge.

> 1.2) Or, the feedback information is altered by decompressor 
> 2 after it has been passed from the UDVM, i.e. the 
> decompressor adds the hash for the uncompressed message 
> before the feedback information is passed to the local compressor.

I'm not proposing the above, because it would still require an
explicit 'r' bit to be defined in the announcement information.

> Of course message m must be made available for compressor 2 
> if it is supposed to use shared compression. But, that is up 
> to the implementation of the SigComp end-point.
> 

The drawback of having an explicit 'r' bit in the announcement
information is that due to IPR concerns, the functionality is optional
at the decompressor. So even if the compressor downloads the correct
bytecode to use the 'r' bit, it runs the risk that the information
will simply be discarded because the decompressor doesn't know how to
use it.

The advantage of replacing the 'r' bit with an MD5 instruction is that
there's no (known) IPR on calculating an MD5 hash per se, so a
compressor that downloads the bytecode for shared mode can guarantee
that it will be supported at the decompressor.

> > I don't see a need for the 'b' bit ever to be set to 0.
> 
> I agree, as the SigComp drafts bans this then the 'b' bit is 
> never set to 0. For simplicity of the implementation we 
> should also include the shared_identifier along with the 'b' 
> bit in the feedback data.

If the 'b' bit is always set to 1, then we can implement the
acknowledgement optimisation of Section 6.2.2 without relying that
the decompressor knows about 'b' bit functionality. This is the same
(good) situation as we have for the 'r' bit - a compressor can always
use the optimisation without needing to rely on extra functionality
being available at the decompressor.

Regards,

Richard

---

From: "Hans Hannu (EPL)" <[email-masked]>
Date: Tue, 5 Mar 2002 15:49:38 +0100

Hi Richard,

> hash that the 'r' bit would acknowledge.

Ok, but to totally get rid of the 'r' bit in the feedback data, the "MD5" instruction must also make a request for a state create for the range of the UDVM memory it is calculated over. Otherwise we still need to indicate to the endpoint, using an 'r'-bit, that this current decompressed message may be used for compression.

Best Regards
/Hans H

---

From: "Price, Richard" <[email-masked]>
Date: Tue, 5 Mar 2002 15:46:16 -0000

> Ok, but to totally get rid of the 'r' bit in the feedback 
> data, the "MD5" instruction must also make a request for a 
> state create for the range of the UDVM memory it is 
> calculated over. Otherwise we still need to indicate to the 
> endpoint, using an 'r'-bit, that this current decompressed 
> message may be used for compression.

Hi Hans,

Would the above be a problem? One minor issue is that the
current SigComp draft only allows one state creation request
per message, but this is a restriction I was going to remove
anyway once I got around to it. Carsten suggested having a
maximum limit of 4 state requests per message, which would be
more than enough to support shared mode.

Regards,

Richard

---

From: "Hans Hannu (EPL)" <[email-masked]>
Date: Wed, 6 Mar 2002 14:00:18 +0100

Hi Richard,

I do not think that being able to make an additional state creation request per message should cause implementation problems. However, a limit as suggested by Carsten is probably a good idea.

BR
/Hans H

---

From: [email-masked]
Date: Wed, 6 Mar 2002 11:33:15 -0600

> The base SigComp draft contains a very simple algorithm for 
> deleting state when memory is full. If an endpoint wants to 
> establish new state and there isn't enough memory, the oldest
> items of state are deleted to free up enough room for the 
> new item. See Section 6.2 for further details.

Richard,

I would say section 6.2 in base SigComp draft uses the similar
idea as that of section 5.6 in extended draft. But they operate
at different level. Section 5.6 works within a state. It 
addresses not only the memory constraint but also the freshness
of the dictionary content within a state. So, I don't think
they are mutual exclusive. 

BTW, the text in section 5.6 is intended to be generic. In the 
simplest form, the algorithm can be FIFO (as in section 6.2 in
the base draft). This should have been mentioned as an example.

BR,
Zhigang

---

From: "Price, Richard" <[email-masked]>
Date: Thu, 7 Mar 2002 15:09:18 -0000

Hi Zhigang,

I agree that the simple FIFO algorithm used in the base SigComp draft
does not preclude the use of a more advanced state deleting algorithm,
if one happens to be available (e.g. the "checkpoint state" algorithm
as you describe in sigcomp-extended).

Of course, a copy of the more advanced algorithm will need to be made
available at both the compressor and the decompressor before it can be
used. Do you have any thoughts on how to do this? Could it be achieved
by providing a suitable UDVM instruction (e.g. the UDVM could mark each
state with a priority value that determines the order in which states
are deleted?)

Would there be any IPR concerns on including the above UDVM instruction
within the base SigComp draft?

Thanks,

Richard

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> Sent: Wednesday, March 06, 2002 5:33 PM
> To: Price, Richard; [email-masked]
> Cc: [email-masked]
> Subject: RE: [rohc] Comments on sigcomp-extended-02
> 
>

---

From: [email-masked]
Date: Thu, 7 Mar 2002 18:59:47 -0600

Hi Richard,

Comments inline.

BR,
Zhigang

> From: ext Price, Richard [mailto:[email-masked]]
> Sent: March 07, 2002 9:09 AM
> To: Liu Zhigang.C (NRC/Dallas); [email-masked]
> Cc: [email-masked]

Some additional clarifications. Section 5.5 in extended SigComp draft 
is more than just a state deleting scheme. It also provides a 
solution for state synchronization when SigComp runs over unreliable 
transport, which is missing in the base draft. In fact, the simple FIFO 
algorithm in the base draft may end up with different set of states 
between compressor and decompressor if message misordering happens. 

> by providing a suitable UDVM instruction (e.g. the UDVM could 
> mark each state with a priority value that determines the order in which states
> are deleted?)
> 
> Would there be any IPR concerns on including the above UDVM 
> instruction within the base SigComp draft?

I'm not sure about the answer. There might be. But how is the explicit 
acknowledgement handled in this regard? The situation for section
5.5 is very similar to explicit ACKs. In fact, the procedure includes
explicit ACKs.

---

From: "Hans Hannu (EPL)" <[email-masked]>
Date: Fri, 8 Mar 2002 10:01:42 +0100

Hi Zhigang, Richard,

> In fact, the 
> simple FIFO 

True, this may happen, and this may be a problem if explicit acknowledgements are not received, i.e. the compressor may not advance to use the newest state in the compression process. 

However, a simple note to Section 6.2 of the base draft may limit the problem 

"The state handler keeps track of which endpoint created each item of
state; when a particular endpoint exceeds its allocated memory limit
then sufficient items of state created by the same endpoint are
deleted (oldest state first) until enough memory is available to
accommodate the new state.
Note: the state handler must not delete the items of state that was most recently used by a particular endpoint."

BR
/Hans H

---

From: "Price, Richard" <[email-masked]>
Date: Mon, 11 Mar 2002 16:42:13 -0000

> between compressor and decompressor if message misordering happens.

As with so much of SigComp, the amount of robustness provided against
message misordering is left as an implementation decision at the compressor.

Suppose that we want to be robust against messages arriving up to n - 1
messages out of sequence, and suppose also that the compressor sends out
the following message flow:

Message 1:   References an item of state (and creates new state)

Message 2:   Creates new state
    :                 :
Message n:   Creates new state

In the worst-case misordering scenario, Message 1 will arrive at the
decompressor just after Message n. In order to ensure that Message 1 can
still be successfully decompressed, the compressor must not create so much
state in the following messages that the state referenced in Message 1
is deleted.

In practice the above rule is easy to enforce, because a UDVM algorithm
often just dumps the contents of the UDVM memory as state after each message
and retrieves it in the following message. For a simple algorithm of this
nature, robustness against messages arriving up to n - 1 messages out of
sequence is achieved when the application-defined parameter
maximum_state_size is set to n.

The main restriction in the current SigComp draft is that all state is
created equal - each item of state has a fixed lifetime and there is no way
to protect certain important items of state that might be referenced by many
subsequent messages. It would be nice to add this ability if we can find a
way to include it without raising any IPR issues.

> 5.5 is very similar to explicit ACKs. In fact, the procedure includes

Explicit ACKs are handled by offering a very general UDVM instruction to
send some "announcement" information from Endpoint A to Endpoint B. If
Endpoint A sends a state identifier to Endpoint B then it is indicating
that it has made the corresponding item of state available.

The above mechanism is sufficiently general to avoid Hans' IPR concerns
on explicit ACKs; in fact there are many other uses for the mechanism,
such as indicating support for a range of well-known decompression
algorithms. However it can be used to implement an explicit ACK scheme
just by downloading the correct UDVM bytecode.

It would be great if we could do the same for state deleting algorithms.
Would it be possible to provide a generic UDVM instruction for changing
the order in which state is deleted, so that there are no IPR concerns
on the instruction itself but it could be used to implement the
checkpoint state algorithm as a piece of UDVM bytecode?

Regards,

Richard

---

From: [email-masked]
Date: Wed, 13 Mar 2002 19:59:17 -0600

Richard,

What I have in mind is to use one of the reserved bit in Figure 4
of extended-02 as follows (perhaps need to update other fields too):

     0   1   2   3   4   5   6   7
   +---+---+---+---+---+---+---+---+
   | 1   1   1   1   1 |  length   |
   +---+---+---+---+---+---+---+---+
   |                               |
   :   state_identifier (n-bytes)  :
   |                               |
   +---+---+---+---+---+---+---+---+
   | s | a | r | b | c | Reserved  |
   +---+---+---+---+---+---+---+---+
   |                               |
   : shared_identifier (n-bytes)   : Present if 's' is set
   |                               |
   +---+---+---+---+---+---+---+---+
   |                               |
   :  acked_identifier (n-bytes)   : Present if 'a' is set
   |                               |
   +---+---+---+---+---+---+---+---+
   |           Possible            |
   :        Compressed data        :
   |                               |
   +---+---+---+---+---+---+---+---+

(snip)

   'c' : if set to 1, indicates the state created by this message
         is a checkpoint state (see section 5.5). Note: the bit is
         ignored if this message does not create a new state.

However, I think we can add STATE_FREE(%id_start, %id_length) to
the base draft as suggested by Adam during offline discussion 
two weeks ago. It allows the compressor to explicitly delete
a state on the decompressor side. Security issue has been discussed
and to my understanding there is no new risk. 

BR,
Zhigang

---

From: "Hans Hannu (EPL)" <[email-masked]>
Date: Thu, 14 Mar 2002 09:20:22 +0100

Hi Zhigang, Richard,

Just a reminder, Figure 4 of extended-02 is an Example of a SigComp message supporting explicit acknowledgements and shared compression. The extended SigComp message except for the first byte and the state_identifier is an implementation issue.   

[Zhigang wrote:]

>          ignored if this message does not create a new state.

The issue for standardization is whether the announcement format needs to be modified to provide the functionality of Section 5.5, i.e. carry a 'c'-bit to the state-handler. 

For shared compression the issue of providing the r-bit functionality without modifying the announcement format has been solved by changing the MD5 instruction to also do a request for a state creation, see message: 
https://www1.ietf.org/mail-archive/working-groups/rohc/current/msg00152.html .

I guess Richard's question is about whether we can do something similar for the 'c-bit'.

BR
/Hans H

---

From: "Jan Christoffersson (EPL)" <[email-masked]>
Date: Thu, 14 Mar 2002 09:53:54 +0100

Hi Zhigang and others,

I really fancy the following suggestion you have in your posting:

''However, I think we can add STATE_FREE(%id_start, %id_length) to the base draft as suggested by Adam during offline discussion two weeks ago. It allows the compressor to explicitly delete a state on the decompressor side. Security issue has been discussed and to my understanding there is no new risk. ''

In fact, a similar idea was raised in http://www.cdt.luth.se/rohc/msg03652.html, but at the time it seemed to be impossible due to security considerations. 

Are there any new facts of what can be done from the security perspective?

In any case, if this is allowed i think we should definitely include it in the draft.

Regards 
/Jan

---

From: [email-masked]
Date: Thu, 14 Mar 2002 10:58:07 -0600

> Currently the base draft just deletes state using a FIFO algorithm,
> but having an explicit STATE-FREE instruction would be even better
> provided that there are no IPR issues. 

At least I'm not aware of IPR claims on this STATE-FREE instruction 
alone. But I don't know if other people are.

> I believe that SigComp-05
> prevents this from adding new security risks. The reason for this
> is that STATE-FREE is only a request to delete state rather than
> an explicit command; the actual state deletion can only be carried
> out with the permission of the application.

Yes, that's my understanding of the offline discussion too. Every
transaction must be approved by the application and we put all
the security burden on the application.

However, now I'm thinking about the case where the application 
has no security mechanism (e.g. no authentication) and thus
cannot tell if a message was sent by a malicious endpoint. 
The STATE-FREE in my view does introduce new vulnerability
and should not be used. Or equivalently, we require the application
must ignore STATE-FREE request if it is unsure.

(Hmm, this leads me to an even a bigger question. More to come.)

BR,
Zhigang

---

From: "Price, Richard" <[email-masked]>
Date: Thu, 14 Mar 2002 17:16:28 -0000

> must ignore STATE-FREE request if it is unsure.

Hi,

I believe that this is already covered by SigComp-05. State and
announcement data can only be saved if the application provides the
identity of the sending endpoint; if it can't do this then the data
is discarded. The STATE-FREE instruction would just create a
particular kind of announcement data, which would be discarded in
the same manner if the application didn't have a security mechanism.

Regards,

Richard

---

From: [email-masked]
Date: Thu, 14 Mar 2002 20:13:00 -0600

Hi Hans,

> Just a reminder, Figure 4 of extended-02 is an Example of a 
> SigComp message supporting explicit acknowledgements and 
> shared compression. The extended SigComp message except for 
> the first byte and the state_identifier is an implementation issue.   

Thanks. Then, what I proposed is just another version of the example.
A side question: why do we have both above example and section 6.2.1?

> The issue for standardization is whether the announcement 
> format needs to be modified to provide the functionality of 
> Section 5.5, i.e. carry a 'c'-bit to the state-handler. 
> 
> For shared compression the issue of providing the r-bit 
> functionality without modifying the announcement format has 
> been solved by changing the MD5 instruction to also do a 
> request for a state creation, see message: 

I actually have a few questions on the new solution.
Does that mean MD5 instruction always triggers a request to 
create a state? Do we want that? If not, need to add another
operand to MD5?

Either way, how does the decompressor know that a 
received message should be treated as carrying
'r' bit or not? 
 
> I guess Richard's question is about whether we can do something similar for the 'c-bit'.

I'm still trying to understand how we remove r-bit and b-bit.
If that is clear, I think we can use similar solution for c-bit.

BR,
Zhigang

---

From: "Hans Hannu (EPL)" <[email-masked]>
Date: Sun, 17 Mar 2002 05:04:33 +0100

Hi Zhigang,

Trying to catch up with the emails...

> A side question: why do we have both above example and section 6.2.1?

Sometimes examples are good, and I thought so in this case.
 
> create a state? Do we want that? If not, need to add another

I think you have a point we should add another operand to MD5 to indicate 
whether a state should be created or not, and also a return indication 
whether the creation was successful or not. The latter could be done by 
adding yet another operand or stating that the returning hash value is set 
to "zero?" on failure.

> received message should be treated as carrying

The compressor would upload the UDVM with the appropriate "SigComp 
message parser".

BR
/Hans H

---

From: [email-masked]
Date: Sat, 16 Mar 2002 23:48:11 -0600

> The compressor would upload the UDVM with the appropriate "SigComp 

(Just to be sure I get it right.)
So, we still have 'r' bit in SigComp message *if* the sender wants
to implement the optimized ACK. The change is that the position
and format of 'r' bit now becomes implementation issue (the UDVM code
can parse it). It's no different than any other bits in a message.

Then, I think 'c' bit can be treated the same way. However,
we need to add an operand to instructions that create state.
The operand indicates if newly created state is a checkpoint. 
(I remember Richard mentioned similar point.)

BR,
Zhigang

---

From: "Hans Hannu (EPL)" <[email-masked]>
Date: Sun, 17 Mar 2002 15:57:29 +0100

> can parse it). It's no different than any other bits in a message.

Yes, it is an implementation issue.

BR
/Hans H