Subject: [rohc] Dynamic update of resources in SigComp

From: [email-masked]
Date: Thu, 5 Jun 2003 09:23:22 +0300

Hi,

I would like to get the clarification about the the dynamic
update of resources in the SigComp endpoints (I could not
find the clear answer in the RFC3320).

Is it possible that two SigComp endpoints using dynamic
compression updates i.e. decrease or increase sms,
dms or cpb values dynamically? And if it possible, how
the compartments can be synchronized?

best regards,
	paulius
_______________________________________________
Rohc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rohc

---

From: "Lajos Zaccomer (ETH)" <[email-masked]>
Date: Thu, 5 Jun 2003 08:34:20 +0200

Hi,

The updates of the parameters may happen at any time with the help of the returned parameters, but an endpoint is not allowed to decrease its parameters, only to keep the originals or to increase them.

As you cannot force another endpoint to send its parameters, true synchronization is impossible (lack of signalling - NACK was partly introduced to eliminate this shortcoming). If a compartment is out of synchron, the only thing it can do is to fall back to the initial state, just as if nothing has happened before.

BR/Zacco

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: 2003. június 5. 8:23
To: [email-masked]
Subject: [rohc] Dynamic update of resources in SigComp

Hi,

best regards,
	paulius

---

From: "Price, Richard" <[email-masked]>
Date: Thu, 5 Jun 2003 14:59:23 +0100

> The updates of the parameters may happen at any time with the 
> help of the returned parameters, but an endpoint is not 
> allowed to decrease its parameters, only to keep the 
> originals or to increase them.

Hi,

This was true for an early version of SigComp, but we remembered
to fix it before publishing the final RFC ;-)

An endpoint can now choose to reduce the amount of resources that
it offers on-the-fly, provided that it does so in a robust manner
(i.e. it needs to make sure that the remote endpoint knows that
the available resources have been reduced, before it actually
reclaims them).

One possible way to keep the endpoints in sync is to use the
SigComp Feedback mechanism. In this case, the following two steps
need to be taken:

1. The endpoint sends a message announcing the reduced resources,
   (but does not make the actual reductions just yet). It also
   requests an acknowledgement of the message to be returned via
   the SigComp Feedback mechanism.

2. When the acknowledgement is returned, the endpoint knows that
   the original message has been successfully received. It can then
   safely reclaim the resources without causing decompression
   failures to occur.

> As you cannot force another endpoint to send its parameters, 
> true synchronization is impossible

True synchronisation is always impossible over an unreliable
transport, but the SigComp Feedback mechanism is designed to
ensure that this isn't a problem. It works much like R-mode of
RFC 3095, using the following general rule:

   "Don't assume that any data has arrived until you get an
    explicit acknowledgement."

So, for example, a compressor should not attempt to access a state
item unless it receives an explicit acknowledgment that the state
has been created by the remote endpoint.

> (lack of signalling - NACK was partly introduced to eliminate
> this shortcoming).

The volume of signalling traffic is under the control of the
compressor. If it wishes, it can use the SigComp Feedback
mechanism to request the explicit acknowledgement of every single
compressed message.

The NACK mechanism can help endpoints to recover if they lose
synchronisation - however, the SigComp Feedback mechanism is
designed to prevent this from occurring in the first place!

> If a compartment is out of synchron, the only thing it can do
> is to fall back to the initial state, just as if nothing has 
> happened before.

True. Note however that a pair of correctly implemented SigComp
endpoints will lose synchronisation roughly once in every 2^48
compressed messages (assuming 6-byte hashes)! So if this happens
regularly, it's a good indication that there's something amiss...

Regards,

Richard

---

From: "Lajos Zaccomer (ETH)" <[email-masked]>
Date: Thu, 5 Jun 2003 16:36:32 +0200

Hi Richard,

See my comments and questions inline! Thank you for the answers in advance!

BR/Zacco

-----Original Message-----
From: Price, Richard [mailto:[email-masked]]
Sent: 2003. június 5. 15:59
To: Lajos Zaccomer (ETH); [email-masked]'; [email-masked]
Subject: RE: [rohc] Dynamic update of resources in SigComp

Hi,

>> Where is this specified in the RFC?

>> I didn't mean the unreliable layer. I meant that I cannot ask an another endpoint to send its parameters, as this activity is only a SHOULD, thus we cannot rely on this.

>> Having the ack does not mean that the state was successfully saved. It means that the sigcomp message was successfully decompressed and the application autenticated it. We have no information if the state was indeed saved.

>> I think that it was rather misleading to call the normal feedback remote compressor initiated explicit acknowledgement, as it is described in RFC 3321 as something new. But this is not the point. The normal feedback (or remote compressor initiated explicit acknowledgement) has the above detailed shortcoming. While the local endpoint initiated explicit acknowledgement has another shortcoming: there may be up to four state creates and only one of them can be explicitely acked. But unfortunately we don't know which one, as you said it was an implementation decision. In consequence, this is not reliable, too.

>> This is theory. In reality, there might be other problems (e.g. the feedback is sent back, but the sate could not be saved for the state database is full or for some reason not accessible, or there might be a corrupted message that the SIP parser accepts but the state will not match) and this probablity may be considerably higher.

Regards,

Richard

---

From: [email-masked]
Date: Thu, 5 Jun 2003 12:03:52 -0500

Richard,

> to fix it before publishing the final RFC ;-)

I also remember we had discussion whether to allow reducing resource
dynamically. Decreasing seemed trickier than increasing. So, at one
time, we disallowed it. But I cannot recall details how we reversed
it later? In particular, I couldn't find text in RFC 3320 on this,
or perhaps I missed it?

> has been created by the remote endpoint.

Right. But when an endpoint reduces its SMS, even previously acknowledged
state(s) may have to be deleted. Then the order of deletion is important
to maintain synchronization. Presumably, state_retention_priority should
be used for this purpose (i.e. lowest priority should be deleted first). 
But again, it this clear form RFC 3320? It only takes one or two sentences
to clarify.

BR,
Zhigang

---

From: "Jan Christoffersson (EAB)" <[email-masked]>
Date: Fri, 6 Jun 2003 08:43:11 +0200

Hi Folks,

As far as I remember, there is only one exception to the rule that an endpoint must not decrease its resources and that is if the compressor sets the 'I' bit to 1, then it's ok for the other endpoint to reduce (or set to zero) the state memory size. Apart from this, an endpoint should never decrease its resources.
As Zhigang says, reducing the available recources on the fly would make life much more complicated and this was the reason why the idea was that they should only be increased and never decreased!

However, looking through RFC 3320 I cannot find that this is very clearly written. In section 3.3 it says:
"Each parameter has a minimum value that MUST be offered by all
   receiving SigComp endpoints.  Moreover, endpoints MAY offer
   additional resources if available; these resources can be advertised
   to remote endpoints using the SigComp feedback mechanism."

Unfortunately, to me this is not clear enough. I agree with Zhigang that this should be made clearer. Perhaps this is something for the Implementors Guide to begin with?

Regards,
Jan

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: den 5 juni 2003 19:04
To: [email-masked]; Lajos Zaccomer (ETH);
[email-masked]; [email-masked]
Subject: RE: [rohc] Dynamic update of resources in SigComp

Richard,

BR,
Zhigang

---

From: "Jan Christoffersson (EAB)" <[email-masked]>
Date: Fri, 6 Jun 2003 09:57:44 +0200

Correction: it should be the 'S' bit, not the 'I' bit in my reply below :-)

/Jan

-----Original Message-----
From: Jan Christoffersson (EAB) 
Sent: den 6 juni 2003 08:43
To: [email-masked]'; [email-masked]; Lajos Zaccomer
(ETH); [email-masked]; [email-masked]
Subject: RE: [rohc] Dynamic update of resources in SigComp

Hi Folks,

Regards,
Jan

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: den 5 juni 2003 19:04
To: [email-masked]; Lajos Zaccomer (ETH);

Richard,

BR,
Zhigang

---

From: [email-masked]
Date: Fri, 6 Jun 2003 11:29:29 +0300

Hi all,

I agree with Jan that endpoint must never
decrease the resources.

I see one possible problem with Richard's implementation suggestion:

the client of the SigComp instructs the SigComp
to decrease the sms of the compartment for instance
due to low memory condition in the mobile terminal. 
But endpoint can free actually the resources after
it receives an ack from the remote endpoint. It is possible
that the endpoint can't decrease the resources in order
to avoid synchronization problems. IMO this should not
be allowed to happen in the mobile terminal.

cheers,
	paulius

> From: ext Jan Christoffersson (EAB)
> [mailto:[email-masked]]
> Sent: 06 June, 2003 10:58
> To: Jan Christoffersson (EAB); Liu Zhigang.C (NRC/Dallas);
> [email-masked]; Lajos Zaccomer (ETH); Meskauskas Paulius
> (NMP/Helsinki); [email-masked]
> As far as I remember, there is only one exception to the rule 
> that an endpoint must not decrease its resources and that is 
> if the compressor sets the 'I' bit to 1, then it's ok for the 
> other endpoint to reduce (or set to zero) the state memory 
> size. Apart from this, an endpoint should never decrease its 
> resources.
> As Zhigang says, reducing the available recources on the fly 
> would make life much more complicated and this was the reason 
> why the idea was that they should only be increased and never 
> decreased!
> 
> However, looking through RFC 3320 I cannot find that this is 
> very clearly written. In section 3.3 it says:
> Unfortunately, to me this is not clear enough. I agree with 
> Zhigang that this should be made clearer. Perhaps this is 
> something for the Implementors Guide to begin with?
> 
> Regards,
> Jan
>

---

From: "Lajos Zaccomer (ETH)" <[email-masked]>
Date: Fri, 6 Jun 2003 10:29:45 +0200

Hi,

This is the bit that someone called Stupid bit in SIPit. I found it felicity. I forecast that nobody will use this feature, because it requires additional logic in the bytecode, which of course makes it larger (as we know the difference is actually doubled regarding the UMS), and the ultimate goal when developing bytecodes is to make it very short.
If the user was punished in some way when it had neither used nor released its allocated resources after a while, it could be a successful scheme to force using the S bit.
We should not count on understanding competitors when using each other's resources. If it was expected, you could throw away all security aspects as well.

BR/Zacco

-----Original Message-----
From: Jan Christoffersson (EAB) 
Sent: 2003. június 6. 9:58
To: Jan Christoffersson (EAB); [email-masked]'; [email-masked]; Lajos Zaccomer (ETH); [email-masked]; [email-masked]
Subject: RE: [rohc] Dynamic update of resources in SigComp

/Jan

-----Original Message-----
From: Jan Christoffersson (EAB) 
Sent: den 6 juni 2003 08:43
To: [email-masked]'; [email-masked]; Lajos Zaccomer

Hi Folks,

Regards,
Jan

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: den 5 juni 2003 19:04
To: [email-masked]; Lajos Zaccomer (ETH);

Richard,

BR,
Zhigang

---

From: "Price, Richard" <[email-masked]>
Date: Fri, 6 Jun 2003 12:39:45 +0100

> >> Where is this specified in the RFC?

Hi Zacco,

The following is specified in Section 9.4.9 of RFC 3320:

   Moreover, the responsibility for ensuring that feedback data arrives
   successfully over an unreliable transport lies with the sender.  The
   receiving endpoint always uses the last received value for each field
   in the feedback data, even if the values are out of date due to
   packet loss or misordering.

So, if the receiving endpoint is told by the feedback data that the
amount of available resources has been reduced, it must ensure that it
doesn't use the resources any more (otherwise decompression failure
might occur).

> >> I didn't mean the unreliable layer. I meant that I cannot 
> ask an another endpoint to send its parameters, as this 
> activity is only a SHOULD, thus we cannot rely on this.

This is exactly how things should be, since it would be a very bad
idea to implement an endpoint which crashed every time the remote
endpoint failed to return a feedback message!

Note that even if we made returning the feedback a MUST, it would
still be possible for the message to be lost or corrupted while
travelling between the two endpoints.

> >> Having the ack does not mean that the state was 
> successfully saved. It means that the sigcomp message was 
> successfully decompressed and the application autenticated 
> it. We have no information if the state was indeed saved.

We can guarantee that a state item has been saved if we know two
things:

1. The message which created the state was successfully received,
   decompressed and authenticated.

2. Enough state memory was available to save the state.

We know that 1.) is true by receiving the ACK in the first place.
We know that 2.) is true by reading the state_memory_size parameter
in the feedback, and making sure that it is large enough. So the
feedback data contains all the information we need to guarantee
that the state was successfully saved.

> >> I think that it was rather misleading to call the normal 
> feedback remote compressor initiated explicit 
> acknowledgement, as it is described in RFC 3321 as something 
> new. But this is not the point. The normal feedback (or 
> remote compressor initiated explicit acknowledgement) has the 
> above detailed shortcoming.

I think that the feedback mechanism works fine (i.e. provides
a guarantee that state was saved), provided that you check the
state_memory_size parameter to make sure that the state wasn't
rejected due to insufficient memory.

> While the local endpoint initiated explicit acknowledgement has
> another shortcoming: there may be up to four state creates and
> only one of them can be explicitely acked. But unfortunately we
> don't know which one, as you said it was an implementation decision.
> In consequence, this is not reliable, too.

Local endpoint initiated acks can acknowledge more than one state
item (see Figure 13 of RFC 3320 - there is room for an arbitrary
number of state identifiers).

> >> This is theory. In reality, there might be other problems 
> (e.g. the feedback is sent back, but the sate could not be 
> saved for the state database is full or for some reason not 
> accessible,

This situation MUST be indicated using the state_memory_size
parameter in the returned SigComp parameters. If an endpoint
can't save the state for whatever reason (e.g. it's run out of
memory), then the state_memory_size MUST be set to 0. This
allows the remote endpoint to work out whether or not the state
was saved successfully.

> or there might be a corrupted message that the 
> SIP parser accepts but the state will not match) and this 
> probablity may be considerably higher.

Corrupted messages can be detected by a CRC or a hash over the
original uncompressed message. As usual with SigComp, the
endpoint which sent the message is free to choose how much
protection it wishes to provide against corrupt messages. For
example, if we use a 6-byte hash to protect the message then
only 1 in 2^48 corrupt messages will fail to be detected.

Regards,

Richard

---

From: "Price, Richard" <[email-masked]>
Date: Fri, 6 Jun 2003 13:13:27 +0100

> it later? In particular, I couldn't find text in RFC 3320 on this,

Hi Zhigang,

IIRC, we discussed this issue in one of the phone conferences before
last-calling the standard. The three main points which we all seemed
to agree on were as follows:

1. Reducing the amount of state memory on-the-fly is a Bad Thing!

2. There may be rare cases where an endpoint has no choice but to
   reduce the state memory (e.g. if it's broken).

3. If 2 occurs, it's better to let the remote endpoint know that the
   state memory is no longer available than to pretend that
   everything's still ok.

From the above, we decided not to disallow the possibility that an
endpoint might have to reduce its offered resources mid-call.
Instead, we decided that the receiver should always use the
resources offered in the last successfully authenticated feedback
data. From RFC 3320:

> But again, it this clear form RFC 3320? It only takes one or two sentences

I think that the order in which states are deleted is clear from RFC 3320:

      The order in which the existing state items are freed is
      determined by the state_retention_priority, which is set when the
      state items are created.  The state_retention_priority of 65535 is
      reserved for locally available states; these states must always be
      freed first.  Apart from this special case, states with the lowest
      state_retention_priority are always freed first.  In the event of
      a tie, then the state item created first in the compartment is
      also the first to be freed.

However, I agree that more clarification of the issue would be useful
in general. I think that it would be good to clarify at least the following:

*   Endpoints SHOULD make every effort to keep a constant amount of
    resources available for a compartment, as it's awkward to keep
    adjusting the resources on-the-fly.

*   If an endpoint really needs to reduce the amount of resources for
    a compartment, it SHOULD tell the remote endpoint that it has done
    so (using the SigComp Feedback mechanism).

*   The remote endpoint MUST use only the resources offered in the
    last successfully authenticated feedback data, even if this is
    less than was offered previously. However, since this behaviour
    is rare, there's no need to worry about getting a good compression
    ratio any more (e.g. it would be ok for an endpoint just to fall
    back to stateless compression, rather than worry about which
    states have been deleted and which are still available).

Regards,

Richard

---

From: "Price, Richard" <[email-masked]>
Date: Fri, 6 Jun 2003 13:40:08 +0100

> size. Apart from this, an endpoint should never decrease its 

Hi Jan,

I agree that an endpoint SHOULD never decrease the amount of
resources which it offers (unless the remote endpoint indicates
that the resources are no longer needed). However, if something
goes wrong and an endpoint is forced to reduce the available
resources for whatever reason, it's much better to do this in a
way that prevents decompression failures from occurring. In
particular, the SigComp feedback mechanism SHOULD be used to
tell the remote endpoint that the resources are no longer
available, and the remote endpoint MUST act on this information
(e.g. by falling back to stateless compression).

> why the idea was that they should only be increased and never 

Same applies for increasing the resources on-the-fly as well
(this never causes decompression failure, but it's awkward for
an endpoint to take advantage of extra resources mid-call, so
the majority of endpoints will simply ignore the additional
resources altogether).

Much better to decide on the amount of available resources when
the compartment is established, and to keep them constant for the
lifetime of the compartment.

> something for the Implementors Guide to begin with?

Agree completely. Let's add a section to the Implementer's Guide
to clarify the situation.

Regards,

Richard

---

From: "Price, Richard" <[email-masked]>
Date: Fri, 6 Jun 2003 13:53:56 +0100

> I agree with Jan that endpoint must never decrease the resources.

Hi Paulius,

A well-behaved implementation should definitely try to avoid
decreasing the resources on-the-fly. However, if it can't avoid
doing this for whatever reason, then it's much better to decrease
the available resources in a way that prevents decompression
failures from occurring.

> be allowed to happen in the mobile terminal.

The "safe" way to decrease the resources on-the-fly is to announce
the decrease using the SigComp feedback mechanism, and to defer
actually reclaiming the resources until the relevant message is
ACKed.

In some cases even this may be impossible (e.g. if the memory
simply stops working!). Of course, if this happens then we may
get decompression failures, so we should try to avoid this worst-
case scenario if at all possible.

Regards,

Richard

---

From: [email-masked]
Date: Fri, 6 Jun 2003 11:52:00 -0500

Richard,

I agree with your reasoning that an endpoint may be *forced* to
reduce its offered resources (who can prevent an endpoint from
malfunction/crash?). However, I disagree with your above statement. 

I think we have been discussing synchronization issue. From that
point of view, (monotonically) increasing offered resources clearly
makes it much simpler than allowing decreasing resources. A SigComp 
sending point can just rely on the latest announcement from its peer, 
without worrying about loss and misordering of the announcements. 
This is not true for decreasing.

Now, you seem to say increasing resources on-the-fly is not very
useful. This is another issue. And I'm not sure that is the case.
For example, a larger SMS can allow more states being created 
which in turn may improve compression ratio. A larger DMS can
allow a larger ring buffer which may also lead to better performance.

In summary, I think the clarification should focus on synchronization
issue, i.e., how it works and what are the potential problems. This
is the responsibility of a protocol (or rather its designers). As
to how an implementation uses the synchronization mechanism, we should 
be open-minded and let implementers make their own minds. 

(In fact, if dynamically increasing resource is not useful at all, why 
it has not been excluded from SigComp in the first place?)

BR,
Zhigang