Subject: [rohc] NBO- TCP/IP EPIC profile

From: Julije Ozegovic <[email-masked]>
Date: Fri, 01 Mar 2002 14:08:02 +0100

Hi Mark,

let's go on one question at the time, it became hard do read 
everything in one single mail :)

>>
>> ** NBO:
>>
>> can we introduce something like SWAP-LSB in the toolbox and let normal 
>> method choice take place, e.g.:
>>
>>    IP_ID = LSB(80%,x,y) | SWAP-LSB(20%,x,y)
> 
> 
> 
> Are you saying that you would like to have a 'SWAP-' version of *every* 
> method in the toolbox?  Isn't this adding just a little more complexity 
> than a simple, in-place transformation?  The whole point is that the 
> byte-swappedness of a field is orthogonal to the encoding method. That's 
> why it got split from the INFERRED-SCALED method in the first place.
> 
> In fact, your example perfectly illustrates that orthogonality.  It is 
> clear that the IP ID is going to use LSB encoding.  However, you are 
> saying that (a priori) you know that 20% of the IP ID values will be 
> byte swapped and 80% will be in NBO.  My crystal-ball gazing just isn't 
> that good at predicting what's going to appear as input ;-)
> 
> Contrast this with our current suggestion:
>     foo = byte-swap
>           encode-foo
>               encode-swap-flag
> 
>     byte-swap = NBO(16)
>     encode-foo = LSB(...)
>     encode-swap-flag = STATIC(99%) | IRREGULAR(1,1%)
> 
> What I am saying here is that the field is put into NBO.  Then I encode 
> the (NBO'd) field with LSB (or whatever).  Then I encode the swap-flag. 
>  Now, what I saying here is that the probability that the byte-swap 
> state *changes* is 1%.  This is very different from saying that the 
> probability of byte-swapping *is* 1%, for example.
> 
> I can be confident that the byte-swap state won't change frequently.  I 
> have no idea what the ratio of byte-swapped to non-byte-swapped values 
> will be...

ok, BUT!

please, tell me about any other METHOD that is byte-order 
sensitive, except LSB. Remember "msb" function mentioned in 
EPIC-lite draft, which does almost the same thing.

Besides, it seem that all fields of the TCP/IP header are sent 
to the network with EXACTLY SPECIFIED byte order, NBO anyway. 
The only exception is IP_ID, which "some" hosts send in reversed 
order. Than, if NBO for all fields except IP_ID MUST be 
followed, it means that internal byte order is implementation 
specific, and is NOT needed to be concerned with profile.

Moreover, it seem to me that it is unnecessarily complicated to 
use NBO() inside the FORMAT, SWAP-LSB is much more 
"format-friendly". That means that structure like:

     IP_ID = FORMAT(LSB(100%,x,y));(SWAP-LSB(100%,x,y))

can be more effective than NBO approach. This is the essence of 
FORMAT usage: to split processing of differently behaving TCP flows.

Conclusion is: NBO is effective for LSB coding only, and can 
optimally be processed with SWAP-LSB (ok, in my opinion :)

best regards,

Julije, Maja and Tijana

_______________________________________________
Rohc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rohc

---

From: Wang Hui <[email-masked]>
Date: Sat, 2 Mar 2002 09:54:20 +0800 (CST)

Hello folks here,

I am new to the 'header compression' field.  I read some papers and
drafts, and have some questions.

[1]
ROHC scheme discussed in the WG is said to be combined with the layer 2
techniques.  So there will be a lot of rohc over xxx document in future,
since layer 2 technologies are various.  I saw from the webpage of this
WG, and only found a 'rohc over ppp' draft.  Is there any work carried on
to deal with other layer 2 techonologies?

[2]
Are there some testing implementations?  Or testing results of the scheme
proposed in this WG?  Where to find them?

[3]
How to deal with the handover in the mobile ip?  While the mobile node
moving to another domain, how could the compression context in its old
access router be moved to the new access router fast?

....

I am new to this research field, so these questions might be too simple.
And I also expect a expert to give me a road to step into this field.

Thanks in advance.

Regards,
Wang Hui

--
 Wang Hui
 InfoNet Lab, USTC
 http://mail.ustc.edu.cn/~whui
 Tel: +86-551-3603634

---

From: "West, Mark (ITN)" <[email-masked]>
Date: Mon, 04 Mar 2002 09:38:41 +0000

Hi Julije,

 > let's go on one question at the time, it became hard do read everything
 > in one single mail :)

Probably a good idea!

 >
 >
 >>>
 >>> ** NBO:
 >>>
 >>> can we introduce something like SWAP-LSB in the toolbox and let
 >>> normal method choice take place, e.g.:
 >>>

[my last rant removed...]

 >
 >
 > ok, BUT!
 >
 > please, tell me about any other METHOD that is byte-order sensitive,
 > except LSB. Remember "msb" function mentioned in EPIC-lite draft, which
 > does almost the same thing.

Well, anything that does not just treat the field as an opaque bunch of 
bits (like IRREGULAR and STATIC) may be sensitive.  I'll give you  some 
specific examples towards the end of my response...

 >
 > Besides, it seem that all fields of the TCP/IP header are sent to the
 > network with EXACTLY SPECIFIED byte order, NBO anyway. The only
 > exception is IP_ID, which "some" hosts send in reversed order. Than, if
 > NBO for all fields except IP_ID MUST be followed, it means that internal
 > byte order is implementation specific, and is NOT needed to be concerned
 > with profile.

Right!  Any field for which the representation matters has a specified 
representation 'on the wire', which is NBO.  The IP ID is a slightly 
bizarre case, in that it is just a descriminator between packets.  It 
just happens that the easiest/most common way of implementing this is 
with a counter and we (try to) exploit that in header compression.

Having said that, just because it's rare doesn't mean that it won't 
occur in other protocols (or protocol implementations).

So, we accept (I think) that there may be some (very few) fields that 
are treated as 'numeric' and which may not have a well-defined 
byte-order.  Given  this, there has to be some way of handling it. 
Since EPIC is fundamentally trying to be protocol independent, it seems 
unwise to make this an IP ID *specific* feature.

Also, I don't follow why you claim that this is not a profile concern - 
  I tried to interpret this in the sense of, "we shouldn't concern 
ourselves with implementation dependencies like byte-swapping."
But 3095 does it and you're suggesting SWAP-LSB, rather than ignoring 
it.  So, I don't get it.  Sorry...

 >
 > Moreover, it seem to me that it is unnecessarily complicated to use
 > NBO() inside the FORMAT, SWAP-LSB is much more "format-friendly". That
 > means that structure like:
 >
 > can be more effective than NBO approach. This is the essence of FORMAT
 > usage: to split processing of differently behaving TCP flows.
 >

The essence of FORMAT is to allow different sets of encoding methods to
be applied, specifically where the format of the compressed packets 
changes.
However, looking at IP ID, if it behaves sequentially, then regardless
of the byte-order the same set of 'formats' will be used.  (That is, the
same number of bits will be used for LSB with the same interpretation 
interval).  So it's not really changing the format, is it?

Also, harking back to another point that you made previously, 
introducing 'FORMAT' here doubles the number of format sets (and 
consequently the total number of formats).

FORMAT also requires the use of IR-DYN packets, whereas NBO need not.
(Granted, it's very likely due to the low probability of a change, but
that's up to the packet generation)

If the use of FORMAT was able to avoid introducing a new method, then I
might concede the point.  However, since we have to add something 
(either NBO or SWAP-LSB or whatever), I don't see that making use of
FORMAT is necessary.

NBO is 'lighter weight' and more specific than FORMAT.  In this case, I
happen to think that this is good.

 > Conclusion is: NBO is effective for LSB coding only, and can optimally
 > be processed with SWAP-LSB (ok, in my opinion :)
 >

This is actually not true (in my opinion ;-)
NBO is effective for any encoding that makes use of arithmetic 
operations.  If you were to look at an RTP profile, for example, you 
will see that the IP ID is encoded relative to the master sequence 
number.  This is done through the INFERRED-OFFSET command (which is 
effectively a subtraction).  Clearly the base and the value to be 
encoded need to be in the same byte-order.  Thus, LSB is not the only 
method that benefits from NBO and, so, I stand by our claim that NBO is 
the most suitable solution...

(In passing, IRREGULAR-PADDED, VALUE and SCALE could also be argued to 
be byte-order dependent)

 > Julije, Maja and Tijana

Cheers,

Mark.

-- 
Mark A. West, Consultant Engineer
Roke Manor Research Ltd., Romsey, Hants.  SO51 0ZN
Phone +44 (0)1794 833311   Fax  +44 (0)1794 833433

(Yes, I do know that my disclaimer is in an attachment.  And, no, I
didn't ask for it to be that way)

---

From: Julije Ozegovic <[email-masked]>
Date: Mon, 04 Mar 2002 13:45:02 +0100

Hi Mark,

nice to hear from you ;)

As you can realize, the difference in our approaches is because 
you are doing top-down, while myself I am concerned with 
bottom-up design. This is why we give the same problem different 
flavors.

I can agree with all your writing, but also I have a strong 
feeling that the point is missed. What is of concern follows:

1. ALL fields are sent to the network in NBO
2. Only IP_ID can have ARBITRARY value
3. Only "some" hosts (not to mention which ones)
    reverse byte order for IP_ID

==> therefore, there is no possibility
     for NBO confusion except for IP_ID

4. IP_ID is sometimes generated randomly,
    and sometimes just incremented
5. Subsequent IP_ID numbers are assigned to packets
    which can belong to different data flows

==> We can expect IP_ID to be periodic to some extent,
     depending on the number of data flows
     and their respective rates at the sender

6. LSB coding is therefore optimal for IP_ID,
    RTP profile with INFERRED-OFFSET is possibly
    a misuse, however it is official.
    What I have proposed can be modified to

    IP_ID = LSB(70%,x,y) | SWAP-LSB(20%,x,y) | IRREGULAR(10%)

7. NBO() is ambiguous, the question is:
    how application can recognize swapped field?
8. of course, it is possible
    - to compare new value with the old one from the context
    - decide whether LSB coding is appropriate
    - if not, swap bytes
    - test LSB on swapped value
    - if not, it should be random
    - set NBO flag
    - communicate NBO flag through IR/DYN
    - send subsequent packets with swapped LSB
9. What if IP_ID is random?
    there is a possibility that LSB or NBO-LSB
    succeed on random IP_ID alternatively;
    in worst case random IP_ID will cause
    sequence of IR/DYN packets!

==> SWAP-LSB approach is optimal
     - it uses basic ID_bits mechanism,
     - it does not need separate NBO indicators

NBO-flag:

from proposed profile, as far as I can see, NBO-flag is passed 
as IRREGULAR through IR/DYN, and just tested STATIC in CO; 
therefore it has the same mechanism and importance of the format 
selector itself.

Now about format:

I did not propose SWAP-LSB to be used inside FORMAT, just said 
that it CAN be used, and that it is more format-friendly.

Format is mentioned because its main purpose is to generate 
separate format sets for differently behaving TCP/IP flows, and 
IP_ID behavior is an example of that. Why separate formats to 
sets? Just to make sure that some important formats of one type 
of flows will not be discarded because of low overall 
probability. That's why we can expect NBO/SWAP-LSB inside FORMAT.

What is of concern here, NBO inside FORMAT is double indication 
(format selector + NBO selector). It seem to be too complex for 
me, somehow makes me nervous (from the application point of view).

Finally, about IRREGULAR-PADDED, VALUE and SCALE:

I agreee that they are NBO sensitive, but whether that results 
with any practical consequence?

Best regards,
Julije

West, Mark (ITN) wrote:

> byte-order.  Given  this, there has to be some way of handling it. Since 
> EPIC is fundamentally trying to be protocol independent, it seems unwise 
> to make this an IP ID *specific* feature.
> 
> ------------------------------------------------------------------------
> 
> The information contained in this e-mail and any attachments is confidential to Roke 
> Manor Research Ltd and must not be passed to any third party without permission. This 
> communication is for information only and shall not create or change any contractual 
> relationship.
> 
> RMRL-Disclaimer.txt
> 
> Content-Type:
> 
> text/plain
> Content-Encoding:
> 
> 7bit
> 
>

---

From: "West, Mark (ITN)" <[email-masked]>
Date: Mon, 04 Mar 2002 15:12:55 +0000

Hi Julije,

I don't know what it's going to take to convince you of my point of 
view, but I should warn you that I don't give up easily...

;-)

Mark.

Julije Ozegovic wrote:

> Hi Mark,
> 
> nice to hear from you ;)
> 
> As you can realize, the difference in our approaches is because you are 
> doing top-down, while myself I am concerned with bottom-up design. This 
> is why we give the same problem different flavors.

I'm not sure that I do realise why we have this difference.  Anyway, 
without worrying too much about the why, let's try and resolve it.

> 
> I can agree with all your writing, but also I have a strong feeling that 
> the point is missed. What is of concern follows:
> 

I'm inclined to agree that a point is being missed.  Which point and by 
whom is less clear!  Now - if you can just agree with my conclusions...

;-)

> 2. Only IP_ID can have ARBITRARY value

<pedantry>
Many fields can have arbitrary values.  The IP ID is interesting 
because, although it can have an arbitrary value, it is commonly 
implemented in a more structured way.
</pendantry>

>     for NBO confusion except for IP_ID

If you are talking about a universe consisting only of TCP/IP header 
compression, maybe.

For RTP the encoding is different.  Who knows for other protocols?

(As a hypothetical example, if the Timestamp mechanism from RFC 1323 
were only used for RTTM and not for PAWS, then the same could apply to 
the TS values.  Clearly this is not the case, but there may be similar 
issues with other protocols.  We just can't say...)

>    and sometimes just incremented

Yes.

>     and their respective rates at the sender

Both these points are covered in RFC 3095 and the TCP behavior model 
draft <draft-west-tcpip-field-behavior-00.txt> [-01 has been submitted], 
where 3 classes of IP ID behaviour are described.

However, what this boils down to is that we either see the IP ID as 
random or more-or-less sequential.

>    a misuse, however it is official.

A *misuse*!?  Have I missed something here?  Did I just hallucinate the 
entire history of RFC 3095?  (I hope not as this would make me question 
my already limited grasp on sanity)

Treating the IP ID as an offset from the RTP SN is the only way of 
achieving the maximum compression ratio for ROHC RTP.  In the case where 
the IP ID moves in step with the RTP SN, it is clearly the most 
efficient way of dealing with the IP ID as it can be elided altogether. 
  When the IP ID jumps, it is no less efficient to handle it this way.
(Ultimately, of course, LSB encoding is used here, but on the offset 
rather than the absolute value).

>    IP_ID = LSB(70%,x,y) | SWAP-LSB(20%,x,y) | IRREGULAR(10%)

Hang on - haven't we been here before?

Let me try and state, as clearly and unambiguously as possible what the 
above line actually says:

"70% of the time, standard LSB encoding will be used.  20% of the time, 
the same LSB encoding will be used, but with byte-swapped input.  10% of 
the time, the full 16-bit IP ID value will be sent."

Or:

"70% of my packets will have IP ID in NBO and 20% will have byte-swapped 
IP IDs."

I really need to talk to your supplier of tea-leaves, crystal-balls or 
however else you can determine the percentages of byte-swapped values 
that your compressor/decompressor will have to processs ;-)

More importantly, you are introducing a 'hidden' NBO flag into the 
Huffman prefixes.  You are creating two encodings which are equivalent 
except for the byte order and which are separate nodes in the tree. 
Whether the value is byte-swapped or not will be encoded in *every* 
packet.  How can this possibly be efficient?

>    how application can recognize swapped field?

Hold on a second!  NBO is a *component* of SWAP-LSB.  It has to be!  So 
NBO is *exactly* as ambiguous (or unambiguous) as SWAP-LSB.

>    - send subsequent packets with swapped LSB

Well, basically, yes.

I could pick nits, but it's really not worth it.

>    sequence of IR/DYN packets!

Random IP ID behaviour would have to be managed regardless of the 
byte-swap issue.  This is the case where FORMAT is necessary.  It is 
clearly useful here since the entire basis of the encoding changes:  the 
full 2-octets of data are sent in every packet instead of using a choice 
of more efficient encodings.

And, yes, a dumb implementation of FORMAT will cause the low-probability 
IRREGULAR encoding from the sequential behaviour version to be used 
repeatedly.  Probably causing a lot of IR-DYN packets to be sent.
Dumb implementations of FORMAT cannot hurt interoperability, but they 
can affect efficiency.  Which is another reason to only use them where 
it's necessary.

>     - it does not need separate NBO indicators

It's not optimal.  (Even for very lax definitions of optimal...)
It *does* need a separate NBO indicator but it hides it in the indicator 
flags prefix, rather than admitting to it.

<challenge>
It's trivial to implement SWAP-LSB, anyway, so why don't we compare the 
two approaches and see who gets the lower average compressed packet sizes?
</challenge>

> NBO-flag:
> 
> from proposed profile, as far as I can see, NBO-flag is passed as 
> IRREGULAR through IR/DYN, and just tested STATIC in CO; therefore it has 
> the same mechanism and importance of the format selector itself.
> 

There are 2 crucial differences between NBO and FORMAT.
- You *can* change NBO in a compressed packet.  You *can't* change FORMAT.
- FORMAT (at least) doubles the number of formats (or makes your 
format-set vs. max-formats trade-off harder).  NBO doesn't...

> Now about format:
> 
> I did not propose SWAP-LSB to be used inside FORMAT, just said that it 
> CAN be used, and that it is more format-friendly.
> 

See above.  But if you *were* to use SWAP-LSB, then you would *have* to 
use it in FORMAT.  I still regard this as using a sledgehammer to crack 
a nut...

> Format is mentioned because its main purpose is to generate separate 
> format sets for differently behaving TCP/IP flows, and IP_ID behavior is 
> an example of that. Why separate formats to sets? Just to make sure that 
> some important formats of one type of flows will not be discarded 
> because of low overall probability. That's why we can expect 
> NBO/SWAP-LSB inside FORMAT.
> 

No.  FORMAT is, as I think I've mentioned before, a way of choosing 
between different encoding sets.  Random vs sequential IP ID is an 
excellent example of this.

However, byte-swapping can be considered orthogonally to the encoding 
issue.  That is, with NBO we test the byte order and change it if 
necessary.  Then we apply the *same* LSB encodings.

> What is of concern here, NBO inside FORMAT is double indication (format 
> selector + NBO selector). It seem to be too complex for me, somehow 
> makes me nervous (from the application point of view).
>

I don't see the problem here, frankly.
The FORMAT selects between random and non-random behaviour.  NBO lets it 
account for byte-swapping in the non-random case.  Again, it is obvious 
that SWAP-LSB has *exactly* the same issues.

> I agreee that they are NBO sensitive, but whether that results with any 
> practical consequence?
> 

I'm obviously missing something here.  If you introduce NBO, then you 
can use any other method with it, such as those above.  Ok, it may be 
relatively unlikely that this will happen.  But, if we go down the 
SWAP-LSB route, then to achieve the same level of flexibility, you 
should have SWAP-IRREGULAR-PADDED, SWAP-VALUE and SWAP-SCALE, and...

And, don't forget that terribly misused encoding INFERRED-OFFSET.  If 
you accept that we need SWAP-INFERRED-OFFSET, then you're trying to 
replace my one flexible encoding component with two inflexible encodings.

> Julije

I'm going to try something dangerous and summarise:

- IP ID is one concrete example of a field that does not have a defined 
representation and so, because of implementation issues, may have a 
byte-swapping problem
- It is necessary to account for this in an EPIC profile
- The encodings used for the normal and byte-swapped representations are 
(ignoring the byte-swapping) the same
- We cannot know in advance what proportion of packets will contain 
byte-swapped values
- The byte-swapping state will not change frequently
   (and should only be sent when it changes)
- There are at least two candidate encodings (LSB and INFERRED-OFFSET)
- Any solution must look at the historical values of the field and 
decide whether or not it is considered byte-swapped

Regarding the solutions (my opinion, of course ;-)

- SWAP-LSB lacks flexibility and forces the use of FORMAT, which is 
undesirable.
- NBO is a flexible approach (equivalent to the NBO flag in RFC 3095, 
but in a more generic way)

If you want to change my mind, you will need to convince me of:

- How SWAP-LSB can be used *without* FORMAT and without reducing the 
encoding efficiency
- How SWAP-LSB's decision on byte-swapping is different from (more 
importantly, better than) the NBO approach
- How this approach can be extended to cover the other 4 methods that 
are affected by byte ordering.

ok?

---

From: Julije Ozegovic <[email-masked]>
Date: Mon, 04 Mar 2002 21:32:54 +0100

Hi!

> representation and so, because of implementation issues, may have a 

ok

> - It is necessary to account for this in an EPIC profile

ok

> (ignoring the byte-swapping) the same

ok

> - We cannot know in advance what proportion of packets will contain 

well, at least with same precision as all other probabilities in 
profile!

>   (and should only be sent when it changes)

it should not change for the flow, this is the strong point of 
the NBO approach!

> - There are at least two candidate encodings (LSB and INFERRED-OFFSET)

ok

> decide whether or not it is considered byte-swapped

ok

> Regarding the solutions (my opinion, of course ;-)

of course!

> - SWAP-LSB lacks flexibility and forces the use of FORMAT, which is 

it does not force the use of the format!

NBO is "more flexible" when you need it, but that is also one 
more state for the context!

> but in a more generic way)

let it be part of the standard then, with syntax defined

> - How SWAP-LSB can be used *without* FORMAT and without reducing the 

just using common OR functionality

> importantly, better than) the NBO approach

again, just using common OR functionality:
try one, than another, use first that succeeds

(it is, as you said, hidden NBO in ID_flags, but that is the 
function of ID_flags: to identify format)

> are affected by byte ordering.

does it need to?

> ok?

OK ;-)

julije

PS. anyway, we are going to do it NBO way, since it is in the 
standard (really, is it?)

---

From: "West, Mark (ITN)" <[email-masked]>
Date: Tue, 05 Mar 2002 16:31:12 +0000

Hi Julije,

[much general agreement removed]

> well, at least with same precision as all other probabilities in profile!
> 

Hmmm.  Not quite!  If I took a single 'representative' flow and used it 
to decide on the probabilities to assign to the various choices, I would 
have to choose probabiliies of '0%' for all the swapped/non-swapped 
versions (since this doesn't change within a flow).

Anyway, I don't think that we'll gain much more by discussing this issue 
further.  And, if we really wanted to, we could settle it by comparing 
implementations.  But I'd rather not...

> it should not change for the flow, this is the strong point of the NBO 
> approach!
> 

Agreed!

>> undesirable

> it does not force the use of the format!

It doesn't force it, but it needs it for effiency.

> NBO is "more flexible" when you need it, but that is also one more state 
> for the context!
> 

Agreed.  I don' think that the state is a high price to pay (ROHC has 
it, for instance), but you are correct.

> let it be part of the standard then, with syntax defined

Well, we haven't got a standard, yet ;-)

But, it's part of the current draft:

http://www.dmn.tzi.org/ietf/rohc/draft-ietf-rohc-epic-lite-01.txt

Enjoy..!

>> If you want to change my mind, you will need to convince me of:

[snip]

> 
> (it is, as you said, hidden NBO in ID_flags, but that is the function of 
> ID_flags: to identify format)
> 

Yes, this is true.  But if, as you agree, the NBO flag stays the same 
for the whole flow, why encode it in every packet?

>> are affected by byte ordering.

I think so.  But, as I said earlier, I'm biased!

> OK ;-)

ok!

> julije
> 
> PS. anyway, we are going to do it NBO way, since it is in the standard 
> (really, is it?)

*sigh*

If you're happy to do it this way, then I'm not entirely sure what we're 
arguing about.  (Sorry - discussing!)

But I've enjoyed the debate..!

Cheers,

Mark.

---

From: Julije Ozegovic <[email-masked]>
Date: Wed, 06 Mar 2002 12:03:07 +0100

Hi Mark,

concerning new EPIC-lite (01) draft is published, and it 
generally covers the TCP/IP profile published earlier, much of 
our discussion is obsolete.

About NBO issue, if it stays in (future) standard, it should be 
implemented for compatibility. "SWAP-LSB" approach needs much 
more effort and support to be adopted, and it doesn't seem to be 
possible at the moment.

However, the main idea of making the whole thing more simple 
must not be neglected. This concerns both "language developers" 
as well as "profile developers". Language is (almost) defined, 
and it should be complete. Now it is the responsibility of 
profile standardization process to find optimum between 
compression and computation efficiency.

Best regards,
Julije

West, Mark (ITN) wrote:

>> NBO is "more flexible" when you need it, but that is also one more 
>> state for the context!
>>
> 
> 
>> (it is, as you said, hidden NBO in ID_flags, but that is the function 
>> of ID_flags: to identify format)
>>
> 
>