Subject: RE: [rohc] ROHC FN review

From: "Finking, Robert" <[email-masked]>
Date: Wed, 8 Nov 2006 08:25:00 -0000

Hi Carsten,

Many thanks for the good work! We wanted to wait until you'd finished
your comments before making a final decision, but the way we're feeling
at the moment we don't want to make any major changes to the body of the
draft. None of us can spend much more time on this now, so we definitely
don't want to have to go to another WGLC. I think the general consensus
was that it might be good to include the ABNF in a separate appendix -
we'll need to confer about this though!

Thanks again for all the work you've put in to the ABNF and also in
reviewing the document.

Regards

Raffles

> -----Original Message-----
> From: Carsten Bormann [mailto:[email-masked]] 
> Sent: Wednesday, November 08, 2006 2:46 AM
> To: Carsten Bormann
> Cc: [email-masked]
> Subject: Re: [rohc] ROHC FN review
> 
> 
> ROHCv2 was too easy -- validates, too.
> 
> As the grammar is distributed throughout the document, I have  
> attached a consolidated version.
> I would like to see this (or some form of this) in the FN 
> spec, after  
> we have clarified the issues brought up by Mark and me (and other  
> reviewers, I hope).
> 
> Gruesse, Carsten
> 
> ALPHA = %x41-5A / %x61-7A
> C = %d67
> COMMENT = "//" *(SP / HTAB / VCHAR) CRLF
> COMPRESSED = %d67.79.77.80.82.69.83.83.69.68
> CONTROL = %d67.79.78.84.82.79.76
> CRLF = %x0A / %x0D.0A
> DEFAULT = %d68.69.70.65.85.76.84
> DIGIT = %x30-39
> ENFORCE = %d69.78.70.79.82.67.69
> HTAB = %x09
> INITIAL = %d73.78.73.84.73.65.76
> LENGTH = %d76.69.78.71.84.72
> NL = COMMENT / CRLF
> S = *WS
> SP = %x20
> STRCHAR = SP / HTAB / %x21 / %x23-7E
> STRQ = %x22
> THIS = %d84.72.73.83
> U = %d85
> UNCOMPRESSED = %d85.78.67.79.77.80.82.69.83.83.69.68
> UPCASE = %x41-5A
> VALUE = %d86.65.76.85.69
> VARIABLE = %d86.65.82.73.65.66.76.69
> VCHAR = %x21-7E
> WS = SP / HTAB / NL
> attname = ( U / C ) ( LENGTH / VALUE )
> attref = fieldnameref "." attname
> constdef = constname S "=" S expn S ";"
> constname = UPCASE *(UPCASE / DIGIT / "_")
> encspec = "'" *("0"/"1") "'" / id [ S "(" S expn S *( "," S expn S )  
> ")"]
> enforcer = ENFORCE S "(" S expn S ")" S ";"
> expn = *(expnc S "?" S expn S ":" S ) expnc
> expn0 = "(" S expn S ")" / VARIABLE
> expn1 = expn0 / attref / constname / litval / id
> expn2 = [("+"/"-"/"!"/"#") S] expn1
> expn3 = expn2 [S "^" S expn3]
> expn4 = *(expn3 S ("*"/"/"/"%") S) expn3
> expn5 = *(expn4 S ("+"/"-") S) expn4
> expn6 = expn5
> expn7 = *(expn6 S ("<"/"<="/">"/">=") S) expn6
> expn8 = *(expn7 S ("=="/"!=") S) expn7
> expn9 = expn8
> expna = expn9
> expnb = *(expna S "&&" S) expna
> expnc = *(expnb S "||" S) expnb
> false = %d102.97.108.115.101
> fielddef = fieldgroup S ["=:=" S encspec S] [lenspec S] ";" 
> fieldgroup = fieldname *( S ":" S fieldname ) fieldname = id 
> fieldnameref = fieldname / THIS fnspec = S *(constdef S) 
> [globctl S] 1*(methdef S) formatdef = formhead S formbody 
> formbody = "{" S *((fielddef/enforcer) S) "}" formhead = 
> UNCOMPRESSED [ 1*WS id ] / COMPRESSED [ 1*WS id ] /  
> CONTROL / INITIAL / DEFAULT
> globctl = CONTROL S formbody
> id = ALPHA *(ALPHA / DIGIT / "_")
> lenspec = "[" S expn S *("," S expn S) "]"
> litval = ["-"] "0b" 1*("0"/"1") / ["-"] "0x" 1* 
> (DIGIT/"a"/"b"/"c"/"d"/"e"/"f") / ["-"] 1*DIGIT / false / 
> true methdef = id S [parmlist S] "{" S 1*(formatdef S) "}" / id S  
> [parmlist S] STRQ *STRCHAR STRQ S ";"
> parmlist = "(" S id S *( "," S id S ) ")"
> true = %d116.114.117.101
> 
> 
> _______________________________________________
> Rohc mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rohc
> 

_______________________________________________
Rohc mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rohc

---

From: "Lars-Erik Jonsson \(LU/EAB\)" <[email-masked]>
Date: Wed, 8 Nov 2006 14:04:53 +0100

When checking all draft dependencies, I noticed one reference in
the FN which we have not changed from Informative to Normative
after we moved the CRC definitions to the framework.

9.2.  Informative References

   [I-D.ietf-rohc-rfc3095bis-framework]
              Jonsson, L., "The RObust Header Compression (ROHC)
              Framework", draft-ietf-rohc-rfc3095bis-framework-01
              (work in progress), July 2006.

/L-E

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 8 Nov 2006 06:36:34 -0800

On Nov 08 2006, at 00:25, Finking, Robert wrote:

> I think the general consensus

An appendix would be a good place.
I just did it inline for the convenience of reviewing the ABNF.

Before the ABNF can be considered "finished" though, we need to  
decide what to make of the following comments/possible resolutions:

-- (re Mark on section 4.2:) Would it be better to say FN identifiers  
are "case insensitive"?
    We could add "it is an error if multiple case spellings of the  
same case insensitive identifier occur in an FN specification.  For  
the purposes of this rule, all reserved words and identifiers (e.g.,  
encoding method names) defined in this specification are assumed to  
have occurred in the case spelling given here.".

-- We currently don't have operators for binary (bitwise) and, or,  
xor, negate (&, |, ^, ~ in C).  As HC is often about bit fiddling, I  
find this somewhat surprising (but then we didn't need it for the  
ROHCv2 profiles).  We would need to find a new name for ^, as we took  
this for exponentiation.  Also, << and >> are missing from C (but  
less essential). (My ABNF has placeholders for these; see the ABNF  
comments.)

As the comments and ABNF are pretty stable now, I'll send the  
entirety of my marked up version (1.7) of draft -11 to the list in a  
separate message for posterity.
(I hope it passes the mailman...)

Gruesse, Carsten

---

From: "Ghyslain Pelletier \(LU/EAB\)" <[email-masked]>
Date: Wed, 8 Nov 2006 15:55:44 +0100

> -- (re Mark on section 4.2:) Would it be better to say FN
> identifiers are "case insensitive"?
>     We could add "it is an error if multiple case spellings
> of the same case insensitive identifier occur in an FN
> specification.  For the purposes of this rule, all reserved
> words and identifiers (e.g., encoding method names) defined
> in this specification are assumed to have occurred in the
> case spelling given here.".

Irrespective the question about having an ABNF definition of the FN in
an appendix or not, my understanding is that the intent of the current
text is the following:

- FN is case sensitive
- for identifiers, I think we only want to state that one should not
  have identifiers that differ from each other only in capitalization.

Robert can correct me if I am wrong here, but I think that it is the
usage of the word "illegal" that is unfortunate here, I don't think that
there is any compelling need to make an exception for identifiers or for
reserved words. Otherwise, if there is a compelling reason to have this
exception, than I think that Carsten's proposed text above seems right.

> -- We currently don't have operators for binary (bitwise)
> and, or, xor, negate (&, |, ^, ~ in C).  As HC is often about
> bit fiddling, I find this somewhat surprising (but then we
> didn't need it for the
> ROHCv2 profiles).  We would need to find a new name for ^, as
> we took this for exponentiation.  Also, << and >> are missing
> from C (but less essential). (My ABNF has placeholders for
> these; see the ABNF
> comments.)

Again, irrespective of the ABNF question ...

We did not include binary operators because there are not needed (and I
cannot see that they would) but more importantly my understanding is
that these are not reversible and thus are not suited for the notation.
If you think otherwise please explain how those fit in the FN.

> entirety of my marked up version (1.7) of draft -11 to the
> list in a separate message for posterity.
 
It would be appreciated by the authors if you could as well prepare a
complete formatted appendix IF this is your proposal regarding ABNF in
FN. This would also surely help in discussing the relevance of having
such appendix and facilitate any agreement as I am not currently
convinced that this is something that is necessary/compelling to have.

And thanks for the comments + review, I haven't answered those yet on
the list but you can already see on the CVS that I have addressed a good
number of those. I still need to sort out a few things with my
co-author.

Regards,

///Ghyslain

---

From: "Ghyslain Pelletier \(LU/EAB\)" <[email-masked]>
Date: Thu, 9 Nov 2006 16:33:48 +0100

Carsten,

Thanks for your review.

I have made some updates and checked them into the CVS:
http://www.dmn.tzi.org/cgi-bin/cvsweb.cgi/fn/draft-ietf-rohc-formal-nota
tion.xml

Some comments I have either not yet addressed or I do not agree with.
I will address only those in this mail. I used your comments v1.7.

For now I will abstract the ABNF question in this mail.
I have ignored comments in between [[ ]].

///Ghyslain

- Context definition

I've updated it as follow (haven't directly taken your suggestion)

      Context is information about the current (de)compression state of
      the flow.  Specifically, a context for a specific field can be
      either uninitialized, or it can include a set of one or more
      values for the field's attributes defined by the compression
      algorithm, where a value may come from the field's attributes
      corresponding to a previous packet.  See also a more generalized
      definition in section 2.2 of [I-D.ietf-rohc-rfc3095bis-framework].

- Control field definition

s/are not/do not represent/

I don't agree to this change. I think a control field does carry a
representation of a part of the uncompressed header. I think that the
only requirement that defines a control field is that it isn't part of
the original uncompressed header.

- Section 3.2.2

...or the special ones "without an uncompressed value".
(or are those "control fields", too?)

I've re-written as follows:

The fields that occur in the compressed formats are either:

   o  fields that occur in the uncompressed format; or
   o  control fields, that have an uncompressed value and that occur in
      the CONTROL section; or
   o  control fields, that do not have an uncompressed value and thus
      defined as part of the compressed format.

Ed.: "Having an uncompressed value" is not a concept that has been
defined.

I've defined it in section 3.2.1:

                                                   A field is said to
   have a value attribute, i.e. a compressed value or an uncompressed
   value, if the corresponding length attribute is greater than zero.

- Section 3.3

Ed.: Is the fact that these have been reversed demonstrating anything?
If yes, note.  If no, order as in UNCOMPRESSED.

I've moved the note:

	   Note that the order of the fields in the compressed format is
	   independent of the order of the fields in the uncompressed
format.

to after the figure with the COMPRESSED format.

- Section 4.1

I don't find a definition of the global control field list.
Assuming this looks like a local one:

What's wrong with the pointer to 4.12.1.3?

- Section 4.7

There also is no subscript, member selection via object name (but see
field
attributes above), member selection via pointer, dereference, address
of, type cast, sizeof, function call, ...

See Robert's answer in a previous mail to the list.
We're not planning to change anything there.

- Section 4.7.2

No unary operators?
No & or | or ~?
No shifts?

The only binary bitwise operator which is reversible is XOR.
The others aren't. Furthermore, none of them is needed.
See a previous mail I sent to the list.
We're not planning to change anything there.

- Section 4.10

Ed.: positive = no zero length field is allowed?

I've added that zero length fields are ok.

- Section 4.11.1-2-3

I've updated the definition to the the FN

- Section 4.11.5

Define ULENGTH (ULENGTH of context value, I presume?)

I've added:

   The value of
   the "ULENGTH" can be derived from the information stored in the
   context.

- Section 4.12.1.3

Ed.: The following example should use [N] type numbers of bits

This is only an exemple, and comments are ok too.
I see no added value in making a change here.

- Section 4.12.1.5

This is starting to become confused between encoding methods and value
bindings.
And: fields don't have values, attributes have.

I've fixed two occurances of field having values.
As there are no other clear proposal in this comment, I am not planning
to change more here.

- 4.12.3.1.  Naming Convention

See other mail exchanges.

Updated the text to:

4.12.3.1  Naming Convention

   When compressed formats are defined, they must be defined using the
   reserved word "COMPRESSED".  Similarly uncompressed formats must be
   defined using the reserved word "UNCOMPRESSED".  After each of these
   keywords, a name may be given for the format.  If no name is given to
   the format, the name of the format is empty.

   Format names, except for the case where the name is empty, follow the
   syntactic rules of identifiers as described in Section 4.2.

   Format names must be unique within the scope of the encoding method
   to which they belong, except for the empty name which may be used for
   one "COMPRESSED" and one "UNCOMPRESSED" format.

- Section 4.12.3.3

Ed.: Add: As there is no alternative encoding given for field_2,
compression using this encoding method will fail for any packet that
does not have the given bits in this field in the uncompressed packet. 

I am not planning to add this text. The example is related to the
structure for the multiple formats, not for discussing the impacts of
defining a format in one way or another, or what makes a format fail or
succeed. This is specific to the encoding method in question.

---

From: "Ghyslain Pelletier \(LU/EAB\)" <[email-masked]>
Date: Thu, 9 Nov 2006 17:10:49 +0100

Carsten, Mark, all,

First, I'd like to thank once again Mark and Carsten for their excellent reviews.

>From my point of view, all the comments of relevance to the correctness of the FN specification have been addressed as agreed; the draft is updated and checked into the CVS:
http://www.dmn.tzi.org/cgi-bin/cvsweb.cgi/fn/draft-ietf-rohc-formal-notation.xml

Please review carefully as I would like to resubmit the draft shortly.

With respect to the comment on ABNF - I am not entirely sure about the relevance of the motivation and/or purpose of including ABNF into an Appendix in the FN draft, as it seems to be the proposal from Carsten. What I mean with this is that I don't see that it has any significant impact of the correctness and the quality of the FN specification, actually. The text already defines the syntax properly and nothing is missing, and this seems to be confirmed by the general consensus and by the reviewers as well. As far as I understand, some of the comments received were derived from the exercise of translating the FN into ABNF, but now these comments have been addressed. Point being, I think this is too small an addition to the draft to put more efforts on this, plus the necessary review of that additional information.

So, as I'd _really_ like to expedite this draft (as well as RoHC-TCP), I suggest that we leave it at this, or alternatively I suggest that Carsten writes a complete formatted appendix, gets it verified asap, and gets asap consensus to include it. The word here being asap.

And to clarify my personal opinion here -- I have actually really appreciated Carsten serious and comprehensive review, I think it's been very helpful, I've also looked at the ABNF, and I would like to thank him once more for this. It's just that I think that the timeframe for changes/addition is way overdue, so that we have to draw the line and move on with this one, and since no one else seems to want to do this ... and if this is really deemed useful by some, what about a separate informational draft so that this appendix does not simply become a new bottleneck for FN and RoHC-TCP?

So, dear reviewers and working group members, please review the changes checked into the CVS and give us a shout if we've missed something.

Best regards,

///Ghyslain

> It would be appreciated by the authors if you could as well
> prepare a complete formatted appendix IF this is your
> proposal regarding ABNF in FN. This would also surely help in
> discussing the relevance of having such appendix and
> facilitate any agreement as I am not currently convinced that
> this is something that is necessary/compelling to have.
> 

--
 Ghyslain Pelletier, Dipl. Ing.
 Wireless IP Optimizations
 Ericsson Research, Corporate Unit

 Ericsson AB, Laboratoriegränd 11
 Box 920, S-97128 Luleå, SWEDEN
 Phone : +46 (0)  8 404 29 43
 Mobile: +46 (0) 70 264 83 14
 Ghyslain.Pelletier at ericsson.com
 http://www.ericsson.com

-----------------------------------
 The opinions expressed in this
 message are my personal opinions.
 These opinions are not
 necessarily those of my employer,
 unless stated otherwise.
-----------------------------------

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 9 Nov 2006 08:51:15 -0800

On Nov 09 2006, at 08:10, Ghyslain Pelletier ((LU/EAB)) wrote:

> alternatively I suggest that Carsten writes a complete formatted  
> appendix, gets it verified asap, and gets asap consensus to include  
> it. The word here being asap.

Will do.  I'm at the IETF this week and still have to prepare for the  
ROHC segment in today's IPDVB meeting, but packaging the ABNF  
shouldn't be too hard.
(I'm done with verifying it against the extant FN samples -- but I  
have to look at your changes to the FN spec proper.)

I case you wonder why I did this work: I seriously dislike languages  
specified in English only, and I believe we need tool support for  
validating FN specs that is actually rooted in the spec.   
(Apparently, you do have a tool -- but it didn't catch the 8_bit bug,  
as it wasn't rooted in the text of the spec.)

I assume we did decide to fix the "8_bit" etc. identifiers in TCP-12,  
not in FN.

Gruesse, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 9 Nov 2006 10:15:17 -0800

Mini-review of the differences between fn-11 and CVS 1.83 below.

Excellent editing job, Ghyslain!

Working on aligning ABNF with spec now.

Am I right that we don't have unary + or - except on literals?
(It's not used in TCP or ROHCv2, so I'm taking this out of the ABNF.)

Gruesse, Carsten

exists which meet
->
exists that meets

3.2.1 should say that xLENGTH are measured in bits.

"decoding fails if the binding for an attribute fails."
->
...for this format...?

Just deleting the "on the wire" leaves some sentences somewhat  
ambiguous.  E.g., see the first sentence of 4.4.2.

4.7.2: Was there any decision to remove '#'?  (It is not needed right
now, so the YAGNI principle says this decision is the right one.)

(Strictly speaking, the definition of modulo would have to say that  
the value returned is non-negative and less than y, if y is  
positive.  I'm not even sure we care about the case that y is negative.)

Definition of VARIABLE: simplify to "non-negative integer".

4.11.5: Moved example tcp_sequence_number: "This takes up 14 bits"  
now needs to be qualified as CLENGTH (the previous paragraph talks  
about ULENGTH so this is confusing).

crc: s/i.e.:/Written as a field definition:/ (or at least go back to  
previous capitalization).

(4.12.3.1: see previous discussion)

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 9 Nov 2006 10:52:56 -0800

On Nov 09 2006, at 08:10, Ghyslain Pelletier ((LU/EAB)) wrote:

> I suggest that Carsten writes a complete formatted appendix, gets  
> it verified asap, and gets asap consensus to include it. The word  
> here being asap.

-- I have checked in a new appendix A (1.84 in the CVS).
    A formatted version of the draft draft is at http://www.tzi.org/ 
~cabo/fn-12-preview.txt

-- I have checked that this syntax parses TCP-12 (after fixing the  
non-identifier field names) and ROHCv2-00.
    I believe I have removed all remnants of C grammar that are not  
in FN.  I have also removed the '#' operator.
    It is still possible that the language of this ABNF is larger  
than the one specified by ROHC-FN.
    (It certainly is as there are no semantic checks.)  I do think  
the syntax is reasonably minimal.
    Additional eyeballs welcome, of course.

-- Now we need to decide for the RFC:

a) No formal syntax.

b) Formal syntax in appendix, informative.

c) Formal syntax in appendix, normative (this would also require  
changing the reference to RFC4234 to normative).

As I believe a formal syntax is most useful if it is binding, I  
prefer alternative c.

Gruesse, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 9 Nov 2006 15:12:47 -0800

> Mini-review of the differences between fn-11 and CVS 1.83 below.

...followed by a mini-review of the non- (missing) differences.

Ed.:
The intro of 4.7 can still be confused with a normative list of what  
the differences between C90 and FN are, which it isn't.

There are still a few instances of "perfectly allowable", but the RFC  
editor is likely to fix those.

Technical:
I'd still like to have non-empty identifiers on formats... (already  
mentioned in parent.)

I'm not sure I have an answer to my comment on 4.12.3.2.

In summary:
-- We have a fully baked spec of the FN language (just needs to be  
pulled out of the oven)
-- We have two non-trivial profiles that make use of the FN

Color me happy...

Gruesse, Carsten

---

From: Carsten Bormann <[email-masked]>
Date: Sun, 19 Nov 2006 22:48:47 +0100

> the syntax is reasonably minimal.

I made one mistake:
The ABNF I checked in (in 1.84) contains C language conditional  
expressions (of the form: cond ? trueexpr : falseexpr).
These are not explicitly listed in -11, only what I thought to be  
implicitly included by the introduction text in 4.7.
I believe we have now decided that this text in 4.7 is introductory  
and not normative.
Conditional expressions are not used by TCP-13 or ROHCv2, either.
So this was a case of "the language of this ABNF is larger than the  
one specified by ROHC-FN".
I'm now reasonably certain this was the last such case (of course,  
within the limits that you can express syntax by a notation like the  
ABNF).

I have checked in a 1.86 on the authors' CVS.
This also adds a small clarification in the introductory sentence  
("fnspec" is the start rule of the grammar).
The fixed ABNF continues to:
-- parse TCP-13 (with the 8_bit etc. change discussed here) and  
ROHCv2-00 (using a home-grown ABNF compiler, though);
-- be accepted by Bill Fenner's ABNF checker http://rtg.ietf.org/ 
~fenner/abnf.cgi;
-- be accepted by Jutta Degener's abnfgen.

Gruesse, Carsten

PS.: Yes, my little misunderstanding is one of the reasons why I like  
to have an ABNF.

PPS.: Jutta Degener's abnfgen is a nice tool to check ABNF grammars.
Here is one example of "valid FN syntax" generated by that tool for  
your amusement.
Those who have tools that parse FN, I'd like to hear about your core- 
dumps :-)

CONTROL
	{ENFORCE (	//
!P ^J^L-

   N^T *N*!M^W   -!false ^!J^D !=!TG^!THIS.UVALUE ^R^O||	H^E/ !V^A-W/ 
P^Y+J- !Y^E%!false^M^E
 >!S^N  %  !(N) ^T^Z+
!(U)^ J^Q	%!F^!Q^M!=	//
O^N % P/E/!R^H	
   <	  W^I% !Q^C
   -K^V- !Z^E*M^C %!(G) ^ D^Q!=	//
   ! Z  ^ !J^Q+//
	! (V)^ !P^E

*! 0B0
^ !(N)^ X^H&&
!false ^U^G
%  !Z^Y *!THIS.UVALUE^!D^Z	
	  -!J^E  *  I^T/ Y/L/!false ^L^E-		O^C % N%!T^O
-! THIS.UVALUE^!H^P*
!(U)^H^M
%	!TI
^!(N) ^C^E//		
	//
	
== //
!SX   ^	  !N ^ Z^C<=	! false^  !Q^Y*!  7^  !(X) ^V^I+!  W7^!K ^ A^D *! 
THIS.ULENGTH ^
! false ^  !J^Q
)	;  v  ://
//
V=:='0101'[
C*J^Z<   T-C*J <=  T >= H^P -Q%B+ !U^I&&	  O/U ==Y+Y==U%S +M^J 		  &&  
P-G <=F>J-Z&&
!U^A 	
==

!F^Z%  H^Q *!Z^ Z^S]  ;   }w3_(
NVbs_3_ZE	,	Sq__	//
//
,	m ,v//
	,hX6//h
	)	" "	//
//,
//
; 	  Q8(	K	 ,	 	//
z){//	
UNCOMPRESSED	r8//J	
		{}UNCOMPRESSED		V_e_9z4C{
ENFORCE(P<=I<H-A!= B > R>=H-J 	
   || 	
X+N*S%B^C	);}} J(
k_,x){UNCOMPRESSED	 	k__{
}	}
Compilation finished at Sun Nov 19 22:08:50