Subject: [Sip] SIP ABNF Grammar bugs?

From: Hall Bill-P26432 <[email-masked]>
Date: Tue, 12 Mar 2002 11:04:32 -0700

SIP Spec Rep,

I have appreciated all of the SIP ABNF grammar improvements over the last few releases.  Thanks!  

I have a SIP parser that operates by interpreting the grammar almost exactly as specified in the spec (bis-09).  However, if I follow the grammar exactly, I am unable to properly parse some of the supposedly correctly formatted "SIP Test Messages" files (http://www.cs.columbia.edu/~hgs/sip/sipit/testmsg.html ).  In particular, 

1) Commentary in the spec states that HCOLON allows whitespace, but no line break, before the colon.  However, in file test1.txt a Call-ID header is specified as "CaLl-Id<cr><lf><sp>:<sp>[email-masked]<cr><lf>" (a line break before the colon).  To get the file to parse (assuming it is correct), I changed the grammar definition of HCOLON from 

HCOLON  =  *( SP / HTAB ) ":" SWS

to

HCOLON  =  SWS ":" SWS

2) I believe the grammar is missing optional WSP (whitespace) between the body of a message-header and the trailing CRLF. In file test1.txt the "v:" header ends with ";<sp>hidden<sp><sp><sp><cr><lf>".  To get the file to parse I changed the grammar definition of message-header from 

message-header = (Accept 
               / Accept-Encoding 
               ...
               /  WWW-Authenticate
               /  extension-header) CRLF

to 

message-header = (Accept 
               / Accept-Encoding 
               ...
               /  WWW-Authenticate
               /  extension-header) *WPS CRLF   <-- note the *WPS

3) There seems to be missing whitespace before delay in the Timestamp header definition.  I changed the Timestamp definition from

Timestamp  =  "Timestamp" HCOLON 1*(DIGIT)
                      [ "." *(DIGIT) ] [ delay ]

to

Timestamp  =  "Timestamp" HCOLON 1*(DIGIT)
                      [ "." *(DIGIT) ] [ SWS delay ]  <-- note the SWS

I also suggest the following changes to the grammar to aid folks who are parsing SIP based solely on the grammar definition:

1) Commentary states that a URI must be represented as a name-addr instead of an addr-spec if it contains certain special chars (e.g., ";").  The grammar does not enforce this, but it could.  The definitions for name-addr and addr-spec allow the same characters in the URI.  The definition for addr-spec should not allow the special characters.  name-addr could be defined as:

name-addr      =  [ display-name ] LAQUOT special-addr-spec RAQUOT

where the definition of special-addr-spec does allow the special characters.  

2) Do not make choices optional as in 

foo  =  bar / baz / [boo]

(where [boo] is an optional choice).  When bar, baz, and boo are NOT present in a parse string, this style of rule allows foo to match null instead of failing.  Optional choices are used in rules such as dig-resp, ainfo, and digest-cln.  Similarly, in the following definitions srvr is a redundantly optional choice:

authority      =  srvr / reg-name
srvr           =  [ [ userinfo "@" ] hostport ]

and srvr will always match something (possibly null).  I changed the definition of srvr to make it no longer redundantly optional:

srvr           =  [ userinfo "@" ] hostport

3) Similarly, do not allow elements that are already optional using brackets (e.g., "[ display-name ]") match zero-or-more elements ("*" notation) since this allows the optional element to always parse by matching null.  For instance, 

display-name   =  *(token LWS)/ quoted-string   should be:
display-name   =  +(token LWS)/ quoted-string

4) Do not allow any but the last choice in an alternative match the "generic" or "extension" case since these general cases always match the "specific" cases.  For instance, m-type can be a discrete-type or a composite-type:

m-type           =  discrete-type / composite-type

Both a discrete-type and composite-type can be an extension-token:

discrete-type    =  "text" / "image" / "audio" / "video"
                            / "application" / extension-token
composite-type   =  "message" / "multipart" / extension-token

If the m-type is "multipart" in a parse string, then this will match as a discrete-type instead of a composite-type because it will match as a discrete-type extension-token.  

I would define m-type as

m-type           =  discrete-type / composite-type 
                           / extension-token

and remove extension-token from the definition of discrete-type and composite-type.  

5) When a choice, A, matches a subset of another choice, B, always list A first.  For example, a telephone-subscriber without a password is just a special form of a user.  When userinfo is defined as:

userinfo         =  [ user / telephone-subscriber [ ":" password ]]

something that looks like a telephone-subscriber will match as a user.  I change the rule to:

userinfo         =  [ telephone-subscriber [ ":" password ] / user ] 

Similarly, since host matches a subset of token, I changed:

gen-value      =  token / host / quoted-string    to:
gen-value      =  host / token / quoted-string

For similar reasons I changed:

codings          =  content-coding / "*"    to:
codings          =  "*" / content-coding

extension-token  =  ietf-token / x-token    to:
extension-token  =  x-token / ietf-token

host             =  hostname / IPv4address / IPv6reference   to:
host             =  IPv4address / hostname / IPv6reference

hexpart        =  hexseq / hexseq "::" [ hexseq ] / "::" [ hexseq ]   to:
hexpart        =  "::" [ hexseq ] / hexseq "::" [ hexseq ] / hexseq

Also, since iana-token is a proper subset of extension-token, then m-subtype, which is defined as:

m-subtype        =  extension-token / iana-token

will always match extension-token and never match iana-token.  iana-token seems useless.  

6) To better allow lookahead to differentiate alternatives I changed

IPv6address    =  hexpart [ ":" IPv4address ]         to:
IPv6address    =  hexpart ":" IPv4address / hexpart 
  
7) An accept-range can end in a semicolon-separated list of accept-params and m-parameters.  How are accept-params and m-parameters distinguished?

8) To maintain parallelism with the other dig-resp elements that give names to their parameter values, I suggest changing 

auth-param        =  auth-param-name EQUAL
                             ( token / quoted-string )

to

auth-param        =  auth-param-name EQUAL auth-param-value
auth-param-value  =  token / quoted-string

Similarly, the value portion for the following productions could be given names:

info-param
transport-param
user-param
handling-param
domain
opaque
stale
algorithm
Timestamp
via-received

Note that production rules such as:

method-param      =  "method=" Method
ttl-param         =  "ttl=" ttl
maddr-param       =  "maddr=" host
        
don't follow the 

xyz-param = "xyz=" xyz-param-value

pattern, but their param values can be referred to by name (Method, ttl, and host).  

9) Why can't media-range:

media-range    =  ( "*/*"
                          / ( m-type SLASH "*" )
                          / ( m-type SLASH m-subtype )
                          ) *( SEMI m-parameter )
be defined as

media-range    =  ( m-type SLASH m-subtype )
                         *( SEMI m-parameter )

since m-type and m-subtype can both match "*"?

I'd appreciate knowing where I have gone off the rails in any of the above analysis/suggestions.  

Thanks,

--
Bill Hall            Email: [email-masked]
Motorola             Voice: 480-732-2292
2501 S. Price Rd.     Page: 1-800-759-8352
Chandler, AZ  85248         pin 1367646

_______________________________________________
Sip mailing list  https://www1.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [email-masked] for questions on current sip
Use [email-masked] for new developments on the application of sip