Subject: [MMUSIC] Problems with RFC 2327 vs RFC 4566, and between 4567 and 4568

From: Randell Jesup <[email-masked]>
Date: Mon, 04 Jun 2007 15:21:55 -0400

"Dan Wing" <[email-masked]> writes:
>> fyi,
>> I did some analysis in scope of SDP usage at H.248 interfaces:
>> 
>> AVD-3020: Comparison of SDP variants between RFC 4566 and RFC 2327
>> 
>> http://ftp3.itu.int/av-arch/avc-site/2005-2008/0703_She/AVD-3020.zip
>
>Very nice analysis.

One item that's mentioned obliquely but not commented upon is the change
in token, and the impact that has on attribute (a=) names.  We've run
across a problem talking to a vendor's RFC 2327 implementation, which
rejects any invite with a=key-mgmt.  Their point-of-view is that dashes
in an attribute name violate the BNF in RFC 2327.  They suggest using
a=keymgmt (which no one else uses, and isn't registered).

My commentary:
RFC 2327 is internally inconsistent, in that it allows X-* attributes but
the BNF given doesn't allow them (alpha-numerics only for attribute names).
In this case, the text would seem to override the BNF, at least for X-.
The text also says to ignore unknown atributes, but that assumes you've
been able to parse the attribute name.

RFC 4566 allows '-' (and a lot of other characters) in attribute names.
There's no discussion of the compatibility impact of this.

RFC 4567 defines a=key-mgmt, with no reference to RFC 2327.  RFC 4568
suggests people use "a=keymgt" (referencing RFC 4567), which doesn't
actually exist in RFC 4567(!)

Houston, we have a problem.  :-)

So: how do we resolve the disagreements between specs (4567 and 4568 in
particular), and why is there no commentary in 4566 or especially 4567
about compatibility with attribute names?  And why was a name chosen that
might break backwards compatibility?  Was this issue known before it became
an RFC?  And how will this be handled for the next attribute named?  Should
we suggest people stay away from non-alpha-numerics in attribute names?

And what are our options dealing with a *strict* 2327 grammar
implementation?  They don't expect to support 4566 until the end of 2009,
if it doesn't slip.... :-(   We could use a "private" attribute as suggested,
at least for them, but then interconnections with other networks may be an
issue, and that would be a violation (though very minor) of RFC 4566.

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

_______________________________________________
mmusic mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/mmusic

---

From: Randell Jesup <[email-masked]>
Date: Mon, 04 Jun 2007 16:12:37 -0400

"Dan Wing" <[email-masked]> writes:
>Randell Jesup wrote:
>> One item that's mentioned obliquely but not commented upon is 
>> the change in token, and the impact that has on attribute (a=) names.  
>> We've run across a problem talking to a vendor's RFC 2327 implementation, 
>> which rejects any invite with a=key-mgmt.  Their point-of-view is that 
>> dashes in an attribute name violate the BNF in RFC 2327.  They suggest 
>> using a=keymgmt (which no one else uses, and isn't registered).
>
>Such an implementation would also reject:

[ huge list of a= values with dashes removed ]

>This vendor needs to reconsider their desire for interoperability.

Agreed.

>> My commentary:
>> RFC 2327 is internally inconsistent, in that it allows X-* 
>> attributes but the BNF given doesn't allow them (alpha-numerics 
>> only for attribute names).  In this case, the text would seem to 
>> override the BNF, at least for X-.  The text also says to ignore 
>> unknown atributes, but that assumes you've been able to parse 
>> the attribute name.
>> 
>> So: how do we resolve the disagreements between specs (4567 
>> and 4568 in particular), and why is there no commentary in 4566 
>> or especially 4567 about compatibility with attribute names?  And 
>> why was a name chosen that might break backwards compatibility?  Was 
>> this issue known before it became an RFC?  And how will this be 
>> handled for the next attribute named?  Should we suggest people 
>> stay away from non-alpha-numerics in attribute names?
>
>I think it's too late for such encouragement.

Sure, but there's still an issue here, or at least something needing
clarification: Is RFC 4568 trying to refer to a=key-mgmt when it uses
a=keymgt?

It seems like RFC 4566 section 10 (Summary of changes since RFC 2327)
should have mentioned this issue, and others where complying with 4566
would make you (at least in theory) not interoperable with RFC 2327.
All 4566 says about it is:

   The ABNF grammar in Section 9 has been extensively revised and
   updated, correcting a number of mistakes and incorporating the RFC
   3266 IPv6 extensions.  Known inconsistencies between the grammar and
   the specification text have been resolved.

>> implementation?  They don't expect to support 4566 until the 
>> end of 2009, if it doesn't slip.... :-(  
>
>Full compliance with RFC4566 isn't necessary to be more liberal
>in accepting "-" in an unknown attribute name.

Agreed.

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 4 Jun 2007 12:58:44 -0700

> -----Original Message-----
> From: Randell Jesup [mailto:[email-masked]] 
> Sent: Monday, June 04, 2007 12:22 PM
> To: Dan Wing
> Cc: [email-masked]; 
> [email-masked]; [email-masked]
> Subject: Problems with RFC 2327 vs RFC 4566, and between 4567 and 4568
> 

  a=rtcp-xr 		(RFC3611, standards track)
  a=phone-context 	(RFC2848, standards track)
  a=Q763-nature 		(RFC2848, standards track)
  a=Q763-plan 		(RFC2848, standards track)
  a=source-filter		(RFC4570, standards track)
  a=rtcp-fb			(RFC4585, standards track)

Working Group Internet Drafts:
  a=accept-types        (draft-ietf-mmusic-file-transfer-mech)
				(draft-ietf-simple-message-sessions)
				(draft-niemi-simple-chat)
  a=accept-wrapped-types(draft-ietf-mmusic-file-transfer-mech)
				(draft-niemi-simple-chat)
  a=file-selector		(draft-ietf-mmusic-file-transfer-mech)
  a=file-disposition    (draft-ietf-mmusic-file-transfer-mech)
  a=file-date		(draft-ietf-mmusic-file-transfer-mech)
  a=file-icon	      (draft-ietf-mmusic-file-transfer-mech)
  a=file-range          (draft-ietf-mmusic-file-transfer-mech)
  a=loopback-source     (draft-ietf-mmusic-media-loopback)
  a=loopback-mirror     (draft-ietf-mmusic-media-loopback)
  a=loopback-mode   	(draft-ietf-mmusic-media-loopback)
  a=ice-pwd             (draft-ietf-mmusic-ice)
  a=ice-ufrag		(draft-ietf-mmusic-ice)
  a=-m			(draft-ietf-mmusic-sdp-capability-negotiation)
  a=dccp-service-code   (draft-ietf-dccp-rtp)

Individual Internet Drafts:
  a=qos-mech-send
(draft-polk-mmusic-qos-mechanism-identification)
  a=qos-mech-recv
(draft-polk-mmusic-qos-mechanism-identification)
  a=flute-tsi		(draft-mehta-rmt-flute-sdp)
  a=flute-ch		(draft-mehta-rmt-flute-sdp)
  a=FEC-declaration	(draft-mehta-rmt-flute-sdp)
  a=FEC-OTI-extension   (draft-mehta-rmt-flute-sdp)
  a=content-desc        (draft-mehta-rmt-flute-sdp)
  a=udp-setup  	 	(draft-saito-mmusic-sdp-ike)
  a=switch-context-id   (draft-einarsson-mmusic-rtsp-macuri)
  a=ssrc-group          (draft-lennox-mmusic-sdp-source-attributes)
  a=floor-control       (draft-even-xcon-pnc)
  a=zrtp-zid		(draft-zimmermann-avt-zrtp)
  a=zrtp-sas		(draft-zimmermann-avt-zrtp)

> We could use a "private" attribute as suggested, at least for 
> them, but then interconnections with other networks may be an

Untenable, as this isn't the only attribute name affected by a 
strict parser.

-d

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 4 Jun 2007 13:30:50 -0700

...
> clarification: Is RFC 4568 trying to refer to a=key-mgmt when it uses

I can only expect that was the intent, yes.  This should be noted 
via http://www.rfc-editor.org/errata.html (which isn't ideal, but it's
all we have).  Might want to ping the authors first to make sure that
was their intent.

>    the specification text have been resolved.

Without going into a case-by-case analysis of those changes, I dunno
if there would be much value in highlighting "-" in attribute names;
highlighting it might cause some readers of the errata to assume that
was the only change, which could make RFC4566 'compliance' worse (if
that was thought by some implementor to be the only substantive 
change to the grammar).

-d

---

From: Christian Groves <[email-masked]>
Date: Tue, 05 Jun 2007 11:40:00 +1000

Hello,

The document Albrecht originally pointed to is now an Appendix to ITU-T 
H.248.49. This was done to ensure that Albrecht's work was captured. The 
latest draft can be found at:
http://ftp3.itu.int/av-arch/avc-site/2005-2008/0703_She/TD-62.zip

I think Randell's point that the use of "-" in attribute names between 
RFC2327 and RFC4566 should also be captured in the Appendix. Better that 
these sorts of issues are documented. Something along the lines of:

        "I.2.2.1 RFC 4566, “a=” attribute

The syntax of RFC2327 does not support the use of “-“ in an attribute 
name however the syntax of RFC4566 has been updated to support the 
inclusion of “-“. Therefore the use of attribute names containing “-“ is 
problematic for RFC2327 implementations however several examples of 
attribute names containing “-“ were registered prior to the definition 
of RFC4566. RFC2327 Implementors may consider exceptions when parsing an 
“a=” where these attribute names containing “-“ are involved. "

Another way to lessen the problem would be that the internet drafts 
using "-" in the attribute name remove "-" from the draft before going 
RFC and the IANA registration procedures be updated to ensure that "-" 
isn't used in the future for names. From looking at the IANA 
registrations the widespread use of "-" seems to be a relatively recent 
phenomena. I guess those who have implemented the drafts wouldn't be 
happy though :).....

Regards, Christian

---

From: "Dan Wing" <[email-masked]>
Date: Mon, 4 Jun 2007 19:22:36 -0700

I agree such issues should be documented.  Add a caveat like:

    Beyond the addition of "-" in attribute names, there are
    additional grammar differences between RFC2327 and
    RFC4566; enumerating those changes is left as an exercise
    for the reader

and everything would be covered.

-d

> From: Christian Groves [mailto:[email-masked]] 
> Sent: Monday, June 04, 2007 6:40 PM
> To: Dan Wing
> Cc: 'Randell Jesup'; [email-masked]; 
> [email-masked]; [email-masked]
> Subject: Re: [MMUSIC] RE: Problems with RFC 2327 vs RFC 4566, 
> and between 4567 and 4568
> 
> Hello,
> 
>         "I.2.2.1 RFC 4566, "a=" attribute
> 
> The syntax of RFC2327 does not support the use of "-" in an attribute 
> inclusion of "-". Therefore the use of attribute names 
> containing "-" is 
> attribute names containing "-" were registered prior to the 
> definition 
> "a=" where these attribute names containing "-" are involved. "
>

---

From: "Christer Holmberg (JO/LMF)" <[email-masked]>
Date: Tue, 5 Jun 2007 07:32:32 +0200

Hi,

Another difference between 2327 and 4566 is that some of the media types were removed from 4566, based on claims they aren't used anywhere. 

At least one of them is.

But, since that didn't affect the ABNF I guess it shouldn't be a problem from a parser perspective.

Regards,

Christer 

> From: [email-masked] 
> [mailto:[email-masked]] On Behalf 
> Of Dan Wing
> Sent: 5. kesäkuuta 2007 5:23
> To: 'Christian Groves'
> Cc: [email-masked]; [email-masked]; 
> [email-masked]
> Subject: Re: [Sip-implementors] [MMUSIC] RE: Problems with 
> RFC 2327 vs RFC4566, and between 4567 and 4568
> 
> > Subject: Re: [MMUSIC] RE: Problems with RFC 2327 vs RFC 4566, and 
> > between 4567 and 4568
> > 
> > Hello,
> > 
> > The document Albrecht originally pointed to is now an Appendix to 
> > ITU-T H.248.49. This was done to ensure that Albrecht's work was 
> > captured. The latest draft can be found at:
> containing "-" 
> > is problematic for RFC2327 implementations however several 
> examples of 
> > of RFC4566. RFC2327 Implementors may consider exceptions 
> when parsing 
> > an "a=" where these attribute names containing "-" are involved. "
> > 
> > registrations the widespread use of "-" seems to be a relatively 
> > recent phenomena. I guess those who have implemented the drafts 
> > wouldn't be happy though :).....
> > 
> > Regards, Christian
> > 
> > Dan Wing wrote:
> > > ...
> > >   
> Sip-implementors mailing list
> [email-masked]
> https://lists.cs.columbia.edu/cucslists/listinfo/sip-implementors
>

---

From: Colin Perkins <[email-masked]>
Date: Tue, 5 Jun 2007 09:57:33 +0100

On 5 Jun 2007, at 02:40, Christian Groves wrote:
> I think Randell's point that the use of "-" in attribute names  
> between RFC2327 and RFC4566 should also be captured in the  
> Appendix. Better that these sorts of issues are documented.  
> Something along the lines of:
>
>
> The syntax of RFC2327 does not support the use of “-“ in an  
> attribute name however the syntax of RFC4566 has been updated to  
> support the inclusion of “-“. Therefore the use of attribute names  
> containing “-“ is problematic for RFC2327 implementations however  
> several examples of attribute names containing “-“ were registered  
> prior to the definition of RFC4566. RFC2327 Implementors may  
> consider exceptions when parsing an “a=” where these attribute  
> names containing “-“ are involved. "

It's not that simple, since RFC 2327 is inconsistent. The ABNF allows  
only alphanumeric characters in attribute names, but the text only  
states that attribute names "must be in the US-ASCII subset of  
ISO-10646/UTF-8".

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Tue, 5 Jun 2007 10:01:37 +0100

Christer,

There were some media type values referenced in RFC 2327, but removed  
in RFC 4566. This was because they were under-specified, due to not  
being valid top-level media types, not because they were unused. If  
they're used, then someone should document their semantics (i.e.  
register them as top-level media types).

Colin

On 5 Jun 2007, at 06:32, Christer Holmberg (JO/LMF) wrote:
> Another difference between 2327 and 4566 is that some of the media  
> types were removed from 4566, based on claims they aren't used  
> anywhere.
>
> At least one of them is.
>

-- 
Colin Perkins
http://csperkins.org/

---

From: Christian Groves <[email-masked]>
Date: Tue, 05 Jun 2007 20:55:39 +1000

Hello Dan,

Are you referring to additional differences between the grammar of 
attribute names or differences between the RFC2327 and RFC4566 syntax in 
general?

If you are referring to the general case as H.248.49 Appendix I.2 (and 
in particular I.2.4) does enumerate the other changes. I think your text 
is good for a stand alone description of the problem but given that it 
will be integrated into the existing document I think the spirit of your 
proposal is covered in the existing text.

Regards,
Christian

---

From: Christian Groves <[email-masked]>
Date: Tue, 05 Jun 2007 21:13:09 +1000

Hello Colin,

Thanks for the extra information. Would the following text capture your 
clarification?

Dan: I've also added some additional text to address your proposal.

The ABNF syntax of RFC2327 does not support the use of "-" in an attribute 
name however despite stating that attributes names must be in the US-ASCII subset of ISO-10646/UTF-8. The ABNF syntax of RFC4566 was updated to support the 
inclusion of "-" to address this inconsistency. Therefore the use of attribute names containing "-" is problematic for RFC2327 implementations as several examples of attribute names containing "-" were registered prior to the 
definition of RFC4566. RFC2327 Implementors may consider exceptions when 
parsing an "a=" where attribute names containing "-" are involved.

Beyond the addition of "-" in attribute names, the RFC4566 ABNF "token" syntax  defines additional characters (see item 22/I.2.4.2) that would also pose similar problems."

Regards, Christian

Colin Perkins wrote:
>> between RFC2327 and RFC4566 should also be captured in the Appendix. 
>> Better that these sorts of issues are documented. Something along the 
>> lines of:
>>
>>
>> inclusion of “-“. Therefore the use of attribute names containing “-“ 
>> is problematic for RFC2327 implementations however several examples 
>> of attribute names containing “-“ were registered prior to the 
>> definition of RFC4566. RFC2327 Implementors may consider exceptions 
>> when parsing an “a=” where these attribute names containing “-“ are 
>> involved. "
>

---

From: Colin Perkins <[email-masked]>
Date: Tue, 5 Jun 2007 14:49:59 +0100

Christian,

The text, as with much of the text in this document, gives the  
impression that there are meaningful "RFC 2327 implementations", and  
that the compatibility issues are due to RFC 4566 changing the  
specification. I think that's greatly misleading, and gives the  
impression that there are more issues than actually exist.

In reality, RFC 2327 is self-contradictory and unclear in a number of  
places, and useful implementations rely on a large number of  
extensions and interpretations of the standard. RFC 4566, while not  
perfect, greatly clarifies the text to fix those problems, and to  
reflect existing practice. It's my belief that there are no more  
compatibility issues between an "RFC 2327 implementation" and one  
based on RFC 4566, than between any two RFC 2327 implementations.

Colin

On 5 Jun 2007, at 12:13, Christian Groves wrote:
> Thanks for the extra information. Would the following text capture  
> your clarification?
>
> The ABNF syntax of RFC2327 does not support the use of "-" in an  
> attribute name however despite stating that attributes names must  
> be in the US-ASCII subset of ISO-10646/UTF-8. The ABNF syntax of  
> RFC4566 was updated to support the inclusion of "-" to address this  
> inconsistency. Therefore the use of attribute names containing "-"  
> is problematic for RFC2327 implementations as several examples of  
> when parsing an "a=" where attribute names containing "-" are  
> involved.
>
> Beyond the addition of "-" in attribute names, the RFC4566 ABNF  
> "token" syntax  defines additional characters (see item 22/I.2.4.2)  
> that would also pose similar problems."
>
> Regards, Christian
>
> Colin Perkins wrote:
>>> support the inclusion of “-“. Therefore the use of attribute  
>>> names containing “-“ is problematic for RFC2327 implementations  
>>> however several examples of attribute names containing “-“ were  
>>> registered prior to the definition of RFC4566. RFC2327  
>>> Implementors may consider exceptions when parsing an “a=” where  
>>> these attribute names containing “-“ are involved. "
>>
>> It's not that simple, since RFC 2327 is inconsistent. The ABNF  
>> allows only alphanumeric characters in attribute names, but the  
>> text only states that attribute names "must be in the US-ASCII  
>> subset of ISO-10646/UTF-8".
>>
>

-- 
Colin Perkins
http://csperkins.org/

---

From: "Joe Stone \(joestone\)" <[email-masked]>
Date: Tue, 5 Jun 2007 10:17:25 -0700

If we're keeping score, here's a RFC 2327 <-> 4566 compatibility
issue...

RFC 4566 opted to add the following sentence to the definition of the
"a=fmtp" SDP attribute,

	At most one instance of this attribute is allowed for each
format.

There are countless I-Ds and RFCs which leverage multiple instances of
the "a=fmtp" SDP attribute per media format. 

Joe

-----Original Message-----
From: Colin Perkins [mailto:[email-masked]] 
Sent: Tuesday, June 05, 2007 8:50 AM
To: Christian Groves
Cc: Dan Wing (dwing); [email-masked]; [email-masked];
[email-masked]
Subject: Re: [MMUSIC] RE: Problems with RFC 2327 vs RFC 4566,and between
4567 and 4568

Christian,

In reality, RFC 2327 is self-contradictory and unclear in a number of
places, and useful implementations rely on a large number of extensions
and interpretations of the standard. RFC 4566, while not perfect,
greatly clarifies the text to fix those problems, and to reflect
existing practice. It's my belief that there are no more compatibility
issues between an "RFC 2327 implementation" and one based on RFC 4566,
than between any two RFC 2327 implementations.

Colin

On 5 Jun 2007, at 12:13, Christian Groves wrote:
> attribute name however despite stating that attributes names must be 
> in the US-ASCII subset of ISO-10646/UTF-8. The ABNF syntax of
> attribute names containing "-" were registered prior to the definition

> of RFC4566. RFC2327 Implementors may consider exceptions when parsing 
> an "a=" where attribute names containing "-" are involved.
>

>>> Better that these sorts of issues are documented.
>>> The syntax of RFC2327 does not support the use of "-" in an  
>>> support the inclusion of "-". Therefore the use of attribute  
>>> names containing "-" is problematic for RFC2327 implementations  
>>> however several examples of attribute names containing "-" were  
>>> Implementors may consider exceptions when parsing an "a=" where  
>>> these attribute names containing "-" are involved. "
>>

-- 
Colin Perkins
http://csperkins.org/

---

From: Colin Perkins <[email-masked]>
Date: Tue, 5 Jun 2007 21:15:27 +0100

> 	At most one instance of this attribute is allowed for each

Yes, because otherwise the usual mapping to media type parameters is  
lost.

> the "a=fmtp" SDP attribute per media format.

The only case I'm aware of is a mistake RFC 3189. Do you know of  
other examples?

Colin

---

From: Christian Groves <[email-masked]>
Date: Wed, 06 Jun 2007 12:17:59 +1000

Hello Colin,

I agree there are going to be compatibility issues between RFC2327 
implementations, or RFC4566 implementations or SIP implementations etc. 
If humans are involved then there will be some error. Its not the scope 
of the Appendix to discuss this.

The Appendix documents the compatibility issues which resulted from the 
text of the RFC upgrade. There are compatibility issues between the text 
of the two RFCs.  Even if  I had a "meaningful" RFC2327 implementation 
(based on extensions and "interpretations") the  result of adopting 
RFC4566 would be that I would need to look to see what the impacts are. 
The place I would start to look is by comparing the two RFCs for 
changes. Isn't it better to highlight what the differences are so that 
they are known?

Several people have commented that Albrecht's analysis has been helpful 
for them because the summary of changes documented in RFC4566 wasn't 
detailed enough. That's partly why we've adopted it. Albrecht's analysis 
is usually thorough and methodical so I can't see that there was any 
desire to be misleading. If there's specific text in the Appendix that's 
erroneous or misleading please flag it so we can address it, or if you 
have some modified text even better :) . As co-author of RFC4566 I'm 
sure people will appreciate any extra guidance you can propose for the 
text.

Regards, Christian

Colin Perkins wrote:
>> in the US-ASCII subset of ISO-10646/UTF-8. The ABNF syntax of RFC4566 
>> was updated to support the inclusion of "-" to address this 
>> inconsistency. Therefore the use of attribute names containing "-" is 
>> problematic for RFC2327 implementations as several examples of 
>> when parsing an "a=" where attribute names containing "-" are involved.
>>
>>> allows only alphanumeric characters in attribute names, but the text 
>>> only states that attribute names "must be in the US-ASCII subset of 
>>> ISO-10646/UTF-8".
>>>
>>