Subject: [Sip] Question regarding conflicting grammar for IPV6 SIP URI and RFC 3986

From: "Szilagyi, Mike" <[email-masked]>
Date: Thu, 18 Dec 2008 10:52:37 -0500

RFC 3986 defines the following grammar for a URI:

   URI           = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

   hier-part     = "//" authority path-abempty
                 / path-absolute
                 / path-rootless
                 / path-empty

A SIP URI falls into the 'path-rootless' rule:

   path-rootless = segment-nz *( "/" segment )

   segment-nz    = 1*pchar

   pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"

   pct-encoded   = "%" HEXDIG HEXDIG

   unreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~"

   sub-delims    = "!" / "$" / "&" / "'" / "(" / ")"

                 / "*" / "+" / "," / ";" / "="

path-rootless does not allow "[" or "]" characters, so the SIP-URI grammar for IPV6 violates RFC 3986 grammar.

SIP-URI          =  "sip:" [ userinfo ] hostport
                    uri-parameters [ headers ]

hostport         =  host [ ":" port ]
host             =  hostname / IPv4address / IPv6reference
hostname         =  *( domainlabel "." ) toplabel [ "." ]
domainlabel      =  alphanum
                    / alphanum *( alphanum / "-" ) alphanum
toplabel         =  ALPHA / ALPHA *( alphanum / "-" ) alphanum
IPv4address    =  1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT
IPv6reference  =  "[" IPv6address "]"  <-- *** violates RFC 3986 ***
IPv6address    =  hexpart [ ":" IPv4address ]
hexpart        =  hexseq / hexseq "::" [ hexseq ] / "::" [ hexseq ]
hexseq         =  hex4 *( ":" hex4)
hex4           =  1*4HEXDIG
port           =  1*DIGIT

These, though valid SIP URIs are not valid URIs:

    <sip:[2001:db8::10]>
    <sip:caller@[2001:db8::1]>

Has this already been addressed?

Regards,
Mike
_______________________________________________
Sip mailing list  https://www.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [email-masked] for questions on current sip
Use [email-masked] for new developments on the application of sip

---

From: "Brett Tate" <[email-masked]>
Date: Thu, 18 Dec 2008 16:31:27 -0500

Greetings,
 
Mike's interpretation looks correct.  Is this something that should be
fixed?  If so, should the fix be placed within
draft-ietf-sip-ipv6-abnf-fix?
 
Thanks,
Brett

________________________________

	From: [email-masked] [mailto:[email-masked]] On
Behalf Of Szilagyi, Mike
	Sent: Thursday, December 18, 2008 10:53 AM
	To: [email-masked]
	Subject: [Sip] Question regarding conflicting grammar for IPV6
SIP URI andRFC 3986

	   URI           = scheme ":" hier-part [ "?" query ] [ "#"
fragment ]

	                 / path-absolute

	                 / path-rootless

	                 / path-empty

	   segment-nz    = 1*pchar

	   pchar         = unreserved / pct-encoded / sub-delims / ":" /
"@"

	path-rootless does not allow "[" or "]" characters, so the
SIP-URI grammar for IPV6 violates RFC 3986 grammar.

	domainlabel      =  alphanum

	IPv4address    =  1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT "."
1*3DIGIT

	IPv6reference  =  "[" IPv6address "]"  <-- *** violates RFC 3986
***

	hexpart        =  hexseq / hexseq "::" [ hexseq ] / "::" [
hexseq ]

	hex4           =  1*4HEXDIG

	port           =  1*DIGIT

	    <sip:[2001:db8::10]>

	Regards,

	Mike

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 18 Dec 2008 17:37:11 -0500

Brett Tate wrote:
> fixed?  If so, should the fix be placed within draft-ietf-sip-ipv6-abnf-fix?

Funny - this just came up in another conversation I was having.
It was pointed out to me that the 3261 ABNF for the sip URI is probably 
derived from RFC-2396 which has since been obsoleted by RFC-3986. 
RFC-3987 explicitly allows the [ ] notation for IPv6.

So yeah, the URL syntax should probably be corrected.

	Thanks,
	Paul

> Thanks,
> Brett
> 
>     ------------------------------------------------------------------------
>     *From:* [email-masked] [mailto:[email-masked]] *On
>     Behalf Of *Szilagyi, Mike
>     *Sent:* Thursday, December 18, 2008 10:53 AM
>     *To:* [email-masked]
>     *Subject:* [Sip] Question regarding conflicting grammar for IPV6 SIP
>     URI andRFC 3986
> 
>     A SIP URI falls into the ‘path-rootless’ rule:
> 
>      
> 
>     path-rootless does not allow “[“ or “]” characters, so the SIP-URI
>     grammar for IPV6 violates RFC 3986 grammar.
> 
>      
> 
> ------------------------------------------------------------------------
>

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Thu, 18 Dec 2008 16:42:50 -0600

> draft-ietf-sip-ipv6-abnf-fix?

What should the fix be?  Mike's claim is that rfc3261
violates rfc3986, yet rfc3986 is internally consistent
in that it requires IPv6 literals to be enclosed in "[" and "]",
viz:

    IP-literal = "[" ( IPv6address / IPvFuture  ) "]"

At the same time, rfc3261 is internally consistent in that
it requires IPv6reference to be enclosed in "[" and "]", viz:

   IPv6reference  =  "[" IPv6address "]"

Furthermore, I don't think the intent is to produce SIP-URI (as
defined in rfc3261) from a URI (as defined in rfc3986), is it?
Note that rfc3986 defines URI as:

   URI = scheme ":" hier-part ...
   heir-part = "//" ...

If this was true, a SIP URI would need to be produced as:

   sip://[2001:db8:10] ...

In other words, I am trying to understand what the exact
problem is before we try to figure out where to put the
fix in...

Thanks,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
Web:   http://ect.bell-labs.com/who/vkg/
_______________________________________________
Sip mailing list  https://www.ietf.org/mailman/listinfo/sip
This list is for NEW development of the core SIP Protocol
Use [email-masked] for questions on current sip
Use [email-masked] for new developments on the application of sip

---

From: "Brett Tate" <[email-masked]>
Date: Thu, 18 Dec 2008 18:25:58 -0500

Intentional or not, the sip-uri (excluding IPv6reference) appears to not
violate extending rfc2396's absoluteURI because the definition was a
structured subset of opaque_part.  However the introduction of brackets
to enclose an IPv6 address causes the sip-uri to no longer be a valid
absoluteURI (rfc2396) or absolute-URI (rfc3986).

I'm currently not sure if a fix is really needed or how best to correct
it.  Escaping the brackets might work; however that might cause more
headaches.

> -----Original Message-----
> From: Vijay K. Gurbani [mailto:[email-masked]] 
> Sent: Thursday, December 18, 2008 5:43 PM
> To: Brett Tate
> Cc: [email-masked]
> Subject: Re: [Sip] Question regarding conflicting grammar for 
> IPV6 SIP URI andRFC 3986
> 
> Brett Tate wrote:
> > Greetings,
> >  
> violates rfc3986, yet rfc3986 is internally consistent in 
> that it requires IPv6 literals to be enclosed in "[" and "]",
> viz:
> 
> At the same time, rfc3261 is internally consistent in that it 
> requires IPv6reference to be enclosed in "[" and "]", viz:
> 
> Furthermore, I don't think the intent is to produce SIP-URI 
> (as defined in rfc3261) from a URI (as defined in rfc3986), is it?
> problem is before we try to figure out where to put the fix in...
> 
> Thanks,
> 
> - vijay
> --
> Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent 1960 
> Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)

---

From: Hadriel Kaplan <[email-masked]>
Date: Thu, 18 Dec 2008 18:37:22 -0500

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Brett Tate
> Sent: Thursday, December 18, 2008 6:26 PM
> To: Vijay K. Gurbani; [email-masked]
> Subject: Re: [Sip] Question regarding conflicting grammar for IPV6 SIP URI
> andRFC 3986
>

Section 19.1.2 of 3261:
   Note that character escaping is not allowed in the host component of
   a SIP or SIPS URI (the % character is not valid in its expansion).

-hadriel

---

From: "Hisham Khartabil" <[email-masked]>
Date: Fri, 19 Dec 2008 11:08:41 +1100

RFC3896, section 3.2.2 says:

"A host identified by an Internet Protocol literal address, version 6
   [RFC3513] or later, is distinguished by enclosing the IP literal
   within square brackets ("[" and "]").  This is the only place where
   square bracket characters are allowed in the URI syntax."

It looks like RFC3896 has the error, the definition for URI needs to updated
to be the following:

                  / authority

Regards,
Hisham

---

From: Hadriel Kaplan <[email-masked]>
Date: Thu, 18 Dec 2008 19:09:57 -0500

The SIP URI is not the only one with this problem - RFC 3860 (im-URI), RFC 3859 (pres-URI), RFC 3508 (h323-URI), and who knows what else violate that rule.

Seems to me any "fixing" should be done to RFC 3986, probably as an errata.

-hadriel

> Vijay K. Gurbani

---

From: Paul Kyzivat <[email-masked]>
Date: Thu, 18 Dec 2008 19:14:36 -0500

> absoluteURI (rfc2396) or absolute-URI (rfc3986).

3986 accomodates IPv6 addresses within in "authority" which seems to 
serve roughly the same role as "[ userinfo ] hostport" in SIP-URI. But 
"authority" is always preceded by "//", so it doesn't work for sip.

To fit a sip uri within 3986 I think you must start with:

    hier-part     = path-rootless

    segment-nz    = 1*pchar

Not only does the above exclude the IPv6 syntax, it also excludes lots 
of other things that are commonly used in the userinfo portion of sip URIs.

I don't know if this is of practical concern or not. The SIP URI is 
defined by 3261. Certainly 3261 didn't call for it to be compatible with 
3986. Yet it would be *nice* if it was for compatibility with software 
that treats URIs, including SIP URIs generically. But I don't know how 
important that is in practice.

To fit 3986 well, I think would have to aim for fitting into:

       authority   = [ userinfo "@" ] host [ ":" port ]

But that would require SIP-URIs like sip://[email-masked]
and I don't think we are going to migrate to that any time soon.

Don't mailto URIs have similar problems? What are they doing about it?

	Thanks,
	Paul

---

From: "Brett Tate" <[email-masked]>
Date: Fri, 19 Dec 2008 08:25:23 -0500

Hi Hadriel,

Yes, that is one of the headaches. :)  

> Behalf Of 
> > Subject: Re: [Sip] Question regarding conflicting grammar 
> for IPV6 SIP 
> > URI andRFC 3986
> >
> > Intentional or not, the sip-uri (excluding IPv6reference) 
> appears to 
> > not violate extending rfc2396's absoluteURI because the 
> definition was 
> > a structured subset of opaque_part.  However the introduction of 
> > brackets to enclose an IPv6 address causes the sip-uri to 
> no longer be 
> > a valid absoluteURI (rfc2396) or absolute-URI (rfc3986).
> >
> > I'm currently not sure if a fix is really needed or how best to 
> > correct it.  Escaping the brackets might work; however that might 
> > cause more headaches.
> 
> Section 19.1.2 of 3261:

---

From: "Brett Tate" <[email-masked]>
Date: Fri, 19 Dec 2008 09:15:59 -0500

> Not only does the above exclude the IPv6 syntax, 
> it also excludes lots of other things that are 
> commonly used in the userinfo portion of sip URIs.

Like what?  The following is the sub-delims expansion:

sub-delims = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" /
"="

<snip>

> What are they doing about it?

I assume sip is not the only URI with the issue; however I haven't
checked.  

Here is the IANA list: http://www.iana.org/assignments/uri-schemes.html

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Fri, 19 Dec 2008 13:45:59 -0600

All: So, to summarize a WG position on this and chart a course
forward to close this item, these are the facts:

1) RFC3261 is internally consistent in specifying IPv6 literals
  (i.e., enclosed in "[" and "]").

2) RFC3986 is internally consistent in IPv6 literals as well.

3) It was not the intention of rfc3261 to make the SIP URI
  compatible with rfc2396 URI (rfc3986 came later.)  And in fact,
  if we made the SIP URI compatible with rfc3986, we won't get
  too far for the reason I pointed out early:

       sip://[2001:db8:10] ...

  Which, I presume, is not going to happen any time soon (and
  as Hadriel points out, other URIs (im, pres, etc.) suffer
  the same fate.

Therefore, the only conclusion we can arrive to is that
rfc3986 should be modified.

If so, then what should the modification be and who should
do it?  For the "who": Given that rfc3986 did not come out of a
WG, there is no mailing list to address this issue on.  So,
the best we can do is as Hadriel suggested: issue an errata.

For the "what", any suggestions?  I think Hisham's suggestion
is probably the least intrusive one to rfc3986's ABNF while
allowing one to generate rfc3261 SIP URIs.  To recap:

OLD:

NEW:
URI           = scheme ":" hier-part [ "?" query ] [ "#" fragment ]
hier-part     =  "//" authority path-abempty
                  / path-absolute
                  / path-rootless
                  / path-empty
                  / authority   ***** Newly added line *****

Does this cause any other problems?

Thoughts, comments?  If you agree, please say so; that way
we can summarize this as the WG position allowing for the
next steps to happen (i.e., file errata, etc.)

Thanks,

- vijay

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 19 Dec 2008 14:55:51 -0500

Vijay,

I agree with the direction - to change 3986.
I *think* the proposed syntax sounds right, though I haven't studied the 
grammar carefully to ensure that it is then unambiguous. (But I think I 
recall seeing that it is already ambiguous and falls back on the "first 
matching production wins" to resolve that conflict.)

	Thanks,
	Paul

---

From: Hadriel Kaplan <[email-masked]>
Date: Fri, 19 Dec 2008 15:04:26 -0500

It's ambiguous with path-rootless, no?
But anyway I think the AD's should weigh in on what WG should work on the fix.  The fix needs to be applicable to all URIs, and I don't think we have all the right folks to know what the rfc3986 semantics really means generally.

-hadriel

> From: [email-masked] [mailto:[email-masked]] On Behalf Of Paul
> Kyzivat
> Sent: Friday, December 19, 2008 2:56 PM
> To: Vijay K. Gurbani
> Cc: IETF SIP List
> Subject: Re: [Sip] Summary (Was: [Re: Question regarding conflicting
> grammar for IPV6 SIP URI andRFC 3986])
>
> Vijay,
>

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Fri, 19 Dec 2008 15:11:40 -0600

> It's ambiguous with path-rootless, no?

I don't think so.  I don't think you can generate a token with
"[" and "]" using the path-rootless production rule (unless
they are escaped.)

> But anyway I think the AD's should weigh in on what WG should work on
> the fix.  The fix needs to be applicable to all URIs, and I don't
> think we have all the right folks to know what the rfc3986 semantics
> really means generally.

Understood; however, I was asked by the chairs to round up
a WG summary on this one.  I suspect that once we do so,
the chairs will get the ADs involved to move this along in
the shape they see fit.

Hence, please do eyeball the path-rootless production rule
and make sure my understanding is correct.

Thanks,

- vijay

---

From: Hadriel Kaplan <[email-masked]>
Date: Fri, 19 Dec 2008 16:15:18 -0500

> Sent: Friday, December 19, 2008 4:12 PM
>
> Hadriel Kaplan wrote:

Right, but I meant it's ambiguous when it's NOT an IPv6 address.  Which will probably be a lot of the time. :)

Also, I don't really understand what the semantics means for "authority" vs. "path-rootless" in that RFC as defined.  What does an app do differently in those two cases?

-hadriel

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Fri, 19 Dec 2008 15:31:23 -0600

> It's ambiguous with path-rootless, no?

vkg> I don't think so.  I don't think you can generate a token with
vkg> "[" and "]" using the path-rootless production rule (unless
vkg> they are escaped.)

> Right, but I meant it's ambiguous when it's NOT an IPv6 address.
> Which will probably be a lot of the time. :)
> 
> Also, I don't really understand what the semantics means for
> "authority" vs. "path-rootless" in that RFC as defined.  What does an
> app do differently in those two cases?

OK, I see what you mean, and I am not sure I have a good
answer, either.

Presumably we (i.e., the SIP WG) can provide a statement
on why we think that rfc3986 should be changed, and how it
can be changed, but here is a (hopefully small) list of
potential questions and side-effects we don't have an answer
to.

Then have someone in the right authority arbitrate the ABNF
fix.

Thanks,

- vijay

---

From: Dean Willis <[email-masked]>
Date: Sun, 21 Dec 2008 22:28:18 -0600

> Then have someone in the right authority arbitrate the ABNF

Personally, I think the URI/v6 problem has a very large scope. I'd be  
tempted to raise the question on the IETF general list and ask where  
the work should go.

--
Dean

---

From: "Dale Worley" <[email-masked]>
Date: Mon, 22 Dec 2008 17:06:14 -0500

On Thu, 2008-12-18 at 19:09 -0500, Hadriel Kaplan wrote:
> The SIP URI is not the only one with this problem - RFC 3860 (im-URI),
> RFC 3859 (pres-URI), RFC 3508 (h323-URI), and who knows what else
> violate that rule.
> 

Let's get a grip on the origin of the problem.

The generic URI grammar was stimulated by HTTP URIs.  They assume that
*if* the URI contains the designation of a host, it will be flagged by
the initial "//" -- and the syntax allows "all the standard syntaxes for
hosts" in that location.  Now if SIP URIs we constructed on that format,
we'd have "sip://[email-masked]".  But I don't expect that to change
any time soon.

So let's ask, why are SIP URIs written as they are?  Clearly, it's
because they're based on e-mail addresses, or rather, the "mailto:" URI
scheme:  "mailto:[email-masked] --> sip:[email-masked]".  Hmmm,
doesn't that mean that mailto: URIs have the same problem?  And, as
Hadriel notes, *all* the URIs based on the mailto: URI?  What have the
"mailto" people done about this?

That doesn't take long to look up:  Page 2 of RFC 2368 says:

     mailtoURL  =  "mailto:" [ to ] [ headers ]
     to         =  #mailbox
     headers    =  "?" header *( "&" header )
     header     =  hname "=" hvalue
     hname      =  *urlc
     hvalue     =  *urlc

   "#mailbox" is as specified in RFC 822 [RFC822]. This means that it
   consists of zero or more comma-separated mail addresses, possibly
   including "phrase" and "comment" components. Note that all URL
   reserved characters in "to" must be encoded: in particular,
   parentheses, commas, and the percent sign ("%"), which commonly occur
   in the "mailbox" syntax.

So if you want an IPv6 address in your mailto: URI, you're going to have
to write the [...] as %5b...%5d.

As far as I can tell, the only reason that this rule does not extend
(uniformly!) to SIP URIs is that RFC 3261 explicitly forbids it.
Perhaps we could allow %-escapes in host-parts as part of the IPv6
upgrades to our software...

Dale

---

From: Eric Burger <[email-masked]>
Date: Mon, 22 Dec 2008 18:22:42 -0500

URI's would be the domain of APPS, I would think. Introducing the Apps  
AD's to the conversation.

On Dec 19, 2008, at 3:04 PM, Hadriel Kaplan wrote:

> But anyway I think the AD's should weigh in on what WG should work  
> on the fix.  The fix needs to be applicable to all URIs, and I don't

---

From: Cullen Jennings <[email-masked]>
Date: Wed, 31 Dec 2008 09:02:03 -0700

WIth my AD hat on ...

If you want to update/discuss/change/abuse 3986, probably the best  
place to to that would be on the APPS area list at [email-masked]

It would be up to the Apps ADs, but I suspect that change to the BNF  
suggested below is too significant a change to be done with an Errata.

Cullen