Subject: Re: [Sipping] draft-york-sipping-p-charge-info-12: ABNF

From: Dan York <[email-masked]>
Date: Thu, 1 Dec 2011 09:08:38 -0500

Paul,

Many thanks as always for your helpful comments.  The question seems to be that we want to define ABNF that describes an example like this:

P-Charge-Info: <sip:6835555555;npi=001;noa=[email-masked]>

The optional npi and noa parameters are parameters that are on the left side of the @ symbol in a SIP URI.

In looking at the original ABNF we had in there, it does seem to me that it *was* incorrect, as it would have required a URI of something like one of these:

P-Charge-Info: <sip:[email-masked];npi=001;noa=3>
P-Charge-Info: <sip:[email-masked]>;npi=001;noa=3

I'm guessing the former... but it's irrelevant - the point is that either of these would be wrong.  We need the ABNF to specify that THIS is the correct notation:

P-Charge-Info: <sip:6835555555;npi=001;noa=[email-masked]>

So we really do want to specify somehow that "chargeparam" is part of the "userinfo" section of the SIP URI, and perhaps specifically the "user" section given these definitions:

> RFC 3261:
> 
> userinfo = ( user / telephone-subscriber ) [ ":" password ] "@"
> user = 1*( unreserved / escaped / user-unreserved )
> 
> RFC 2806:
> 
> telephone-subscriber = global-phone-number / local-phone-number

Any ABNF experts out there able to help me sort this out?  The current formal definition is:

----
   The syntax of the P-Charge-Info header is described as follows:

         P-Charge-Info = "P-Charge-Info" HCOLON (name-addr / addr-spec)
                 ; name-addr and addr-spec are specified in RFC 3261
             charge-param = npi-param / noa-param / generic-param
             npi-param = ";npi" EQUAL npi-value
                 ; generic-param is specifed in RFC 3261
             npi-value = 3DIGIT
             noa-param = ";noa" EQUAL noa-value
             noa-value = gen-value

   The SIP URI is the billing indicator that is passed between the
   parties.

   charge-param is used as a userinfo parameter in P-Charge-Info.
----

Thanks,
Dan

On Nov 29, 2011, at 8:59 PM, Paul Kyzivat wrote:

> On 11/30/11 3:38 AM, Dan York wrote:
>> Brett, (and replying from a slightly different address so that it will
>> go to the SIPPING list)
>> 
>> Thank you for the feedback and question. The ABNF in the draft has
>> evolved over the past almost-4 years as various people more literate
>> than I in ABNF have given us feedback and we've updated the draft.
>> 
>> In the ABNF section, "chargeparam" is intended to represent that you
>> could optionally have the "noa", "npi" parameters - or any other generic
>> parameters found in RFC 3261(such as "user=phone")
> 
> Including generic-param is a mechanism for making the syntax compatible with future enhancements. But allowing it syntactically doesn't specify how parameters that match generic-param are to be processed if the are present on this header. Typically you would specify in the draft that they should be ignored unless the behavior is defined by some other specification.
> 
>> Originally, the ABNF read:
>> 
>>          P-Charge-Info = "P-Charge-Info" HCOLON (name-addr / addr-spec)*
>>                  (SEMI charge-param)
>>                  ; name-addr and addr-spec are specified inRFC 3261  <http://tools.ietf.org/html/rfc3261>
>>              charge-param = npi-param / noa-param / generic-param
>> 
>> 
>> I thought that was fairly clear and made sense. However, I changed the
>> ABNF in rev -10 in October 2010 to more simply:
>> 
>>          P-Charge-Info = "P-Charge-Info" HCOLON (name-addr / addr-spec)
>>                  ; name-addr and addr-spec are specified inRFC 3261  <http://tools.ietf.org/html/rfc3261>
>>              charge-param = npi-param / noa-param / generic-param
>> 
>> 
>> after someone strongly made the case that the "* (SEMI charge-param)"
>> was not required because it was a "userinfo parameter" to the
>> name-addr/addr-spec element.
> 
> That is something very different. What you have above are *header* parameters for the P-Charge-Info header.
> 
> It sounds like you are talking about TEL-URI parameters when the tel uri has been converted to a sip URI. But if so, then you should be defining an extension to the tel-uri syntax. And then you would need to define the semantics relative to the tel-uri. (It isn't really kosher to define the parameters on the tel-uri but then only define their semantics relative to the P-Charge-Info header.)
> 
> IMO its wrong to make this change. Rather you should go back to defining these explicitly as header params for P-Charge-Info.
> 
> 	Thanks,
> 	Paul
> 
>> Unfortunately, the email exchange about
>> this seems to have NOT taken place on the mailing list but rather in a
>> private email exchange - and I no longer have access to the archives of
>> the email account where that occurred (I am no longer with Voxeo) - so I
>> don't know who it was that argued for this change.
>> 
>> I'm directly cc'ing John Haluska as he was involved in with a number of
>> those exchanges and can perhaps clarify this.
>> 
>> In reviewing section 19.1.1 of RFC 3261 (
>> http://tools.ietf.org/html/rfc3261#section-19.1.1 ) and sections 19.1.2,
>> 19.1.3, and 19.1.6 as well as the ABNF in section 25, I am guessing that
>> the rationale was because the "charge-param" does fit into the "user"
>> section of the URI.
>> 
>> So that's a roundabout way of saying that it is part of "user", as I
>> interpret the ABNF in RFC 3261.
>> 
>> Do you have suggestions for how to make this clearer in the draft? Would
>> the original ABNF be more useful to you? Should the sentence
>> "charge-param is used as a userinfo parameter in P-Charge-Info" indicate
>> that it is the "user" part of the "userinfo" field?
>> 
>> Thanks,
>> Dan
>> 
>> P.S. After not receiving any feedback for many, many months I suddenly
>> have received two email questions/comments about P-Charge-Info today. I
>> don't know if this is as a result of the mention on a mailing list that
>> Richard Shockey mentioned... but I was surprised.
>> 
>> On Nov 29, 2011, at 1:35 PM, Brett Tate wrote:
>> 
>>> Howdy,
>>> 
>>> Draft-york-sipping-p-charge-info-12 includes the following ABNF
>>> without explicitly indicating if the charge-param is part of user,
>>> telephone-subscriber, or both. I'm not sure how to interpret the
>>> charge-param statement since userinfo has no parameters (although user
>>> and telephone-subscriber can have them).
>>> 
>>> Is charge-param part of user, telephone-subscriber, or both? I
>>> recommend updating section 7 to remove the ambiguity.
>>> 
>>> Thanks,
>>> Brett
>>> 
>>> 
>>> ------
>>> 
>>> Draft-york-sipping-p-charge-info-12:
>>> 
>>> "The syntax of the P-Charge-Info header is described as follows:
>>> 
>>> P-Charge-Info = "P-Charge-Info" HCOLON (name-addr / addr-spec)
>>> ; name-addr and addr-spec are specified in RFC 3261
>>> charge-param = npi-param / noa-param / generic-param
>>> npi-param = ";npi" EQUAL npi-value
>>> ; generic-param is specifed in RFC 3261
>>> npi-value = gen-value
>>> noa-param = ";noa" EQUAL noa-value
>>> noa-value = gen-value
>>> 
>>> The SIP URI contained in the name-addr/addr-spec is the billing
>>> indicator that is passed between the parties.
>>> 
>>> charge-param is used as a userinfo parameter in P-Charge-Info."
>>> 
>>> 
>>> RFC 3261:
>>> 
>>> userinfo = ( user / telephone-subscriber ) [ ":" password ] "@"
>>> user = 1*( unreserved / escaped / user-unreserved )
>>> 
>>> RFC 2806:
>>> 
>>> telephone-subscriber = global-phone-number / local-phone-number
>>> 
>> 
>> --
>> Dan York [email-masked] <mailto:[email-masked]>
>> Phone: +1-802-735-1624 skype:danyork
>> http://www.danyork.com/
>> http://twitter.com/danyork
>> 
>> 
>> 
>> --
>> Dan York [email-masked] <mailto:[email-masked]>
>> http://www.danyork.com/ skype:danyork
>> Phone: +1-802-735-1624
>> Twitter - http://twitter.com/danyork
>> --------------------------------------------------------
>> All comments and opinions are entirely my own and have no connection
>> whatsoever to any employer, past or present. Indeed, by tomorrow even I
>> might be disavowing these comments.
>> --------------------------------------------------------
>> 
>> 
>> 
>> _______________________________________________
>> Sipping mailing list  https://www.ietf.org/mailman/listinfo/sipping
>> This list is for NEW development of the application of SIP
>> Use [email-masked] for questions on current sip
>> Use [email-masked] for new developments of core SIP
> 
> _______________________________________________
> Sipping mailing list  https://www.ietf.org/mailman/listinfo/sipping
> This list is for NEW development of the application of SIP
> Use [email-masked] for questions on current sip
> Use [email-masked] for new developments of core SIP

-- 
Dan York  [email-masked]
http://www.danyork.com/   skype:danyork
Phone: +1-802-735-1624
Twitter - http://twitter.com/danyork
--------------------------------------------------------
All comments and opinions are entirely my own and have no connection whatsoever to any employer, past or present. Indeed, by tomorrow even I might be disavowing these comments.
--------------------------------------------------------

---

From: Brett Tate <[email-masked]>
Date: Thu, 1 Dec 2011 06:22:21 -0800

> > Is charge-param part of user, telephone-subscriber, or both?

> The answer for your question is "both", at least 
> that was the intention.

> So that's a roundabout way of saying that it is 
> part of "user", as I interpret the ABNF in RFC 3261.

If the draft's change was requested by Tolga, it sounds there is some disagreement concerning the intended change or interpretation.

> Do you have suggestions for how to make this clearer in the draft?

Yes; after reaching agreement upon where charge-param is supposed to be placed, be more explicit concerning how it fits into the existing ABNF of the specific URI (if not changing back to being a header parameter).

The following is snippet from section 7:
---
The syntax of the P-Charge-Info header is described as follows:

---

The draft indicates SIP URI and mentions userinfo; is the draft also enabling inclusion within tel URI and other URI schemes?  Note the following section 6.3 snippet: "The content of the P-Charge-Info header is typically simply a SIP URI used as a billing indicator."

If charge-param is allowed within telephone-subscriber (of SIP-URI or tel URI), it should explicitly mention RFC 3966 and indicate how it fits within a telephone-subscriber (i.e. charge-param can be used as a new par when included within telephone-subscriber).  And if doing so is only applicable within P-Charge-Info, charge-param should potentially instead be defined as a P-Charge-Info header parameter.

If charge-param is allowed within user of SIP-URI, it should explicitly indicate how it fits within user and how you know to decode the user portion's parameters without having added a new "user=" value or restricting which "user=" values are valid.  If you are relying upon P-Charge-Info instead of "user=" to imply this decode decision, charge-param should potentially instead be defined as a P-Charge-Info header parameter or limited to being a telephone-subscriber parameter.

---

From: Dan York <[email-masked]>
Date: Thu, 1 Dec 2011 11:00:44 -0500

Brett,

On Nov 30, 2011, at 3:41 PM, Brett Tate wrote:

> If charge-param remains within userinfo, generic-param should likely be changed to a more limited extention parameter since UTF8-NONASCII is not allowed within SIP-URI and tel URI because of escaping limitations.  RFC 3986 provides a mechanism for escaping UTF8-NONASCII within newer URI schemes.

DY> Do you have a suggestion for how I would change this to a more limited parameter within the ABNF?  

DY> Alternatively, what if the ABNF remained as is but the formal syntax included a statement below the ABNF that UTF8-NONASCII is not allowed in the "generic-param" due to limitations in the SIP and tel URIs?    Would that not be sufficient enough for implementors to understand?

Thanks,
Dan

> 
> RFC 3261:
> 
> generic-param  =  token [ EQUAL gen-value ]
> 
> gen-value      =  token / host / quoted-string
> 
> quoted-string  =  SWS DQUOTE *(qdtext / quoted-pair ) DQUOTE
> 
> qdtext     =  LWS / %x21 / %x23-5B / %x5D-7E / UTF8-NONASCII
>

---

From: Brett Tate <[email-masked]>
Date: Thu, 1 Dec 2011 12:01:45 -0800

> Do you have a suggestion for how I would change this 
> to a more limited parameter within the ABNF?  

See http://www.ietf.org/mail-archive/web/sipping/current/msg17796.html for more information.

The following is a potential change for RFC 3966's Telephone-Subscriber.  There is no need to add a new expandable parameter; Telephone-Subscriber already has "parameter".  Any new ones should comply with "parameter"; this includes npi-param and noa-param.

   parameter            = ";" pname ["=" pvalue ]
   pname                = 1*( alphanum / "-" )
   pvalue               = 1*paramchar

       npi-value = p-value
       noa-value = p-value

The Telephone-Subscriber's par is expanded to include two new values: npi-param and noa-param.  The following is the resulting ABNF; I don't recall if currently requires IANA registration:

par = parameter / extension / isdn-subaddress / npi-param / noa-param

The following is the recommended change for user; don't do it. :)  However if you need to do it and remain consistent with the "historic" nature of the draft (i.e. not indicating user=phone) or if need to avoid RFC 3966 non compliance, you have a few options although not sure if any would actually pass SIPCORE scrutiny.

1) You can indicate that user=phone (instead of user=ip) is the default for a P-Charge-Info SIP-URI.  Thus, it can be decoded as a Telephone-Subscriber even though user=phone was not added.  This is similar to RFC 3261 section 19.1.1 snippet.

         "The set of valid telephone-subscriber strings is a subset of
         valid user strings.  The user URI parameter exists to
         distinguish telephone numbers from user names that happen to
         look like telephone numbers.  If the user string contains a
         telephone number formatted as a telephone-subscriber, the user
         parameter value "phone" SHOULD be present.  Even without this
         parameter, recipients of SIP and SIPS URIs MAY interpret the
         pre-@ part as a telephone number if local restrictions on the
         name space for user name allow it."

2) You can indicate that user=pci (instead of user=ip) is the default for a P-Charge-Info SIP-URI.

userinfo         =  ( user / telephone-subscriber / p-ci) [ ":" password ] "@"
p-ci = user *charge-param ; not sure if good enough
charge-param = npi-param / noa-param / parameter

user-param        =  "user=" ( "phone" / "ip" / "pci" / other-user)

3) You can indicate that the P-Charge-Info SIP-URI user can be parsed as p-ci when user=ip (i.e. the default).

userinfo         =  ( user / telephone-subscriber) [ ":" password ] "@"
p-ci = user *charge-param ; not sure if good enough
charge-param = npi-param / noa-param / parameter

---

From: Paul Kyzivat <[email-masked]>
Date: Fri, 02 Dec 2011 12:48:20 +0800

On 12/1/11 10:08 PM, Dan York wrote:
> Paul,
>
> Many thanks as always for your helpful comments. The question seems to
> be that we want to define ABNF that describes an example like this:
>

I have lost track, but I think there were prior proposals to incorporate 
the above parameters into the syntax of the tel URI. But AFAICT that was 
never done. (There were issues with it, raised by me among others.) (If 
they were valid in tel, then they would be valid in sip URIs too, since 
the sip uri can contain telephone-subscriber from the tel uri.)

IIUC, here you are trying to introduce these parameters into the user 
part of a sip uri, when used in P-Charge-Info, without extending them 
for other uses of the sip uri or tel uri. Is that right?

> In looking at the original ABNF we had in there, it does seem to me that
> it *was* incorrect, as it would have required a URI of something like
> one of these:
>

Yes, that is what it would have required. Whether that is correct or 
incorrect is an open question. But I guess it is contrary to your intent.

> I'm guessing the former... but it's irrelevant - the point is that
> either of these would be wrong. We need the ABNF to specify that THIS is
> the correct notation:
>
> So we really do want to specify somehow that "chargeparam" is part of
> the "userinfo" section of the SIP URI, and perhaps specifically the
> "user" section given these definitions:
>
>> RFC 3261:
>>

Technically the parameters you want are already *syntactically* valid, 
in two different ways:

They fit the definition of 'parameter' in tel (RFC 3966),

they are consistent within the definition of 'user' in 3261.

So in some sense you don't need to extend the syntax definitions at all. 
But that doesn't say anything about semantics. So at least you would 
need to define the semantics of these.

BUT...

There is now a registry of parameters for the TEL URI, defined by RFC 
5341. If you *don't* register these parameters there, then you run the 
risk that they might subsequently be registered by somebody else as tel 
uri parameter, with some different meaning.

So, I think you cheat if you don't define these as new tel uri 
parameters. But if you define them as tel uri parameters, then 
presumably they can be used anywhere a tel or sip uri can be used. 
Perhaps they could somehow be defined somehow to limit their use to 
certain contexts, but I don't know how you would go about that.

If these parameters are only to be meaningful in p-charge-info, then the 
right thing to do is define them as header field parameters, as you did 
have them once, even though they are then positioned differently than 
you were hoping for.

	Thanks,
	Paul

>> Including generic-param is a mechanism for making the syntax
>> compatible with future enhancements. But allowing it syntactically
>> doesn't specify how parameters that match generic-param are to be
>> processed if the are present on this header. Typically you would
>> specify in the draft that they should be ignored unless the behavior
>> is defined by some other specification.
>>
>> It sounds like you are talking about TEL-URI parameters when the tel
>> uri has been converted to a sip URI. But if so, then you should be
>> defining an extension to the tel-uri syntax. And then you would need
>> to define the semantics relative to the tel-uri. (It isn't really
>> kosher to define the parameters on the tel-uri but then only define
>> their semantics relative to the P-Charge-Info header.)
>>
>> IMO its wrong to make this change. Rather you should go back to
>> defining these explicitly as header params for P-Charge-Info.
>>
>> Thanks,
>> Paul
>>
>>> <mailto:[email-masked]>
>>> <mailto:[email-masked]>
>>> Use [email-masked]
>>> <mailto:[email-masked]> for questions on current sip
>>> Use [email-masked] <mailto:[email-masked]> for new developments of core SIP
>>
>> Use [email-masked]
>> <mailto:[email-masked]> for questions on current sip
>> Use [email-masked] <mailto:[email-masked]> for new developments of core SIP
>
> --

---

From: Hadriel Kaplan <[email-masked]>
Date: Sun, 4 Dec 2011 16:06:33 +0000

If the goal is to make it an interoperable header to be used outside of a private network, rather than to just document a private header used by a particular vendor, then I would suggest this discussion be moved to DISPATCH rather than this supposedly-dead SIPPING mailing list.

Further, it would probably require changing the header name (per RFC 5727), and getting consensus on the syntax and semantics.  It would hopefully NOT require a new mini-WG, but could be an individual submission handled by the AD or an independent submission to the RFC Editor.

Lastly, I should note that this draft in its current form contradicts some actual deployed usage of this header - in particular, I can't remember where Sonus encodes the noa/npi fields, but I believe Dialogic encodes them as header-params in a "P-Charge-Info" header, not userinfo-params.  IF two vendors use the same header name but in different ways, then I think it argues even more strongly to use a brand-new header name for this draft. (and don't use "Charge", because that's already used by yet another vendor)

-hadriel

On Nov 29, 2011, at 2:01 PM, Richard Shockey wrote:

> Well well isn't this fascinating.
> 
> I was just having a conversation with Dan about this today.
> 
> This draft now takes on increasing significance as it solves a nasty little
> problem of billing in one way SIP traffic (Skype -  Google Voice etal) that
> is vexing the FCC and the carriers as they try to deal with what is
> legalistically called "phantom traffic".   It's the preference I'm told is
> if no calling party number is available use a CIC or OCN code of sorts. In
> two way it could state the preference for billing which is either The CPN or
> 'Charging Number' 
>

---

From: Dan York <[email-masked]>
Date: Sun, 4 Dec 2011 23:24:28 -0500

Hadriel,

Soooo... a little background... this P-Charge-Info draft began life in February *2008* as a simple attempt to register a SIP header with IANA per RFC 3427 and to document the then *current practice* where the P-Charge-Info header was being used by my employer at the time (Voxeo, who I recently left in September 2011) and then subsequently how it was being used by people using Sonus Networks equipment. (See http://tools.ietf.org/html/draft-york-sipping-p-charge-info-01 ) 

That was it... a simple attempt to document how the P-Charge-Info header had been used for several *years* inside *existing* private networks so that it could be registered with IANA per RFC 3427.

We (Voxeo) wanted to register the usage so that when we indicated to carriers that we wanted to use P-Charge-Info to exchange billing info, we could point them to a published RFC that defined the header.  The exchange would happen on the private connection between our SIP cloud and theirs, but we wanted easy documentation we could point to. 

We also didn't want further proliferation of even more private headers that we would need to use and so hoped that documenting one would limit the proliferation of more.

Other people and companies subsequently indicated they wanted to use the header as a way to pass the ISUP Charge Number.  Folks from Telcordia, Alcatel-Lucent and other companies have been helping (along with, of course, Sonus Networks). Spencer Dawkins has also become involved because ATIS is apparently looking to reference this header in one of their standards.

Unfortunately, a couple things happened along the way. 

First, our timing was terrible. We got caught up by the RFC3427bis effort that was underway in 2008/9 and that became RFC 5727 in 2010 and changed the way SIP headers were registered. In the midst of that, I received guidance from several folks that we should wait for the completion of that work as the process of registering SIP headers would be "easier".  Second, it turned out that we did need some clarifications on the passing of the ISUP Charge Number via the NPI and NOA parameters, neither of which I personally worked with, and so it required getting assistance from various parties.  Third, I had some changes within my role at my employer that added some delays on my end.

In the meantime, a good number of folks out there started to seriously use this document and this header and have been asking rather regularly for the past year or so about when this work will be completed so they can have a published document that they can reference.  

At this point, all I want to do is get this document moved along the path toward becoming an Informational RFC.

I actually thought it was good to go until last week when these questions were raised about the ABNF. Not being an ABNF expert, I'm trying to come to closure on what I need to get into the draft so that it will be clear to implementors and will work for them.

> I can't remember where Sonus encodes the noa/npi fields, but I believe Dialogic encodes them as header-params in a "P-Charge-Info" header, not userinfo-params.

Given that my co-author, Tolga Asveren, is from Sonus and he has been the main source of info for the noa/npi parameters, I'm going to believe that Sonus encodes them as userinfo-params.  Unfortunately, given the path this draft has taken, Dialogic could have encoded them as header-params based on earlier versions of this draft (up to -09) where they were shown as being header-params. This was corrected in version -10 in October 2010, but obviously anyone doing an implementation in between would have had the guidance to use header-params.

So again, at this point, I'd really just like to get this draft moved along so that we can document the current usage (except, I guess, for Dialogic) within private networks.

Any ideas on how to get it moving faster would be greatly appreciated.

Thanks,
Dan

P.S. As to the "P-Charge-Info" name and RFC 5727, I'll again note that this originated in 2008 with trying to document current usage and pre-dates RFC 5727 (by 2 years). We address that here:

http://tools.ietf.org/html/draft-york-sipping-p-charge-info-12#section-3

---

From: Hadriel Kaplan <[email-masked]>
Date: Mon, 5 Dec 2011 06:41:03 +0000

Yes I knew most of that, and normally I wouldn't have mentioned it - there's nothing wrong with defining a private header for private use in an info RFC, as far as I'm concerned.  And I think RFC 5727 even lets you keep the "P-" portion if it's just documenting known usage or is grandfathered.

But it's the "ATIS is looking at this" part that would make one think perhaps it should NOT be a vendor-specific private-use header.  And that was the spirit in which I was responding to Richard's email, since he mentioned the FCC.  What two consenting adults wants to do is up to them - what the government mandates of all is a bigger deal. ;)

So anyway... to answer your question of how to get it moving faster, it depends on what you want it for - if you just want to document what's been done and not get any input from others, the fastest way is probably to bypass the IETF.  You can just request it be published by the RFC Editor independently, outside of the IETF.  The designated expert reviewer would hopefully pick up on errors.  But I'm not actually sure this draft would qualify if it uses userinfo-params, because as far as I know they'd have to be registered in the Tel-URI param registry which requires a standards-track doc I think. :(

On the other hand, if you want community feedback and are willing to take input/changes to the proposed header, then this discussion should probably move to DISPATCH since that's the new "sipping" list.

-hadriel