Subject: Re: [alto] Submitting draft-wu-alto-te-metrics as a WG item

From: "Randriamasy, Sabine (Nokia - FR)" <[email-masked]>
Date: Thu, 9 Feb 2017 13:53:11 +0000

Hello Mirja and Altoers,

Should the ALTO WG consider broadening the scope of the ALTO performance metrics draft by having at least a section for potential IPPM metrics and another section covering other network performance metrics?  

Indeed, network performance metrics that are already documented in the IPPM initial registry can just be referenced in the ALTO metrics document. As the IPPM initial registry is in progress, the ALTO metrics draft may include a placeholder for IPPM relevant metrics until they get adopted and specified in IPPM. Definitely, ALTO-IPPM coordination will be fruitful.   

However, quoting Al Morton's e-mail on July 21st: "Another way to interpret what ALTO needs is to look at cost metrics the same as link costs in routing. Routing link costs are usually static and the values might be set relative to some real metric like latency or max link line rate. These involve no measurement at all."

Additionally, some network metric values may stem from both measurements and provider policy as for example, many bandwidth related ALTO metrics. ALTO may convey such information not available via 3rd party measurement tools. Besides, IPPM informational RFC 5136 points the difficulty to have a unified nomenclature for network capacity related measurements.   

To guide users and applications, ALTO performance Metric values may also add abstraction to measurements and/or provide unitless performance scores. Last, given the growing WG interest in wireless networks, nothing precludes ALTO metrics from covering them.    

Thanks,
Sabine

>>-----Original Message-----
>>From: alto [mailto:[email-masked]] On Behalf Of Mirja Kühlewind
>>Sent: jeudi 5 janvier 2017 15:02
>>To: Gurbani, Vijay (Nokia - US) <[email-masked]>
>>Cc: [email-masked]; Qin Wu <[email-masked]>; IETF
>>ALTO <[email-masked]>
>>Subject: Re: [alto] Submitting draft-wu-alto-te-metrics as a WG item
>>
>>Hi all,
>>
>>sorry for being super late.
>>
>>One thing is to align with IPPM, there other question is do we need an own
>>alto registry for these metric or can we use the IPPM registry? I would prefer
>>the second approach if appropriate which would also mean that the other
>>metrics that are not part of the IPPM registry yet, should be registered there.
>>Given this is still work in progress, we definitely should coordinate with the
>>IPPM working group and see if the registration itself should happen in IPPM
>>or ALTO.
>>
>>Mirja
>>
>>
>>> Am 29.08.2016 um 21:36 schrieb Vijay K. Gurbani <vijay.gurbani@nokia-
>>bell-labs.com>:
>>>
>>> On 08/29/2016 03:47 AM, Qin Wu wrote:
>>>> Here is the update of draft-wu-alto-te-metrics to address two open
>>>> issues raised in the last meeting. The diff is:
>>>> https://www.ietf.org/rfcdiff?url2=draft-wu-alto-te-metrics-09
>>>
>>> Qin: Thanks.  More below.
>>>
>>> Folks: Please take a look at the above diffs that address the concerns
>>> in [1].
>>>
>>> Authors: Please wait a few days for any feedback from the WG and
>>> incorporate feedback in a -00 version.  Irrespective of any feedback,
>>> please generate a WG -00 and submit by next week (mid- to late next
>>> week).
>>>
>>> [1] https://www.ietf.org/mail-archive/web/alto/current/msg03277.html
>>>
>>> Thanks,
>>>
>>> - vijay
>>> --
>>> Vijay K. Gurbani, Bell Laboratories, Nokia Networks
>>> 1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60563 (USA)
>>> Email: [email-masked] / [email-masked]
>>> Web: http://ect.bell-labs.com/who/vkg/  | Calendar:
>>> http://goo.gl/x3Ogq
>>>
>>> _______________________________________________
>>> alto mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/alto
>>
>>_______________________________________________
>>alto mailing list
>>[email-masked]
>>https://www.ietf.org/mailman/listinfo/alto

---

From: Qin Wu <[email-masked]>
Date: Fri, 10 Feb 2017 03:01:36 +0000

Align with IPPM, I agree with the second approach, since IPPM is not limited to IP based metric but also cover protocol specific metric and application specific metric, I think use the IPPM registry (https://tools.ietf.org/html/draft-ietf-ippm-metric-registry-10#section-10) is pretty much reasonable to me, we can fill additional ALTO related Performance Metric Registry Entries in this registry.
what is not clear to me is the relation between draft-ietf-alto-performance-metrics and draft-ietf-ippm-initial-registry-02. It seems draft-ietf-ippm-initial-registry-02 intends to cover all the performance metrics which are defined in the IETF and add registry entry for each of these performance metrics. Also I am not sure whether draft-ietf-ippm-initial-registry-02 only focus on 
(1).direct metric can be measured in the network via 3rd party measurement tools 
(2).or some composite metric or derived metric or aggregated performance metrics?
If the answer is draft-ietf-ippm-initial-registry-02 only focus on (1), I think we can cover (2).

-Qin
-----邮件原件-----
发件人: Randriamasy, Sabine (Nokia - FR) [mailto:[email-masked]] 
发送时间: 2017年2月9日 21:53
收件人: Mirja Kühlewind; Gurbani, Vijay (Nokia - US)
抄送: [email-masked]; Qin Wu; IETF ALTO
主题: RE: [alto] Submitting draft-wu-alto-te-metrics as a WG item

Hello Mirja and Altoers,

Thanks,
Sabine

>>Cc: [email-masked]; Qin Wu <[email-masked]>; 
>>IETF ALTO <[email-masked]>
>>One thing is to align with IPPM, there other question is do we need an 
>>own alto registry for these metric or can we use the IPPM registry? I 
>>would prefer the second approach if appropriate which would also mean 
>>that the other metrics that are not part of the IPPM registry yet, should be registered there.
>>Given this is still work in progress, we definitely should coordinate 
>>with the IPPM working group and see if the registration itself should 
>>happen in IPPM or ALTO.
>>
>>Mirja
>>
>>
>>> Folks: Please take a look at the above diffs that address the 
>>> concerns in [1].
>>>
>>> incorporate feedback in a -00 version.  Irrespective of any 
>>> feedback, please generate a WG -00 and submit by next week (mid- to 
>>> late next week).
>>>

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 10 Feb 2017 16:44:57 +0100

Hi Sabine, hi Qin,

adding the ippm chairs and Spencer as the responsible AD.

My short answer: I bought this up for discussion and I don’t have a good final answer yet. I would recommend to discuss this with the ippm group (maybe even ask for further feedback on the ippm mailing list).

Longer answer: The reason why I brought this up is that you not only define cost metrics but you also talk about how to measure these values. This seems duplicating work with ippm and makes it harder to keep different efforts in the IETF aligned. Yes, you are right not all of your metrics map to ippm work but in this case I would probably also point to OSPF-TE and ISIS-TE (as you do in the draft yourself). There are already registries for these value. So I’m still questioning if an own alto registry is needed…?

Mirja

> Am 10.02.2017 um 04:01 schrieb Qin Wu <[email-masked]>:
>

---

From: Qin Wu <[email-masked]>
Date: Wed, 22 Feb 2017 02:52:08 +0000

Hi, Mirja and all:
Sorry for late reply, finally get time to follow up.
I think the section 10 of draft-ietf-alto-performance-metrics-00 doesn't intend to create new registry for each new performance metric instead,
It intends to use the "ALTO cost metric" registry defined in ALTO base protocol (i.e.,[RFC7285] section 14.2) and add a few new entries into this registry.

This registry is not for new performance metric but for new ALTO cost metric type.
In addition, the registry created in ISIS-TE(i.e.,section 12 of RFC7810) is for sub-TLV type not for new performance metric type.
So these registries have different purpose, not sure they will conflict with one another.
My take is
To align with IPPM and ISIS-TE, OSPF-TE, what we need to do is
If the performance metric is measured using IPPM method, we will use the same measurement method defined for IPPM for this metric.
If the performance metric is measured using ISIS-TE method, we will use the same measurement method defined in ISIS-TE for this metric.
If the performance metric is end to end metric, we could use method defined in draft-ietf-pce-pcep-service-aware for this metric.

In addition, I propose to update IANA section to clear confusion as follows:
Section 10 of draft-wu-alto-te-metrics
OLD TEXT:
"
   IANA has added the following entries to the ALTO cost map Properties
   registry, defined in Section 3 of [RFCXXX].

"
NEW TEXT:
"
IANA has created and now maintains the "ALTO Cost Metric Registry", listed in Table 3 of [RFC7285]. This document requests to add the following entries to “ALTO Cost Meric Registry”.
"

-Qin
-----邮件原件-----
发件人: Mirja Kühlewind [mailto:[email-masked]] 
发送时间: 2017年2月10日 23:45
收件人: Qin Wu; Randriamasy, Sabine (Nokia - FR)
抄送: Gurbani, Vijay (Nokia - US); [email-masked]; IETF ALTO; [email-masked]; Spencer Dawkins at IETF
主题: Re: [alto] Submitting draft-wu-alto-te-metrics as a WG item

Hi Sabine, hi Qin,

Mirja

> what is not clear to me is the relation between 
> draft-ietf-alto-performance-metrics and 
> draft-ietf-ippm-initial-registry-02. It seems draft-ietf-ippm-initial-registry-02 intends to cover all the performance metrics which are defined in the IETF and add registry entry for each of these performance metrics. Also I am not sure whether draft-ietf-ippm-initial-registry-02 only focus on (1).direct metric can be measured in the network via 3rd party measurement tools (2).or some composite metric or derived metric or aggregated performance metrics?
> 发件人: Randriamasy, Sabine (Nokia - FR) 
> [mailto:[email-masked]]
> 发送时间: 2017年2月9日 21:53
>>> One thing is to align with IPPM, there other question is do we need 
>>> an own alto registry for these metric or can we use the IPPM 
>>> registry? I would prefer the second approach if appropriate which 
>>> would also mean that the other metrics that are not part of the IPPM registry yet, should be registered there.
>>> Given this is still work in progress, we definitely should 
>>> coordinate with the IPPM working group and see if the registration 
>>> itself should happen in IPPM or ALTO.
>>> 
>>> Mirja
>>> 
>>> 
>>>> [1] 
>>>> https://www.ietf.org/mail-archive/web/alto/current/msg03277.html
>>>> 
>>>> Thanks,
>>>> 
>>>> - vijay
>>>> --

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 24 Feb 2017 14:11:07 +0100

Hi Qin,

see below.

> Am 22.02.2017 um 03:52 schrieb Qin Wu <[email-masked]>:
> 
> Hi, Mirja and all:

Okay. Actually didn’t see this.

> If the performance metric is end to end metric, we could use method defined in draft-ietf-pce-pcep-service-aware for this metric.

Okay. I would recommend to make sure that you don’t duplicate text but refer to existing RFC instead and potentially even refer to those RFC in the registry (if it’s a one-to-one mapping).

> „
The original text is fine. See guidance given in draft-leiba-cotton-iana-5226bis (sec 3.1): you should make sure to clearly identify the existing registry; only name it correctly ("ALTO Cost Metric Registry“) and provide an URL.

Mirja

---

From: Qin Wu <[email-masked]>
Date: Tue, 28 Feb 2017 03:28:25 +0000

Hi, Mirja:
-----邮件原件-----
发件人: Mirja Kühlewind [mailto:[email-masked]] 
发送时间: 2017年2月24日 21:11
收件人: Qin Wu
抄送: Randriamasy, Sabine (Nokia - FR); Gurbani, Vijay (Nokia - US); [email-masked]; IETF ALTO; [email-masked]; Spencer Dawkins at IETF
主题: Re: [alto] Submitting draft-wu-alto-te-metrics as a WG item

Hi Qin,

see below.

> I think the section 10 of draft-ietf-alto-performance-metrics-00 
> doesn't intend to create new registry for each new performance metric instead, It intends to use the "ALTO cost metric" registry defined in ALTO base protocol (i.e.,[RFC7285] section 14.2) and add a few new entries into this registry.

> To align with IPPM and ISIS-TE, OSPF-TE, what we need to do is If the 
> performance metric is measured using IPPM method, we will use the same measurement method defined for IPPM for this metric.

[Qin]: Sounds good idea. Thanks for your proposal.
> Section 10 of draft-wu-alto-te-metrics OLD TEXT:
> "

[Qin]: Good suggestion, thanks.

Mirja

> 抄送: Gurbani, Vijay (Nokia - US); [email-masked]; 
> IETF ALTO; [email-masked]; Spencer Dawkins at IETF
>>>>>> Here is the update of draft-wu-alto-te-metrics to address two 
>>>>>> open issues raised in the last meeting. The diff is:
>>>>> feedback, please generate a WG -00 and submit by next week (mid- 
>>>>> to late next week).
>>>>> 
>>>>> [1]