Subject: Intdir last call review of draft-ietf-ippm-alt-mark-10

From: Brian Haberman <[email-masked]>
Date: Mon, 18 Sep 2017 07:40:27 -0700

Reviewer: Brian Haberman
Review result: Ready with Nits

* The shepherd writeup mentions IPR 2557 in relation to this draft. However,
the IPR declaration is only associated with the original individual draft. The
IPR declaration needs to be updated to refer to the WG draft.

* The introduction says that this technique is needed to help monitor time &
loss sensitive traffic. That would seem to indicate a need to associate these
counters with individual flows. Has anyone done any calculations on how these
counters will scale? Section 5 seems to indicate that there were only a few
number of counters implemented via ACLs.

* The draft sets out to develop this technique without augmenting packet
formats. That makes the statement in section 3.1.1 about how to color packets
is out of scope a bit disingenuous. I would have preferred a brief description
of the few available options on coloring packets in this section.

* The mention of maintaining timestamps for delay measurements is
under-specified. Would the 32-bit NTP timestamp format be sufficient? The
64-bit NTP timestamp format? Guidance on how to carry out experimentation with
this approach seems useful. Does the concept of adding timestamps per color
block conflict with the first bullet of section 7 (only uses features already
available on routers)?

* If the above statement on per-flow counters is not true, can I not accomplish
the same capability by harvesting ifMib stats per interface and compare
xmit/recv/drop stats across the network?

* The description of the Telecom Italia experiment references a 5-minute window
per color. For time sensitive traffic like IPTV, how useful was the 5-minute
reporting window? Were corrective actions taken prior to calls being handled by
support?

* The Telecom Italia experiment writeup does not mention how the counters were
harvested from the routers. Were readily available tools used to do this?

---

From: Fioccola Giuseppe <[email-masked]>
Date: Tue, 19 Sep 2017 12:22:46 +0000

Hi Brian,
Thanks for your valuable review of the document.
Please see my answers inline tagged as [GF].

Best Regards,

Giuseppe

-----Messaggio originale-----
Da: Brian Haberman [mailto:[email-masked]] 
Inviato: lunedì 18 settembre 2017 16:40
A: [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Oggetto: Intdir last call review of draft-ietf-ippm-alt-mark-10

Reviewer: Brian Haberman

* The shepherd writeup mentions IPR 2557 in relation to this draft. However, the IPR declaration is only associated with the original individual draft. The IPR declaration needs to be updated to refer to the WG draft.

[GF]: If needed we can renew the IPR declaration to refer to the WG draft.

* The introduction says that this technique is needed to help monitor time & loss sensitive traffic. That would seem to indicate a need to associate these counters with individual flows. Has anyone done any calculations on how these counters will scale? Section 5 seems to indicate that there were only a few number of counters implemented via ACLs.

[GF]: It is important to say that you can select the flow to monitor in several ways. In general a flow can be defined by a set of selection rules (e.g. headers fields) used to match a subset of the packets. In this way it is possible to control the number of involved nodes, the paths followed by the packets and the size of the flows. That is a general consideration. 
In practice, as an example, the Telecom Italia experiment has been applied to the BackHauling traffic implemented with a VPN MPLS and we consider a flow as all the packets sharing the same source IP address or the same destination IP address, depending on the direction. It has been implemented by using router capabilities, so we developed scripts for the automation, policies for marking and ACLs for counting. Our implementation scale up to 500 flows monitored together on Cisco 7609 router, 1000 IPv4 and 1000 IPv6 flows on Cisco ASR9000. We have also an experimental implementation on dedicated HW that scales more. If you agree I can add more information in Section 5.

* The draft sets out to develop this technique without augmenting packet formats. That makes the statement in section 3.1.1 about how to color packets is out of scope a bit disingenuous. I would have preferred a brief description of the few available options on coloring packets in this section.

[GF]: This sentence may be useful to open the door to future implementation. Section 5 explains the examples of implementation we have for now. But, of course, we can add a reference to section 5 in section 3.1.1.

* The mention of maintaining timestamps for delay measurements is under-specified. Would the 32-bit NTP timestamp format be sufficient? The 64-bit NTP timestamp format? Guidance on how to carry out experimentation with this approach seems useful. Does the concept of adding timestamps per color block conflict with the first bullet of section 7 (only uses features already available on routers)?

[GF]: the first bullet of section 7 means that a basic "homemade" implementation of the method can be possible by using features already available on routers: policy for marking, ACL for counting and Netflow to obtain timestamps of first/last packets of a batch. But in general we hope to have an optimized implementation that, as you point out, makes this sentence not true. So I propose to modify the sentence in the first bullet of section 7. 
Regarding timestamps we can say that all the timestamp formats are allowed, because it's managed out-of-band. The format (NTP or PTP) depends on the precision you want.

* If the above statement on per-flow counters is not true, can I not accomplish the same capability by harvesting ifMib stats per interface and compare xmit/recv/drop stats across the network?

[GF]: Yes you can do that but it is not so precise because you cannot know that nodes you are comparing are referring to exactly the same time interval and exactly the same packets. The great advantage of alternate marking method is that it solves these synchronization issues creating packet batches.

* The description of the Telecom Italia experiment references a 5-minute window per color. For time sensitive traffic like IPTV, how useful was the 5-minute reporting window? Were corrective actions taken prior to calls being handled by support?

[GF]: The choice of 5 minutes period has been made to make it coherent with the reporting window of our NMS. We use a threshold for the packet loss rate in 5 minutes that originates alarms for the Operation department. Another reason is that long windows simplify our "homemade" implementation. For example the Huawei implementation (section 5.2) uses very short windows (seconds).

* The Telecom Italia experiment writeup does not mention how the counters were harvested from the routers. Were readily available tools used to do this?

[GF]: Our scripts on board of the router collect the counters of the flows under monitoring and then send them out to the NMS that will make packet loss calculation. The tools have been developed in our NMS development team. Also Huawei developed this features in its network management tool.

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie. 

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks. 

Rispetta l'ambiente. Non stampare questa mail se non è necessario.

---

From: Brian Haberman <[email-masked]>
Date: Wed, 20 Sep 2017 10:28:56 -0400

Hi Giuseppe,
     I am fine with your proposed changes, but I would recommend that
you wait until the AD or doc shepherd indicate that the changes be made.

Regards,
Brian

---

From: Fioccola Giuseppe <[email-masked]>
Date: Wed, 20 Sep 2017 14:50:33 +0000

Hi Brian,
Ok, Thank you!

Giuseppe

-----Messaggio originale-----
Da: Brian Haberman [mailto:[email-masked]] 
Inviato: mercoledì 20 settembre 2017 16:29
A: Fioccola Giuseppe; [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked]
Oggetto: Re: R: Intdir last call review of draft-ietf-ippm-alt-mark-10

Hi Giuseppe,
     I am fine with your proposed changes, but I would recommend that you wait until the AD or doc shepherd indicate that the changes be made.

Regards,
Brian

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Wed, 20 Sep 2017 10:16:04 -0500

Just on the mechanics -

On Wed, Sep 20, 2017 at 9:50 AM, Fioccola Giuseppe <
[email-masked]> wrote:

>      I am fine with your proposed changes, but I would recommend that you
> wait until the AD or doc shepherd indicate that the changes be made.
>

Giuseppe has responded to his reviewer with proposed text changes, and
(even better) has feedback from his reviewer.

That's what matters most. Submitting an update at that point is often
harmless, but risks getting feedback from other reviewers, and from the
IESG, that touches the same text, so then the editor is left to mix
everything together and have a coherent update come out. Even worse, it can
happen that you get different ADs balloting on different versions of the
draft and providing comments that other ADs have to analyze when balloting.
So, waiting for the telechat, and submitting one update that addresses the
comments received, is fine with me.

Other ADs might have other preferences for a specific draft, but since this
one's mine, let's go with my opinion for now :D ...

Spencer

> > * The shepherd writeup mentions IPR 2557 in relation to this draft.
> However, the IPR declaration is only associated with the original
> individual draft. The IPR declaration needs to be updated to refer to the
> WG draft.
> >
> > * The introduction says that this technique is needed to help monitor
> time & loss sensitive traffic. That would seem to indicate a need to
> associate these counters with individual flows. Has anyone done any
> calculations on how these counters will scale? Section 5 seems to indicate
> that there were only a few number of counters implemented via ACLs.
> >
> > [GF]: It is important to say that you can select the flow to monitor in
> several ways. In general a flow can be defined by a set of selection rules
> (e.g. headers fields) used to match a subset of the packets. In this way it
> is possible to control the number of involved nodes, the paths followed by
> the packets and the size of the flows. That is a general consideration.
> > In practice, as an example, the Telecom Italia experiment has been
> applied to the BackHauling traffic implemented with a VPN MPLS and we
> consider a flow as all the packets sharing the same source IP address or
> the same destination IP address, depending on the direction. It has been
> implemented by using router capabilities, so we developed scripts for the
> automation, policies for marking and ACLs for counting. Our implementation
> scale up to 500 flows monitored together on Cisco 7609 router, 1000 IPv4
> and 1000 IPv6 flows on Cisco ASR9000. We have also an experimental
> implementation on dedicated HW that scales more. If you agree I can add
> more information in Section 5.
> >
> formats. That makes the statement in section 3.1.1 about how to color
> packets is out of scope a bit disingenuous. I would have preferred a brief
> description of the few available options on coloring packets in this
> section.
> >
> > [GF]: This sentence may be useful to open the door to future
> implementation. Section 5 explains the examples of implementation we have
> for now. But, of course, we can add a reference to section 5 in section
> 3.1.1.
> >
> 64-bit NTP timestamp format? Guidance on how to carry out experimentation
> with this approach seems useful. Does the concept of adding timestamps per
> color block conflict with the first bullet of section 7 (only uses features
> already available on routers)?
> >
> > [GF]: the first bullet of section 7 means that a basic "homemade"
> implementation of the method can be possible by using features already
> available on routers: policy for marking, ACL for counting and Netflow to
> obtain timestamps of first/last packets of a batch. But in general we hope
> to have an optimized implementation that, as you point out, makes this
> sentence not true. So I propose to modify the sentence in the first bullet
> of section 7.
> > Regarding timestamps we can say that all the timestamp formats are
> allowed, because it's managed out-of-band. The format (NTP or PTP) depends
> on the precision you want.
> >
> > * If the above statement on per-flow counters is not true, can I not
> accomplish the same capability by harvesting ifMib stats per interface and
> compare xmit/recv/drop stats across the network?
> >
> > [GF]: Yes you can do that but it is not so precise because you cannot
> know that nodes you are comparing are referring to exactly the same time
> interval and exactly the same packets. The great advantage of alternate
> marking method is that it solves these synchronization issues creating
> packet batches.
> >
> > * The description of the Telecom Italia experiment references a 5-minute
> window per color. For time sensitive traffic like IPTV, how useful was the
> 5-minute reporting window? Were corrective actions taken prior to calls
> being handled by support?
> >
> > [GF]: The choice of 5 minutes period has been made to make it coherent
> with the reporting window of our NMS. We use a threshold for the packet
> loss rate in 5 minutes that originates alarms for the Operation department.
> Another reason is that long windows simplify our "homemade" implementation.
> For example the Huawei implementation (section 5.2) uses very short windows
> (seconds).
> >
> > * The Telecom Italia experiment writeup does not mention how the
> counters were harvested from the routers. Were readily available tools used
> to do this?
> >
> > [GF]: Our scripts on board of the router collect the counters of the
> flows under monitoring and then send them out to the NMS that will make
> packet loss calculation. The tools have been developed in our NMS
> development team. Also Huawei developed this features in its network
> management tool.
> >
> >
> > Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle
> persone indicate. La diffusione, copia o qualsiasi altra azione derivante
> dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora
> abbiate ricevuto questo documento per errore siete cortesemente pregati di
> darne immediata comunicazione al mittente e di provvedere alla sua
> distruzione, Grazie.
> >
> > This e-mail and any attachments is confidential and may contain
> privileged information intended for the addressee(s) only. Dissemination,
> copying, printing or use by anybody else is unauthorised. If you are not
> the intended recipient, please delete this message and any attachments and
> advise the sender by return e-mail, Thanks.
> >

---

From: Fioccola Giuseppe <[email-masked]>
Date: Wed, 20 Sep 2017 15:29:27 +0000

Ok Spencer, it makes sense. I’m taking notes of the changes to do and then I will address them together after the telechat.
Best Regards,

Giuseppe

Da: Spencer Dawkins at IETF [mailto:[email-masked]]
Inviato: mercoledì 20 settembre 2017 17:16
A: Fioccola Giuseppe
Cc: Brian Haberman; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Oggetto: Re: R: Intdir last call review of draft-ietf-ippm-alt-mark-10

Just on the mechanics -

On Wed, Sep 20, 2017 at 9:50 AM, Fioccola Giuseppe <[email-masked]<mailto:[email-masked]>> wrote:
Hi Brian,
Ok, Thank you!

Giuseppe

-----Messaggio originale-----
Da: Brian Haberman [mailto:[email-masked]<mailto:[email-masked]>]
Inviato: mercoledì 20 settembre 2017 16:29
A: Fioccola Giuseppe; [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Oggetto: Re: R: Intdir last call review of draft-ietf-ippm-alt-mark-10

Hi Giuseppe,

Giuseppe has responded to his reviewer with proposed text changes, and (even better) has feedback from his reviewer.

That's what matters most. Submitting an update at that point is often harmless, but risks getting feedback from other reviewers, and from the IESG, that touches the same text, so then the editor is left to mix everything together and have a coherent update come out. Even worse, it can happen that you get different ADs balloting on different versions of the draft and providing comments that other ADs have to analyze when balloting. So, waiting for the telechat, and submitting one update that addresses the comments received, is fine with me.

Other ADs might have other preferences for a specific draft, but since this one's mine, let's go with my opinion for now :D ...

Spencer

Regards,
Brian

On 9/19/17 8:22 AM, Fioccola Giuseppe wrote:
> Da: Brian Haberman [mailto:[email-masked]<mailto:[email-masked]>]
> A: [email-masked]<mailto:[email-masked]>
> Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>;
> [email-masked]<mailto:[email-masked]>

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 20 Sep 2017 19:11:08 +0000

Please find two comments inline.

On Sep 20, 2017, at 11:16 AM, Spencer Dawkins at IETF <[email-masked]<mailto:[email-masked]>> wrote:

Just on the mechanics -

Giuseppe

-----Messaggio originale-----
Da: Brian Haberman [mailto:[email-masked]<mailto:[email-masked]>]
Inviato: mercoledì 20 settembre 2017 16:29
A: Fioccola Giuseppe; [email-masked]<mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Oggetto: Re: R: Intdir last call review of draft-ietf-ippm-alt-mark-10

Hi Giuseppe,

Indeed the important part is the dialogue and convergence.

As shepherd, for contrast, I tend to prefer different mechanics: revision numbers are really cheap, and submitting new revisions in response to I-D reviews  with a release-early-release-often approach (instead of bundling main features) has important benefits. The rational is that an editor likely has changes incorporated in her or his working copy, and the next reviewer (Directorate, AD, etc) will likely comment on the same issue. It is also easier to identify where fixes are applied as it simplifies tracking. The exception is during actual ballot where there’s a large number of reviews in a very short timeframe.

Other ADs might have other preferences for a specific draft, but since this one's mine, let's go with my opinion for now :D …

I was just sharing for contrast, clearly let’s go with Spencer’s preference and guidance on this.

Spencer

[…]

> [GF]: If needed we can renew the IPR declaration to refer to the WG draft.

It might not hurt, but at the same time, a question:

RFC 8179 S 5.4.2 talks about inheritance of IPR disclosures:
https://tools.ietf.org/html/rfc8179#section-5.4.2

      IPR
      disclosures against a particular Contribution are assumed to be
      inherited by revisions of the Contribution and by any RFCs that
      are published from the Contribution unless the disclosure has been
      updated or withdrawn.

It is not clear if a “revision” of an I-D as a contribution includes renaming it as WG document, merges, etc.

The data tracker tooling seems to be coded such that, if there is a Replaced_by relationship, the IPR declaration follows:
https://datatracker.ietf.org/ipr/search/?submit=draft&id=draft-ietf-ippm-alt-mark
“Total number of IPR disclosures found: 1."

And contributors all acknowledged IPR 2557.

> * The shepherd writeup mentions IPR 2557 in relation to this draft. However, the IPR declaration is only associated with the original individual draft. The IPR declaration needs to be updated to re…

Does it carry forward?

Thanks,

— Carlos.

---

From: Brian Haberman <[email-masked]>
Date: Wed, 20 Sep 2017 15:21:36 -0400

Hi Carlos,

>> * The shepherd writeup mentions IPR 2557 in relation to this draft. However, the IPR declaration is only associated with the original individual draft. The IPR declaration needs to be updated to r…

I think the answer is "sort of"...

If I look at the datatracker page for the WG, I do see 2557 listed
against the WG draft here:

https://datatracker.ietf.org/wg/ippm/documents/

However, I do not see that declaration when looking at the HTML'ized
version of the draft here:

https://tools.ietf.org/html/draft-ietf-ippm-alt-mark-10

while I do see it listed against the original individual submission here:

https://tools.ietf.org/html/draft-tempia-ippm-p3m-03

The IPR search page does show the indirect disclosure based on the
individual submission.

So, it looks like an issue in the HTML rendering of the WG draft.

Regards,
Brian

---

From: "Carlos Pignataro (cpignata)" <[email-masked]>
Date: Wed, 20 Sep 2017 19:27:16 +0000

Hi, Brian,

On Sep 20, 2017, at 3:21 PM, Brian Haberman <[email-masked]<mailto:[email-masked]>> wrote:

Hi Carlos,

     IPR

So the question:

Does it carry forward?

Although it is listed against the document at
https://datatracker.ietf.org/doc/draft-ietf-ippm-alt-mark/
(See right underneath IANA a highlighted “1”)

And I’d assume the datatracker as a more authoritative rendering.

Thank you for checking. In any case, I appreciate your initial point and I believe being explicit and update the declaration is the way to go.

Thanks,

— Carlos.

Regards,
Brian

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Wed, 20 Sep 2017 16:39:01 -0500

Hi, Carlos and Brian,

On Wed, Sep 20, 2017 at 2:27 PM, Carlos Pignataro (cpignata) <
[email-masked]> wrote:

> Hi, Brian,
>
> On Sep 20, 2017, at 3:21 PM, Brian Haberman <[email-masked]>
> wrote:
>
> Hi Carlos,
>
> The data tracker tooling seems to be coded such that, if there is a
> Replaced_by relationship, the IPR declaration follows:
> Thank you for checking.

Yes, thank you for that.

> In any case, I appreciate your initial point and I believe being explicit
> and update the declaration is the way to go.
>

Clarity is never wrong, but it's also worth Spencer asking the IESG if they
know about this (in)consistency,

I'll let you know what I find out.

Spencer, who has to do SOMETHING around here to be helpful :-)

---

From: Fioccola Giuseppe <[email-masked]>
Date: Thu, 21 Sep 2017 09:59:14 +0000

No problem, we can renew the IPR declaration.
If the inconsistency can be solved in this way, I will proceed.

Giuseppe

Da: Spencer Dawkins at IETF [mailto:[email-masked]]
Inviato: mercoledì 20 settembre 2017 23:39
A: Carlos Pignataro (cpignata)
Cc: Brian Haberman; Fioccola Giuseppe; [email-masked]; [email-masked]; [email-masked]; [email-masked]
Oggetto: Re: Intdir last call review of draft-ietf-ippm-alt-mark-10

Hi, Carlos and Brian,

On Wed, Sep 20, 2017 at 2:27 PM, Carlos Pignataro (cpignata) <[email-masked]<mailto:[email-masked]>> wrote:
Hi, Brian,

Hi Carlos,

     IPR

So the question:

Does it carry forward?

Thank you for checking.

Yes, thank you for that.

In any case, I appreciate your initial point and I believe being explicit and update the declaration is the way to go.

Clarity is never wrong, but it's also worth Spencer asking the IESG if they know about this (in)consistency,

Thanks,

— Carlos.

Regards,
Brian