Subject: Re: [ippm] 1st review of RFC 8321bis

From: Giuseppe Fioccola <[email-masked]>
Date: Wed, 1 Dec 2021 18:58:25 +0000

Hi Martin,
Thanks a lot for your thorough review.
I also agree that for a later version a reorganization of the content can improve the readability.

Regarding your comments, please find my answers inline tagged as [GF].

Best Regards,

Giuseppe

From: Martin Duke <[email-masked]>
Sent: Tuesday, November 30, 2021 11:13 PM
To: [email-masked]; IETF IPPM WG <[email-masked]>
Subject: 1st review of RFC 8321bis

Hello authors,

I've read through this document carefully for the first time. Having done so, I fully support later doing the reorganization that you've proposed. The current draft presents the information in a strange order.

I have no serious issues with the changes from 8321, but I have numerous comments on the 8321 text that ought to be fixed before we take it to Standard.

MAJOR COMMENTS:

- Why is RFC3393 normative if the other packet definition documents (7679, 7680) are informative? IMO they are all informative, but it should at least be consistent.

[GF]: It was my oversight, I will fix it in the next version. In RFC8321, all these RFCs were normative but now can be informative.

- Sec 1. Can you clarify the relationship of this draft with [IEEE-Network-PNPM]? Do they repeat each other, or does that one make a different contribution to this project? If the former, who has change control over the design?

[GF]: It is not a repetition. The paper [IEEE-Network-PNPM] includes RFC8321 methods but it also explains new mechanisms for the packet marking based on hashing techniques, which are not included in RFC8321. These new variations are described in a separate document (draft-mizrahi-ippm-marking). I can also omit the reference if you think it can be confusing.

- I'm concerned about sizing the block based on packet count. It's clearly your less-preferred option, but including it has some implications. For example, passive eavesdropping is much easier, as it would be straightforward to figure out the expected number of packets and therefore figure out the loss rate. If this wasn't a successful part of the experiment, perhaps we should just remove it.

[GF]: Yes, we could remove it in the PS. But, it is worth noting that draft-ietf-ippm-explicit-flow-measurements suggests a marking method (sQuare bit) based on a fixed number of packets to measure packet loss. It is simpler for Client-Server protocols like QUIC. Another option could be to leave the description of the counter-based method but we should improve the security considerations for this aspect. What do you suggest?

- sec 3.2.The guard band calculation does not seem correct to me. If D_max -> D_min, then the guard band is simply A. But the ingress and egress nodes still need to account for the packet delay from end to end! I would suggest

d = A + D_max.

This would also cover the reordering constraint, as the delay can be no more than Dmax.

[GF]: If D_max becomes equal to D_min, this means that all the packets are delayed of the same value. So the batch of packets is simply shifted and no packet is reordered. But, you are right we still need to take into account the delay. Maybe we could consider the average delay value and its standard deviation in the formula:

d = A + D_avg + D_stddev

- sec 3.3.1 "a measurement is valid only if no packet loss occurs and if packet misordering can be avoided". So what it's not? Do we throw out the measurement? How can we detect packet misordering unless it delays a packet over the edge?

[GF]: This approach is affected by the packet reordering that we cannot detect. You can apply it but you are aware that the first packet of the batch in one point may not be the same packet in a second point due to the reordering. In this case you are not sure to compare the same packet and the delay measure is done with this intrinsic error. I can explain this point in the next version.

- Sec 3.3.2 "The conventional [delay] range (maximum-minimum) should be avoided". I don't know how to reconcile this with the guard band calculation, which absolutely uses the range! Is this a valid thing to do or not?

[GF]: In this case I did not mean the delay of the network to calculate the guard band, but I meant the delay you are measuring. I think I can also omit this sentence, it is not important.

- Section 4. This entire section is largely a subset of section 3, and covers the material both (a) inconsistently and (b) worse. There's a reference to "two-way delay" which occurs nowhere else in the document.  The L considerations don't consider delay, unlike Section 3.2. 4.3 says that "a proper interval is out off the scope of this document!" But the document has lots to say about this! Earlier discussions led me to believe you were going to largely get rid of Section 4; perhaps as an intermediate step, in the next draft could we delete the text in that section that no longer applies?

[GF]: Yes, indeed I modified a lot this section in my first local revision of the document. As intermediate step, I will surely delete text that no longer applies in the next version.

- Section 8. The most potent attack on this would appear to be on the data collection protocol, which is out of scope. It might be good to point the reader to some references that discuss security considerations for gathering data.

[GF]: Agree, will do. I also think we have improved a lot the security considerations in draft-ietf-6man-ipv6-alt-mark and we can inherit something from there as well.

NITS: Is there a github? There are numerous editorial cleanup issues that would be much easier to do if I could just submit a PR.

[GF]: It is a possibility. I could also share the xml file if it works for you.

- Abstract: delete the phrase "a method based on"

- Sec 1. s/On the other hand, performance/Performance also

- Sec 3.1 s/packet loss occurred/if packet loss occurred

- Sec 3.1.1 "some applications" are no longer "reported in section 5." Delete the sentence.

- Sec 3.1.2 s/a couple of counters are needed/two counters are needed

- Sec 3.3.1.1 s/implicates/implies

- Update references from RFC8889 to RFC8889bis.

[GF]: I will apply all these minor changes in the next version.

---

From: Martin Duke <[email-masked]>
Date: Wed, 1 Dec 2021 14:34:49 -0800

On Wed, Dec 1, 2021 at 10:58 AM Giuseppe Fioccola <
[email-masked]> wrote:

> *From:* Martin Duke <[email-masked]>
>

> - Sec 1. Can you clarify the relationship of this draft with
> [IEEE-Network-PNPM]? Do they repeat each other, or does that one make a
> different contribution to this project? If the former, who has change
> control over the design?
>
>
>
> [GF]: It is not a repetition. The paper [IEEE-Network-PNPM] includes
> RFC8321 methods but it also explains new mechanisms for the packet marking
> based on hashing techniques, which are not included in RFC8321. These new
> variations are described in a separate document
> (draft-mizrahi-ippm-marking). I can also omit the reference if you think it
> can be confusing.
>

Don't omit it, just explain what the reference is contributing.

> - I'm concerned about sizing the block based on packet count. It's clearly
> your less-preferred option, but including it has some implications. For
> example, passive eavesdropping is much easier, as it would be
> straightforward to figure out the expected number of packets and therefore
> figure out the loss rate. If this wasn't a successful part of the
> experiment, perhaps we should just remove it.
>
>
>
> [GF]: Yes, we could remove it in the PS. But, it is worth noting that
> draft-ietf-ippm-explicit-flow-measurements suggests a marking method
> (sQuare bit) based on a fixed number of packets to measure packet loss. It
> is simpler for Client-Server protocols like QUIC. Another option could be
> to leave the description of the counter-based method but we should improve
> the security considerations for this aspect. What do you suggest?
>

I'm not a practitioner, so I can't say if it's valuable to leave it in the
design or not. But the rest of the document (and 8889) is written under the
assumption that the blocks are time based. For example, if it's count-based
then L needs to be in units of packets, not time. How would that work?

So if we're going to keep packet-based windows in the draft, you either
need to review the whole document and in each place add a paragraph that
starts "if using packet-based windows...", or there could be a section that
enumerates all the ways this mode is different.

>
>
> - sec 3.2.The guard band calculation does not seem correct to me. If D_max
> -> D_min, then the guard band is simply A. But the ingress and egress nodes
> still need to account for the packet delay from end to end! I would suggest
>
>
>
> d = A + D_max.
>
>
>
> [GF]: If D_max becomes equal to D_min, this means that all the packets are
> delayed of the same value. So the batch of packets is simply shifted and no
> packet is reordered. But, you are right we still need to take into account
> the delay. Maybe we could consider the average delay value and its
> standard deviation in the formula:
>
>
>
> d = A + D_avg + D_stddev
>

I don't understand what that formula accomplishes. Dmax absolutely covers
reordering; regardless of reordering, the rightmost packet will not arrive
later than (tx_time + A + D_max).

>
>
> - sec 3.3.1 "a measurement is valid only if no packet loss occurs and if
> packet misordering can be avoided". So what it's not? Do we throw out the
> measurement? How can we detect packet misordering unless it delays a packet
> over the edge?
>
>
>
> [GF]: This approach is affected by the packet reordering that we cannot
> detect. You can apply it but you are aware that the first packet of the
> batch in one point may not be the same packet in a second point due to the
> reordering. In this case you are not sure to compare the same packet and
> the delay measure is done with this intrinsic error. I can explain this
> point in the next version.
>

My point is that this is generally undetectable and therefore is part of
the intrinsic error in the measurement. Maybe your explanation will
indicate this.

>
>
> - Sec 3.3.2 "The conventional [delay] range (maximum-minimum) should be
> avoided". I don't know how to reconcile this with the guard band
> calculation, which absolutely uses the range! Is this a valid thing to do
> or not?
>
>
>
> [GF]: In this case I did not mean the delay of the network to calculate
> the guard band, but I meant the delay you are measuring. I think I can also
> omit this sentence, it is not important.
>

OK

>
>
> - Section 4. This entire section is largely a subset of section 3, and
> covers the material both (a) inconsistently and (b) worse. There's a
> reference to "two-way delay" which occurs nowhere else in the document.
> The L considerations don't consider delay, unlike Section 3.2. 4.3 says
> that "a proper interval is out off the scope of this document!" But the
> document has lots to say about this! Earlier discussions led me to believe
> you were going to largely get rid of Section 4; perhaps as an intermediate
> step, in the next draft could we delete the text in that section that no
> longer applies?
>
>
>
> [GF]: Yes, indeed I modified a lot this section in my first local revision
> of the document. As intermediate step, I will surely delete text that no
> longer applies in the next version.
>

Yes, I think deleted text is fine before we do the reorg. We'll take the
diff between 8321 and the last draft with the old organization and send
that with the request for review.

The 8889 bis review is coming; there are some points relevant to 8321 as
well.

---

From: Giuseppe Fioccola <[email-masked]>
Date: Thu, 2 Dec 2021 10:19:29 +0000

Hi Martin,
My answers inline as [GF].

Regards,

Giuseppe

From: Martin Duke <[email-masked]>
Sent: Wednesday, December 1, 2021 11:35 PM
To: Giuseppe Fioccola <[email-masked]>
Cc: [email-masked]; IETF IPPM WG <[email-masked]>
Subject: Re: 1st review of RFC 8321bis

On Wed, Dec 1, 2021 at 10:58 AM Giuseppe Fioccola <[email-masked]<mailto:[email-masked]>> wrote:
From: Martin Duke <[email-masked]<mailto:[email-masked]>>

[GF]: Ok

I'm not a practitioner, so I can't say if it's valuable to leave it in the design or not. But the rest of the document (and 8889) is written under the assumption that the blocks are time based. For example, if it's count-based then L needs to be in units of packets, not time. How would that work?

So if we're going to keep packet-based windows in the draft, you either need to review the whole document and in each place add a paragraph that starts "if using packet-based windows...", or there could be a section that enumerates all the ways this mode is different.

[GF]: I would only consider the assumption that the blocks are time based. The count based method can be mentioned only in the general description as additional possibility but we can clarify that its detailed specification is out of scope here.

d = A + D_max.

d = A + D_avg + D_stddev

I don't understand what that formula accomplishes. Dmax absolutely covers reordering; regardless of reordering, the rightmost packet will not arrive later than (tx_time + A + D_max).

[GF]: I forgot the multiplier. For a data set, most observations fall within one standard deviation of the average. In particular, 68% of the samples are within 1 standard deviation of the average. 95% of the samples are within 2 standard deviation of the average. 99.7% of the samples are within 3 standard deviation of the mean.
It means that D_avg + 3*D_stddev -> D_max
By the way, to make it simple we can also simply consider D_max in the formula.

My point is that this is generally undetectable and therefore is part of the intrinsic error in the measurement. Maybe your explanation will indicate this.

[GF]: Yes, my intention is to explain that it is undetectable and the intrinsic error cannot be exactly quantified.

OK

Yes, I think deleted text is fine before we do the reorg. We'll take the diff between 8321 and the last draft with the old organization and send that with the request for review.

[GF]: Agree.

The 8889 bis review is coming; there are some points relevant to 8321 as well.

[GF]: Ok, good.

---

From: Martin Duke <[email-masked]>
Date: Thu, 2 Dec 2021 10:41:54 -0800

On Thu, Dec 2, 2021 at 2:19 AM Giuseppe Fioccola <
[email-masked]> wrote:

> *Sent:* Wednesday, December 1, 2021 11:35 PM
> *To:* Giuseppe Fioccola <[email-masked]>
> *Cc:* [email-masked]; IETF IPPM WG <[email-masked]>
> *Subject:* Re: 1st review of RFC 8321bis
>
>
>
>
>
>
>
> [GF]: I would only consider the assumption that the blocks are time based.
> The count based method can be mentioned only in the general description as
> additional possibility but we can clarify that its detailed specification
> is out of scope here.
>
>
>

Moving count-based blocks out of scope sounds like a good solution.

> [GF]: I forgot the multiplier. For a data set, most observations fall
> within one standard deviation of the average. In particular, 68% of the
> samples are within 1 standard deviation of the average. 95% of the samples
> are within 2 standard deviation of the average. 99.7% of the samples are
> within 3 standard deviation of the mean.
>

OK, either D_max or D_avg + 3D_stddev is fine. I agree that your
formulation is superior if you're basing it off measurements.

Martin

---

From: Giuseppe Fioccola <[email-masked]>
Date: Fri, 3 Dec 2021 14:02:51 +0000

Hi Martin,
Ok, I will update this draft soon to address all the agreed changes.

Regards,

Giuseppe

Sent: Thursday, December 2, 2021 7:42 PM

On Thu, Dec 2, 2021 at 2:19 AM Giuseppe Fioccola <[email-masked]<mailto:[email-masked]>> wrote:
Hi Martin,
My answers inline as [GF].

Regards,

Giuseppe

From: Martin Duke <[email-masked]<mailto:[email-masked]>>
Sent: Wednesday, December 1, 2021 11:35 PM
To: Giuseppe Fioccola <[email-masked]<mailto:[email-masked]>>
Cc: [email-masked]<mailto:[email-masked]>; IETF IPPM WG <[email-masked]<mailto:[email-masked]>>
Subject: Re: 1st review of RFC 8321bis

d = A + D_max.

d = A + D_avg + D_stddev

OK, either D_max or D_avg + 3D_stddev is fine. I agree that your formulation is superior if you're basing it off measurements.

Martin