Subject: Re: [ippm] AD review of draft-ietf-ippm-initial-registry-11

From: Mirja Kuehlewind <[email-masked]>
Date: Mon, 2 Sep 2019 13:29:35 +0200

Hi Al,

Thanks for the background info. Sorry, I should have explained myself better. However, I don’t have a strong opinion on which status we end up with but I would really like to make sure that the intended status is well explained in the shepherd write-up in order to provided the other ADs the needed background information during IESG review.

My think was that this document does actually not specify any new metric and as the registry only requires “Expert Review”, informational felt more natural for me. However, this is for the working group to decide!

Mirja

> On 30. Aug 2019, at 22:20, MORTON, ALFRED C (AL) <[email-masked]> wrote:
> 
> Hi Mirja,
> 
> Thanks for reviewing these two memos, 
> registry and initial contents. They added a lot
> of pages to your review load.
> 
> I don't recollect a list discussion on 
> whether PS or Info, but years ago we decided
> that all metric RFCs would go on the Standards Track,
> and devised ways to evaluate whether they should 
> advance based on testing and ultimately 
> revised text. So, it seemed natural to make these
> "tighter specifications of the original metrics"
> PS, and that's been the status since 00.
> 
> With this background, I'm interested to learn why
> Informational seems the right choice to you,
> and we should all think about the implications 
> of Info or PS for the metrics in this 
> draft-*-initial-registry.
> 
> regards,
> Al
> 
>> -----Original Message-----
>> From: Mirja Kuehlewind [mailto:[email-masked]]
>> Sent: Tuesday, August 27, 2019 11:47 AM
>> To: [email-masked]
>> Cc: IETF IPPM WG <[email-masked]>; Michelle Cotton
>> <[email-masked]>
>> Subject: AD review of draft-ietf-ippm-initial-registry-11
>> 
>> Hi authors, hi all,
>> 
>> We need to discuss at least one bigger question on the intended status
>> before we  can move ahead. However, as you can seen I have more
>> questions/comments below.
>> 
>> The bigger status question first:
>> The intended status is PS. The shepherd write-up does not provide any
>> additional information about why this status is appropriate. I would
>> assume that informational is actually more appropriate. I don’t recall out
>> of my head if/when this was discussed and I thought I rather ask than
>> searching the list archive and minutes… Is this the right status and why?
>> And if so, please reflect in the write-up!
>> 
>> 
>> And then a couple more (hopefully) quicker comments/questions:
>> 
>> - I know this was already reviewed by IANA (Michelle is also cc’ed),
>> however, as far as I can see all Administrative Information (Status,
>> Requestor, Revision, and Revision Date) and maybe even the ID don’t need
>> to be part of the template because IANA will anyway assign them based on
>> the instructions given in draft-ietf-ippm-metric-registry. Or did IANA
>> explicitly request to also include these parts?
>> 
>> - Then for the Change Controller that should usually be the IESG rather
>> than the IETF. Did IANA comment on that?
>> 
>> - And one minor point: I guess the secY part of the name could already be
>> filled out, no? However, I should probably have asked that on review of
>> draft-ietf-ippm-metric-registry already but I was actually wondering if
>> the decision to include the RFC number and section in the metric name is
>> appropriate. Can you maybe lay out what the reasons for this are (and
>> maybe also explain in the draft as his might come up again during IESG
>> review otherwise)…?
>> 
>> 
>> Then then on references again:
>> 
>> - This draft references two obsoleted RFCs (RFC2679 and RFC2680). I assume
>> this is an oversight and the references need to be updated?
>> 
>> - Also given you rely heavily on [Trammell-14], I think this needs to be a
>> normative reference (and maybe [Strowes] as well?). The more stable
>> pointer for [Trammell-14] is actually here:
>> https://urldefense.proofpoint.com/v2/url?u=https-
>> 3A__link.springer.com_chapter_10.1007_978-2D3-2D642-2D54999-2D1-
>> 5F2&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=hwJZg-
>> IdxQ9507aw2mjEKKOSWIEVnc8nwr7B_QiOyCU&s=HSEAeC_uI-vofX5ZWPSCH-R4Ph8Gu618-
>> YtZJ4pTAgE&e=
>> But to be honest the split between normative and informational isn’t
>> actually very clear to me here…?
>> 
>> 
>> And on the shepherd write-up:
>> 
>> It’s indicated that not all authors have replied to the IPR question. This
>> need to be checked and respectively reflected in the shepherd write-up
>> before we can move ahead!
>> 
>> 
>> And some smaller technical questions/nits:
>> 
>> - Is it correct that T0 in section 4 is defined in both subsections 4.3.5
>> and 4.4.2?
>> 
>> - Why is an URI given (only) in section 6.1.3?
>>  Also section 7.1.3 and 8.1.3.: http:\\www.iana.org\ ... <name> -> I
>> think this should be: https://urldefense.proofpoint.com/v2/url?u=http-
>> 3A__www.iana.org_&d=DwIFaQ&c=LFYZ-
>> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=hwJZg-
>> IdxQ9507aw2mjEKKOSWIEVnc8nwr7B_QiOyCU&s=9PuQd07oTFlEt8tWIDYz4c-
>> qFUd4zpyT_7QeckyeAsc&e=<name>/
>> 
>> 
>> Some more editorial comments/nits:
>> 
>> - I find it actually unnecessary or rather confusing that most (not all)
>> sections start with a sentence/paragraph explaining what the section is
>> about (e.g. "Additional (Informational) details for this entry”). I would
>> think this is sufficiently explained in draft-ietf-ippm-metric-registry
>> and does not need to be repeated here (in this already long document).
>> Also note that section 8.6 inconsistently doesn’t have any content while
>> 10.6. says “None.”.
>> 
>> - Also these comments in section 7 and 8 are probably supposed to be
>> removed:
>> " <insert name of the output type, raw or a selected summary statistic>”
>> …?
>> 
>> - I found this in section 10.2.2: "@@@@@ others??”. I guess there is
>> something missing? Also in this part, I don’t think SYN and FIN should be
>> set at the same time, no? And rather than providing the Kind and Length of
>> the TSopt, I would recommend to provide a pointer to the respective RFC.
>> 
>> 
>> And finally one/two/three more general question(s) at the end, that I
>> probably should also have asked already on my review for draft-ietf-ippm-
>> metric-registry:
>> 
>> Is it actually intended that basically all the text in this RFC gets
>> copied into the registry? And what is expected to be on the URL page then?
>> Wouldn’t it be necessary to also define a format for that page in order to
>> be of any use?
>> 
>> 
>> Mirja
>> 
>> 
>> 
>> 
>

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Mon, 2 Sep 2019 17:29:18 +0200

hi Mirja,

IIRC there was also an intent to have initial-registry on the standards track as it was initially intended (in part) as input to the LMAP WG before they changed direction (and closed), and initial-registry was to form the basis of the (standard) metrics LMAP would select among.

Cheers,

Brian

> On 2 Sep 2019, at 13:29, Mirja Kuehlewind <[email-masked]> wrote:
> 
> Hi Al,
> 
> _______________________________________________
> ippm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ippm
>

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sat, 7 Sep 2019 21:36:43 +0000

Let me add that there are ~two "private" implementations of the 
registry, BBF's and another in Brazil. Also, BEREC uses
LMAP requirements for their selected system.

Al

> From: Brian Trammell (IETF) [mailto:[email-masked]]
> Sent: Monday, September 2, 2019 11:29 AM
> To: Mirja Kuehlewind <[email-masked]>
> Cc: MORTON, ALFRED C (AL) <[email-masked]>; Michelle Cotton
> <[email-masked]>; [email-masked];
> IETF IPPM WG <[email-masked]>
> Subject: Re: [ippm] AD review of draft-ietf-ippm-initial-registry-11
> 
> hi Mirja,
> 
> IIRC there was also an intent to have initial-registry on the standards
> track as it was initially intended (in part) as input to the LMAP WG
> before they changed direction (and closed), and initial-registry was to
> form the basis of the (standard) metrics LMAP would select among.
> 
> Cheers,
> 
> Brian
> 
> > Thanks for the background info. Sorry, I should have explained myself
> better. However, I don’t have a strong opinion on which status we end up
> with but I would really like to make sure that the intended status is well
> explained in the shepherd write-up in order to provided the other ADs the
> needed background information during IESG review.
> >
> > My think was that this document does actually not specify any new metric
> and as the registry only requires “Expert Review”, informational felt more
> natural for me. However, this is for the working group to decide!
> >
> > Mirja
> >
> >
> >
> >>> 5F2&d=DwIFaQ&c=LFYZ-
> 3A__www.ietf.org_mailman_listinfo_ippm&d=DwIFaQ&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=zRoRqOY3c3gTB3VWWod8whONKOcJ4
> s5Ms5K83keHX44&s=5gzZa0AU5NM5neW3qFk2YA63T7Bq5mGppDa-qvqEIpI&e=
> >

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sun, 8 Sep 2019 12:42:15 +0000

Hi Maria,

We've started the discussion on Intended Status,
so I'll start with the quicker questions and comments
on initial-registry draft below.

Al 

> explicitly request to also include these parts?
[acm] 
I think Michelle and I were more comfortable that the 
each section/template indicated exactly where IANA assignments
were needed. It's the line between the shared responsibility of Expert 
Review and IANA to create a complete entry in the registry.

> than the IETF. Did IANA comment on that?
[acm] 
Yes, this column is something IANA/Michelle asked for, and I think we 
could be convinced that IESG could replace IETF. But to me, 
"Standards Action" is IETF.
 
> maybe also explain in the draft as his might come up again during IESG
[acm] 
The reason is simple, a draft may specify more than one registry entry,
as -initial-registry does.

I left the secY part general in case of changes during review.
It's very easy to fill this field at the end, when there is no 
possible shuffling of section order that then requires name edits, 
too. 

> this is an oversight and the references need to be updated?
[acm] 
Yes, I need to use a text editor and globally update to 7679 and 7680,
XMLMind doesn't seem to let me search in the reference fields easily
(when viewing the XML source).

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sun, 8 Sep 2019 12:56:32 +0000

Hi Mirja, 
I pressed SEND accidentally, even with wrong name, Sorry!
("More coffee, please.") Continuing soon...

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sun, 8 Sep 2019 16:35:45 +0000

Hi Mirja, 
I pressed SEND accidentally, even with wrong name, Sorry!
("More coffee, please.") Continuing on reference topics.

We've started the discussion on Intended Status,
so I'll start with the quicker questions and comments
on initial-registry draft below, on references topics.

Al 

[acm] 

[acm] 
 
[acm] 

[acm] 

> actually very clear to me here…?
[acm]
I welcome any advice on this too. If the reference is essential to 
implementing the specification, then Normative, but I'm unsure about 
making a publication Normative (as opposed to a standard).

I have fixed the pointer, we need it anyway, thanks for that.

> before we can move ahead!
[acm]
If I can know who is missing, I can bug them.

> - Is it correct that T0 in section 4 is defined in both subsections 4.3.5
[acm]
It seems better to define T0 with the Run-time Parameters 
and output reporting.
So deleted in 4.3.2 (Packet Stream Generation), 
retained in 4.3.5 (Run-time) and 4.4.2 (output).
Same for section 5.

> - Why is an URI given (only) in section 6.1.3?
[acm]
This one avoided the deletion of all URNs, with the label URI...

> think this should be: https://www.iana.org/ 
[acm]
just s/http/https/ AFAICT. 
http was correct at some point in this draft's life, fixed both.
> Also note that section 8.6 inconsistently doesn’t have any content while
[acm]
I anticipated that there are many approaches to writing 
registry entries: one will be to simply copy an existing
entry and make changes (without ever reading the Registry 
RFC). What we have mostly are instructions at the Category level,
and maybe a few cases of Column-level instructions that were
left behind after deleting most of them. When I see some text
inside < >, I'll take it out if it doesn't belong.

> " <insert name of the output type, raw or a selected summary statistic>”
[acm]
That's what I'm looking for, thanks.

> the TSopt, I would recommend to provide a pointer to the respective RFC.
[acm]
Yes, I've fixed all of this, thanks.

Please let me know if you want to see diffs with the 
with the drafts and the current working text.

Brian had asked some shepherding questions,
nits and other. I just replied to him and fixed the 
working text of -initial-registry.

---

From: Mirja Kuehlewind <[email-masked]>
Date: Mon, 9 Sep 2019 10:17:43 +0200

Hi Al,

See below.

> On 8. Sep 2019, at 18:35, MORTON, ALFRED C (AL) <[email-masked]> wrote:
> 
> Hi Mirja, 

No worries!

> on initial-registry draft below, on references topics.

Thanks!

> Review and IANA to create a complete entry in the registry.

Okay.

> "Standards Action" is IETF.

At least in the port registry we have the IESG as change controller. I was assuming that the “normal” practice… maybe Michelle can check that.

> possible shuffling of section order that then requires name edits, 

Given we are basically “at the end" I recommend to fill it now.

> (when viewing the XML source).

Good.

> making a publication Normative (as opposed to a standard).

I actually had the feeling it’s essential to implement correctly. Or alternatively maybe revise the text a bit. As long as your have a stable reference I think it can be normative. So I thought publications are fine...

> retained in 4.3.5 (Run-time) and 4.4.2 (output).

Good!

> http was correct at some point in this draft's life, fixed both.

Good!

> inside < >, I'll take it out if it doesn't belong.

This is mostly a question of consistency as well. So please check. There is also text that is not in pointy brackets in some section, which was probably initially copied from the template and is not needed anymore (and got only removed in some sections but not others).

> Yes, I've fixed all of this, thanks.

Thanks!

> with the drafts and the current working text.

No, just go ahead and submit a new version. 

> working text of -initial-registry.

Yes, thanks!

Mirja

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Wed, 11 Sep 2019 20:34:43 +0000

Hi Mirja,

I submitted -12, with secY updated for each name.
We're still waiting on Change controller advice, left as IETF.
Otherwise I think I've fixed all your points, but there 
are many ways to overlook them in a long memo, sorry if I did!

Al

> Sent: Monday, September 9, 2019 4:18 AM
> To: MORTON, ALFRED C (AL) <[email-masked]>
> Cc: [email-masked]; IETF IPPM WG
> <[email-masked]>; Michelle Cotton <[email-masked]>
> Subject: Re: AD review of draft-ietf-ippm-initial-registry-11
> 
> Hi Al,
> 
> 
> See below.
> 
> At least in the port registry we have the IESG as change controller. I was
> assuming that the “normal” practice… maybe Michelle can check that.
> 
> >
> >>
> I actually had the feeling it’s essential to implement correctly. Or
> alternatively maybe revise the text a bit. As long as your have a stable
> reference I think it can be normative. So I thought publications are
> fine...
> 
> >
> >> think this should be: https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.iana.org_&d=DwIFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=_6cen3Hn-
> e_hOm0BhY7aIpA58dd19Z9qGQsr8-
> 6zYMI&m=cP5fvDSvCv8R6d4wfrO1Cm0NBhxnC6MP3bDwAVWfjPg&s=lfu2JrEGk_CoNMtboZKJ
> U7Kc2lv475G7kQiuBXgze4M&e=
> > [acm]
> This is mostly a question of consistency as well. So please check. There
> is also text that is not in pointy brackets in some section, which was
> probably initially copied from the template and is not needed anymore (and
> got only removed in some sections but not others).
> 
> >
> >>