Subject: [urn] RFC 5226 and enhanced versions of "expert review"

From: John C Klensin <[email-masked]>
Date: Sat, 10 Aug 2013 05:46:41 -0400

Hi.

Recent discussions and tentative decisions in the URN WG point
to a sort of enhancement or fork in the way I (and I think
others) have traditionally understood the "expert review"
section of RFC 5226.

To oversimplify, "expert review" seems to be designed for a
situation in which the expert is basically performing an
extended sanity check -- are all of the bits of the template
complete and do they satisfy the rules, does the applicant
understand what they are asking for, and so on.

What the URN discussions seem to highlight is that a few
enhanced requirements would be desirable in some cases, e.g.,

(i) The "expert" has a significant tutorial role, not just a
passive registration-checking one.  Of course, for some
registries, that role is served by requirements for submission
to a mailing list.

(ii) There is enough going on in some registries and the
applications used to get entries into them, that it is better to
have an expert team who can examine an application from
different points of view and discuss it among themselves as well
as with the applicant.   That would be rather more like a design
team than like a shared expert job where two or more people
alternate reviews.  This idea of an "expert team" could be
considered a variation on the "consultation with a set of
technology experts" or "multiple designated experts... work[ing]
together in evaluating a request",described in Section 3.2 of
5526, but would make all of the members of that team accountable
to IANA and the community.

(iii) Especially for URNs and probably for some other things,
requirements for persistence and stability put a real premium on
high-quality and stable documentation.  There are good reasons
to not move to "Specification Required", but equally good
reasons to strongly encourage such specifications, possibly
including "provide document or explain why not" provisions in
templates.  That obviously puts some additional burden on the
expert process to try to cajole the document(s) into existence
or to approve the exception.

(iv) RFC 5226 provides that an expert review is conducted in
accordance with "review criteria as documented with the
protocol" (Section 3.2), but does not contain an explicit
statement that such advice to the expert(s) as to what to review
and consider, what the conditions are for rejection, etc.,
SHOULD be part of any document that specifies "expert review".
Relying one the generic criteria in Section 3.2 is really not
desirable nor is the statement about "required documentation and
review criteria" in Section 4.1.   To be clear, that is "SHOULD"
in the "do it it or explain why now" sense.

In several cases, the right text for expert review appears in
Sections 3.1 and 3.2, but the "Expert Review" definition in
Section 4.1 doesn't explicitly incorporate the relevant parts of
those sections or treat them as options that may be specified.

My reading of 5226 is that it just provides convenient templates
and that all of the above are allowed by putting the right words
into an IANA Considerations section.    There have been problems
before when some IESG member has trying to treat the categories
as normative with only a choice among them (it is easy to read
support for that view into statements like "...guidelines for
authors on the specific text that must be included..." in the
5226 abstract), but I hope those days are over.

So, a question:

Does Peter simply incorporate the URN version of the above into
3406bis with a little textual help from me (if he wants or needs
it)?  Or do I spin up a short I-D that provides an update to
5226 that explicitly adds an enhanced version of expert review
with the above as additional options?  A third possibility is to
do this as an exceptional case for URNs, then treat the URN case
as a sort of process experiment (one that does not require 3933)
to justify a 5226 update somewhat later.

best,
   john

p.s. If an update to 5226 is prepared, there are a few other
glitches that ought to be fixed or patched.  For example, 5226
explicitly contemplates creation of a registry by an independent
submission RFC.  But it says that all designated experts are
appointed by the IESG.  It is not hard to imagine that, as we
continue to open things up, a registry would be created via an
ISE process (or via the IAB or IRTF stream) for which the IESG
should not be the appointment body because there is no "relevant
Area Director" and the IESG lacks any expertise to figure out
who is and is not expert.  So, IMO, 5226upd should, first,
suggest that registry-creating documents specify selection
criteria for experts and, second, make IESG appointments simply
an option for registries created by other than IETF processes.
Perhaps some IESG advice and consent model would be appropriate
for the latter, but that is probably as far as it is reasonable
to go.

Note that the above may also require small adjustments to
statements about removing experts and who settles deadlocks and
deals with non-responsive experts.  Because an RFC is required
to create a registry, "stream manager" language may be
appropriate.

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 12 Aug 2013 13:13:40 -0600

Hi John, thank you for pursuing this topic. I agree with most everything
you've written here. A few comments inline.

> registries, that role is served by requirements for submission

As you are well aware, some tutoring does happen on some of the existing
lists (say, URI scheme registrations), but it is less intensive than
what we foresee for URN namespace registrations.

> in the "do it it or explain why now" sense.

In Berlin, Michelle Cotton pointed me to an I-D that provides more
detailed instructions for reviewers related to a particular registry:

https://datatracker.ietf.org/doc/draft-ietf-ipfix-ie-doctors/

So there is precedent for doing something beyond just pointing to the
buckets in RFC 5226.

> to justify a 5226 update somewhat later.

Given the IPFIX example cited above, I think we can make the guidelines
in 3406bis more detailed than what's provided in RFC 5226 (i.e., your
first option).

> to create a registry, "stream manager" language may be

IMHO that's excellent input for draft-leiba-cotton-iana-5226bis, which
was last updated very recently.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: SM <[email-masked]>
Date: Tue, 13 Aug 2013 21:16:40 -0700

>to IANA and the community.

I like the idea of multiple designated experts working together.  I 
saw a case where that did not work well in practice.  Someone had to 
step in as the person who was assigned the task didn't do 
it.    making the members accountable is not that easy.

>or to approve the exception.

It would be better if the specification is made available in a timely 
manner after the expert review instead of a few years down the line.

>in the "do it it or explain why now" sense.

If I recall correctly the IESG has been asking for better guidance 
for expert reviewers.  The problem might not be the rejection of an 
application.  It looks more like the lack of a "yes" has to be taken as a "no".

Regards,
-sm

---

From: Harald Alvestrand <[email-masked]>
Date: Wed, 14 Aug 2013 12:08:39 +0200

> understand what they are asking for, and so on.

That's almost right - it's been something like 15 years since Thomas and 
I wrote RFC 2434, and memory may be failing....

as I remember it, the "expert review" variant was intended to cover 2 cases:

1) We weren't sure whether the rules prevented all the cases in which 
registrations could be
considered "actively harmful" (if we felt sure, it would be "FCFS" or 
"Specification required").
That's the "guardian" aspect.

2) We felt that the rules weren't codified well enough for people to 
figure out on their own what
they needed to write in a registration template - or for IANA to be able 
to figure out all the ways
in which people could get the template wrong. That's the "guidance" aspect.

> registries, that role is served by requirements for submission

And not always well served by that function (but I digress).

> to IANA and the community.

Thomas and I considered that when writing 2434 - the reason we ended
up using "Designated Expert" was that we felt that someone would have
to be responsible for making the call; calling for a group to come to 
consensus
seemed like the wrong approach for this.
This skepticism towards "delegate to group" was also the reason why 
section 4
of 2434 said that in the case of mailing list review, use of a 
Designated Expert
MUST be specified.

Of course the expert could take advice from anyone, or even delegate
his/her responsibility - that's a lot of the extra text in 5226 section 
3.1 is about.

> or to approve the exception.

Indeed.

> in the "do it it or explain why now" sense.

Yup. One of the reasons why this isn't in the documents (especially 
2434) was because we saw that designated experts would be put in as a 
back-patch for existing registries where there wasn't really energy for 
writing down what the rules should be - the expert was a person of sound 
judgment and a safeguard against insanity, as much as anything else.
> to justify a 5226 update somewhat later.

I think having the URN version of the above as part of the URN registry 
expert instructions is a very reasonable choice, and as far as I can 
tell clearly within what 5226 allows.

Some generic version of "designated experts need to have instructions" 
should be part of 5226bis, if it isn't already clear enough from the 
text. I think the circumstances vary enough that it's good not to put 
too much normative language into 5226bis about what those instructions 
should be.

> to create a registry, "stream manager" language may be
Hm. I was also told recently that W3C could ask for a registry at IANA 
without having IESG approval. W3C doesn't do RFCs most of the time.

So not only streams might need to be named here.

Another "out" is to say that 5226bis (which is an IESG-approved 
document) only applies to registries created through the IETF stream, 
and publish an 1-page (hah) RFC that says "for stream X, 5226bis is 
followed, except that all references to IESG are replaced with <foo>".

My thoughts only, and probably more appropriate for wherever 5226bis is 
being discussed.

---

From: John C Klensin <[email-masked]>
Date: Thu, 15 Aug 2013 14:02:09 -0400

--On Monday, August 12, 2013 13:13 -0600 Peter Saint-Andre
<[email-masked]> wrote:

> Hi John, thank you for pursuing this topic. I agree with most
> everything you've written here. A few comments inline.

Sorry it has taken me so long to get back to you.  Proposed
specific text for a new Section 7 (with appropriate renumbering)
is attached.  It requires some small editorial cleanups as well
-- I'm sending you XML for everything I caught in the hope that,
if you and the WG like the text, this can just be merged modulo
whatever editorial work you decide to do on my first-draft
writing style.

Those reading the attached should note that I've made several
decisions in the direction of keeping this as process-light as
possible, deferring several types of decisions to the IESG,
IANA, the expert team, or otherwise.  My hope in doing so is
that we can end up with as much process as needed and no more
and that, where possible, small process adjustments can be made
as needed without having to either open or ignore this document.

A few comments and an additional suggestion or two below (with
lots of earlier text elided).

>...
> As you are well aware, some tutoring does happen on some of
> the existing lists (say, URI scheme registrations), but it is
> less intensive than what we foresee for URN namespace
> registrations.

I've used "mutual education" in the proposed text for two
reasons: unlike "tutoring", it cannot be interpreted as
pejorative and I actually expect that the subject matter and
namespace experts will educate the IETF-designated ones as much
as vice versa.  I hope the desired level of intensiveness comes
across.

>...
>> statement that such advice to the expert(s) as to what to
>> review and consider, what the conditions are for rejection,
>> etc., SHOULD be part of any document that specifies "expert
>> review". Relying one the generic criteria in Section 3.2 is
>> really not desirable nor is the statement about "required
>> documentation and review criteria" in Section 4.1.   To be
>> clear, that is "SHOULD" in the "do it it or explain why now"
>> sense.
> 
> In Berlin, Michelle Cotton pointed me to an I-D that provides
> more detailed instructions for reviewers related to a
> particular registry:
> 
> So there is precedent for doing something beyond just pointing
> to the buckets in RFC 5226.

There is _lots_ of precedent.  The only issue is whether a stink
arises on either IETF LC or in the IESG.  I guess we will have
to deal with those problems if they arise.

>...
>> So, a question:
>> 
>> Does Peter simply incorporate the URN version of the above
>> into 3406bis with a little textual help from me (if he wants
>> or needs it)?  Or do I spin up a short I-D that provides an
>> update to 5226 that explicitly adds an enhanced version of
>> expert review with the above as additional options?  A third
>> possibility is to do this as an exceptional case for URNs,
>> then treat the URN case as a sort of process experiment (one
>> that does not require 3933) to justify a 5226 update somewhat
>> later.
> 
> Given the IPFIX example cited above, I think we can make the
> guidelines in 3406bis more detailed than what's provided in
> RFC 5226 (i.e., your first option).

Proposed text attached, as noted above.   Putting it in required
tuning elsewhere and I've taken a shot at that.  But it is
editorial, rather than substantive, so it is probably best that
the WG look at it in context with whatever changes Peter is
making.

Added high-level suggestion:

I have not tried to fix (or otherwise modify) the template other
than to tentatively adjust an introductory sentence and add
"this item is required" sentences to the first three items.   I
do suggest that you remove as much discussion as to how specific
subsections should be filled in to Section 6 (or elsewhere).
That would have two advantages.   First, if the template itself
runs three or four pages, it will be perceived as long, complex,
and hard to complete no matter what is actually there.  If we
want to bias things toward getting more namespaces that are in
use registered, we should minimize that perception.  Second,
there is enough information in the non-template parts of this
document that, if someone who wants to submit a registration has
to read it, that would be A Good Thing.

>> explicitly contemplates creation of a registry by an
>> independent submission RFC.  But it says that all designated
>> experts are appointed by the IESG.  It is not hard to imagine
>> that, as we continue to open things up, a registry would be
>> created via an ISE process (or via the IAB or IRTF stream)
>> for which the IESG should not be the appointment body because
>> there is no "relevant Area Director" and the IESG lacks any
>> expertise to figure out who is and is not expert.  So, IMO,
>> 5226upd should, first, suggest that registry-creating
>> documents specify selection criteria for experts and, second,
>> make IESG appointments simply an option for registries
>> created by other than IETF processes. Perhaps some IESG
>> advice and consent model would be appropriate for the latter,
>> but that is probably as far as it is reasonable to go.
>> 
>> statements about removing experts and who settles deadlocks
>> and deals with non-responsive experts.  Because an RFC is
>> required to create a registry, "stream manager" language may
>> be appropriate.
> 
> IMHO that's excellent input for
> draft-leiba-cotton-iana-5226bis, which was last updated very
> recently.

Ack.  Barry, consider yourself provided with input and pass
along as appropriate.

best,
   john

---

From: SM <[email-masked]>
Date: Fri, 16 Aug 2013 21:15:45 -0700

Hi John,

I'll comment on part of the suggested text.

   "6.  Unless mutually agreed by the submitters and expert team in order
        to better respond to questions or improve the specification, the
        length of time between template submission and registration in
        the IANA database is not expected to exceed 30 days."

In another context a submitter mentioned that the evaluation period 
elapsed without the registration request being approved or 
denied.  "Not expected to exceed 30 days" can be overridden by other 
considerations.  There isn't much the submitter can do as the person 
may be unfamiliar with the process or he/she might be considered as 
being difficult if he/she invokes process.  I suggest:

    6.  Unless mutually agreed by the submitters and expert team in order
        to better respond to questions or improve the specification, the
        length of time to deny the registration should not exceed 30 days.

Regards,
-sm

---

From: John C Klensin <[email-masked]>
Date: Sat, 17 Aug 2013 16:20:20 -0400

--On Friday, 16 August, 2013 21:15 -0700 SM <[email-masked]>
wrote:

> In another context a submitter mentioned that the evaluation
> period elapsed without the registration request being approved
> or denied.  "Not expected to exceed 30 days" can be overridden
> by other considerations.  There isn't much the submitter can
> do as the person may be unfamiliar with the process or he/she
> might be considered as being difficult if he/she invokes
> process.  I suggest:
> 

In principle, I agree with you.  In practice, every time we have
tried to write a rule that precise and tie it to a deadline,
we've found that it gets us into trouble.  Maybe "better respond
to questions or improve the specification" is broad enough or
"should not" is still value enough, but I think we should be
quite careful in going very far down that path.

best,
   john