Subject: [hrpc] draft-irtf-hrpc-political - current determination regarding RG last call.

From: [email-masked]
Date: Mon, 14 Oct 2019 10:44:49 -0400

Hi,

Now that the conversation has paused, I believe it is time to say
something about the draft and its readiness for RG last call.  In order
not to bury the lead, my conclusion is that at this time is that it is 
not yet ready.  I will explain

First I want to say that I think this is an important piece of work and
one that needs to be done right.  I think the most value can be achieved
by having a document that does justice to the various sides of this
important question, and one that points the way to have further
discussions about the implications of the question in a balanced, 
informed and considered manner.

I think the document does a decent job of expressing one side of the
argument, though there are places in that discussion that are shy on
references and on arguments that logically show why that view should be
considered the right argument. Nonetheless the basics of that argument
are well established. What I think is missing is an equivalent basis for
opposing views. I think the various detailed discussions of disagreement
support that view.  Opposing views, while in some cases mentioned and
even described briefly, are not given the same depth of analysis that is
given to the predominant view.

In terms of the HRPC RG's view on the document, the group of respondents
was split. There are several who think the document is ready to go.
There are also as many who think it is not. In my opinion, we have not
yet reached rough consensus for publication.

I appreciate the speed with which Niels put out versions with the
specific fixes he believed necessary to satisfy the arguments, but feel
they sometimes missed the point about the document not being a balanced
discussion of the issues. I also think we saw that those with a
different point of view did not agree with the updated presentation.

Regarding the discussion of whether we need RG consensus to publish in
the IRSG stream: as I have said several times, I believe that in
becoming a RG draft, a private draft becomes subjected to the RG rough
consensus process.  It is not that all the issues in the doc need to
have rough consensus agreement, but the document itself needs to be
something the RG supports publishing in the IRSG track. This is the same
process we followed with RFC8280. It is not as if the IRSG RFC track is
the only opportunity people have to publish their ideas or their
advocacy. I believe the IRSG RFC track serves as a way for a RGs to
publish documents that are supported by the RG's research and
discussions. This is something I try to make clear to people when they
request that a private draft become a RG draft, I ask whether people
wanted the draft to become subject to RG rough consensus because that is
a criterion I use in moving the document forward.

As I said above, I think this is an important piece of work and think
that this RG should be able to produce a well researched and broad
document on this subject; one which has often been an elephant in the
room in technical discussions. At this point, I think it is only part
way there.

My current thought is that I would like to see work on this draft
continue. I think it might be good to find an additional editor or two
to work alongside Niels on taking the next steps on the doc. I am
interested in hearing from volunteers for this task.  I would also like
to devote some time in Singapore to a discussion of what content is
needed to complete the draft and to develop a plan for its completion.
The discussion brought up many issues.  I am still working on trying to
abstract a brief view of the issues that people brought up and plan on
presenting that as part of the discussion. There were also a few
suggestions of what the document still needed, which I would like to
explore in open discussion.

I appreciate the amount of thought and effort people put into responding
to my call for feedback. I also appreciate the amount of Niels' work
that has gone into the draft to date.

thanks

avri

---

From: Niels ten Oever <[email-masked]>
Date: Tue, 15 Oct 2019 00:17:45 +0200

Thanks for this Avri.

May I suggest that the RG appoints a document shepherd that can help advise on the (progress of the) content of the document? 

The shepherd could identify on which points there is consensus, where it needs more work, and which points are contentious (which might mean they need more work, or that they should be described in the document as points of contention). 

That could probably help the RG iterate in a constructive and structured fashion, as described in RFC 4858:

  The shepherd must keep the document moving forward, communicating
   about it with parties who review and comment on it.  The shepherd
   must obtain the working group's consensus for any substantive
   proposed changes.  The shepherd is the leader for the document and
   for the working group, and maintains a critical and technical
   perspective.  In summary, the Document Shepherd continues to care for
   a shepherded document during its post-WG lifetime just as he or she
   has done while responsible for the document in the working group.

Best,

Niels

On 10/14/19 4:44 PM, [email-masked] wrote:
> _______________________________________________
> hrpc mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/hrpc
> 

-- 
Niels ten Oever
Researcher and PhD Candidate
DATACTIVE Research Group
University of Amsterdam

PGP fingerprint	   2458 0B70 5C4A FD8A 9488  
                   643A 0ED8 3F3A 468A C8B3

---

From: Melinda Shore <[email-masked]>
Date: Mon, 14 Oct 2019 14:43:59 -0800

On 10/14/19 2:17 PM, Niels ten Oever wrote:
> That could probably help the RG iterate in a constructive and
> structured fashion, as described in RFC 4858:

RFC 4858 describes a document shepherd's responsibilities once
a document has cleared WG last call and is moving through IESG
review, IETF last call, and so on, and I'm not sure is quite
the right model for dealing with contentious documents.  It's
also the case that at the working group level, only chairs
determine consensus.  That said, this is an RG and not an IETF
working group and I think (and Colin can correct me if I'm
wrong) RG chairs have broad latitude when it comes to process.

Personally, I'm not fond of institutionalizing more roles and
more process than is really needed, and I think that many if
not all of the concerns you've raised here and that Avri raised
in her message can be addressed by the addition of a suitable
(fsvo) co-editor.

Melinda

-- 
Melinda Shore
[email-masked]

Software longa, hardware brevis

---

From: farzaneh badii <[email-masked]>
Date: Mon, 14 Oct 2019 23:59:19 -0400

I think Avri has accurately captured the discussion.

Having a couple of co-editors would be good.

Best

Farzaneh

---

From: Eliot Lear <[email-masked]>
Date: Tue, 15 Oct 2019 09:12:51 +0200

Hi Avri,

First, I appreciate the level of effort that Niels is putting into this work.  I want to take issue with two points you made:

> On 14 Oct 2019, at 16:44, [email-masked] wrote:
> 
> considered the right argument.

To me this was never about “sides”.  To begin with there may be a great many sides.

Regardless, once there is a clear and bounded research question, reasonable argumentation and references, and a bounded conclusion, regardless of whether I agree with the work, I am comfortable with it being published.  Some might want to publish opposing argument.  So long as we as a group are open to those publications as well, why would it be necessary to hold up this work?

Having written an economic and policy paper with people who had opposite views[1]*, I don’t recommend the practice.  One gets into all sorts of arguments about methodology and reference quality etc with one’s co-authors, and for what?  What is the value of such a work over two separate works?

Which brings me to my next point:

> consensus process.

Yes, you have said this.  I do not recall you saying why you would take this approach.  Can we please have a discussion as to whether this is the best way forward?  What is the value of rough consensus in this case?  Or perhaps I should ask that question differently: on what do you seek rough consensus: the positions in the document or whether it is simply in good enough form to be sent up to the IRSG?  The latter is a whole lot easier to get to.

Eliot

*I did it once and not again because of the arguments between authors.

[1] http://people.csail.mit.edu/wlehr/Lehr-Papers_files/Lehr%20Lear%20Vest%20TPRC08%20Internet%20Address%20Running%20on%20Empty.pdf <http://people.csail.mit.edu/wlehr/Lehr-Papers_files/Lehr%20Lear%20Vest%20TPRC08%20Internet%20Address%20Running%20on%20Empty.pdf>

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 15 Oct 2019 06:20:59 -0700

I concur partially with Eliot: i don't think this is about sides. I
actually suspect that Niels and I agree to a fair extent about the
political nature of standards development. My concern is not that but
rather whether this document makes interesting/useful claims and whether it
supports them with evidence. At present, it's not clear to me that it does
so, or really even what point it is trying to make.

On Tue, Oct 15, 2019 at 12:13 AM Eliot Lear <[email-masked]> wrote:

> Hi Avri,
>
> First, I appreciate the level of effort that Niels is putting into this
> work.  I want to take issue with two points you made:
>
> Regardless, once there is a clear and bounded research question,
> reasonable argumentation and references, and a bounded conclusion,
> regardless of whether I agree with the work, I am comfortable with it being
> published.  Some might want to publish opposing argument.  So long as we as
> a group are open to those publications as well, why would it be necessary
> to hold up this work?
>
> Having written an economic and policy paper with people who had opposite
> views[1]*, I don’t recommend the practice.  One gets into all sorts of
> arguments about methodology and reference quality etc with one’s
> co-authors, and for what?  What is the value of such a work over two
> separate works?
>
> Yes, you have said this.  I do not recall you saying why you would take
> this approach.  Can we please have a discussion as to whether this is the
> best way forward?  What is the value of rough consensus in this case?  Or
> perhaps I should ask that question differently: on what do you seek rough
> consensus: the positions in the document or whether it is simply in good
> enough form to be sent up to the IRSG?  The latter is a whole lot easier to
> get to.
>

Well, "good enough to send to the IRSG" needs to be judged against some
standard. In any case, having the document be an RFC under the IRTF stream
involves a certain amount of endorsement and so I would want there to be
some level of consensus that the we had confidence in the statements in the
document (just as I would want if I were to be on the PC of a conference
which was considering this as a submission).

-Ekr

---

From: Ted Lemon <[email-masked]>
Date: Tue, 15 Oct 2019 09:37:53 -0400

On Oct 15, 2019, at 9:20 AM, Eric Rescorla <[email-masked]> wrote:
> I concur partially with Eliot: i don't think this is about sides. I actually suspect that Niels and I agree to a fair extent about the political nature of standards development. My concern is not that but rather whether this document makes interesting/useful claims and whether it supports them with evidence. At present, it's not clear to me that it does so, or really even what point it is trying to make.

Another issue, which I think is leading to you phrasing it this way, is that what “political” means varies, and it’s important that the term not be used as if it has a single meaning.  Doing so muddies the communication.  Of course all of the meanings of politics are examples of what is described in the definition that’s given, but that’s not sufficient.   It would vastly improve the document if section two actually enumerated the various meanings of “political” and then if those meanings were used explicitly later in the document.

Examples:

political in effect: because things are a certain way, this affects politics
political in development: because there are competing interests, consensus is reached through politics
historically political: the status quo was arrived at through politics, but those politics are no longer relevant

etc.   I’m leaving it here because there was a really good enumeration of these issues earlier in the discussion, and that would probably be a better taxonomy than one I’d come up with here off the cuff.

---

From: Eliot Lear <[email-masked]>
Date: Tue, 15 Oct 2019 16:11:50 +0200

Hi Eric

> I would want there to be some level of consensus that the we had confidence in the statements in the document (just as I would want if I were to be on the PC of a conference which was considering this as a submission).

Yes.  My point is simply that it is possible for two or more papers to be well written that rely on different data sets or different assumptions to derive different analysis and reach differing conclusions, and so long as I am not being asked to endorse one over the other, and so long as I have learned something interesting, then I would want the work to be published.

Eliot

---

From: [email-masked]
Date: Tue, 15 Oct 2019 10:21:41 -0400

Hi,

Thanks for the questions.

On 15-Oct-19 03:12, Eliot Lear wrote:
> Hi Avri,
>
> First, I appreciate the level of effort that Niels is putting into
> this work.  I want to take issue with two points you made:
>
>> On 14 Oct 2019, at 16:44, [email-masked] <mailto:[email-masked]> wrote:
>>
> regardless of whether I agree with the work, I am comfortable with it
> being published.  Some might want to publish opposing argument.  So
> long as we as a group are open to those publications as well, why
> would it be necessary to hold up this work?
>
> Having written an economic and policy paper with people who had
> opposite views[1]*, I don’t recommend the practice.  One gets into all
> sorts of arguments about methodology and reference quality etc with
> one’s co-authors, and for what?  What is the value of such a work over
> two separate works?

Sides: I think of the 'sides' in the definition of a discussion in a
debate. Has 'side' joined the list of "bad words?" I can find another
word to use.

I too have worked on documents where various people came to the
discussion with differeing world views and perspectives. I think it can
be done.

I see several values in doing so, among them:

- gives a richer document with well developed perspectives.

- the discussion of perspectives in a serious and non antagonistic
manner forces all of us in the RG, and those who read the resultant
work, to think through issues in greater detail. Often that results in
more nuanced ways of looking at issues, often a good thing.  A document
that can do that adds to knowledge and can sometimes be a useful guide
to later discussions.

- avoids, I hope, any argument that the group is somehow publishing a
view that impinges on, or represents, the IETF in any way. We have just
weathered a storm of the IETF powers that be, both directly and behind
the scenes, over the fear that the RG was somehow impinging on the
engineering work of the IETF. I see traces of such concern in some of
the comments.  I think a well balanced work informs without prejudicing.

While I understand that achieving such a goal is difficult and sometimes
takes more time than anyone likes, I think it produces better work.

> Yes, you have said this.  I do not recall you saying why you would
> take this approach.  Can we please have a discussion as to whether
> this is the best way forward?  What is the value of rough consensus in
> this case?  Or perhaps I should ask that question differently: on what
> do you seek rough consensus: the positions in the document or whether
> it is simply in good enough form to be sent up to the IRSG?  The
> latter is a whole lot easier to get to.

The approach was one set at the beginning of the group. At the beginning
we discussed that we would, as close as possible, follow IETF practice
when engaged in RG process and RFC development, while being completely
open to doing work outside of the ISOC umbrella organization in ways
appropriate to other contexts.  If, e.g. the HRPC RG decided it wanted
to submit a contribution to the UN General Assembly, the ITU or a
journal, I would recommend we produce work that was appropriate to that
context.

I am not arguing co-chairs prerogative, except in trying to keep to that
approach. I am happy to have a discussion about changing the basis on
which we work. Perhaps some time in the upcoming meeting could be
devoted to process style. Such discussions do seem to be trending and
perhaps we should join the trend.

As for rough consensus, what I look for is that the document is a fair
expression of the issue.  As we did with RFC8280, and though it took a
while, we achieved rough consensus on publishing the document before I
shepherded it through the IRSG process. This means, at the least,
explaining a variety of perspectives and research approaches, even those
we are not focusing on in other work, in a fair manner that those who
hold that perspective can accept. RFC8280 took a while and was rough
going at times, but I think it was a good outcome.

I argue that there are many venues for work that argues a single
perspective, whether scientific or advocacy. There are many publication
venues for the ping pong of articles that oppose each other.  I never
saw the RFC series in that light, maybe my mistake. I see the RFC track
as one for well formed work that can stand the test of time. Probably a
discussion for a different space. That is, in any case, a criterion I
have in mind.

While I was employed as an academic, I tried to argue, with sometimes
limited success, that RFCs should be seen as the equivalent of published
books because they were just as difficult to produce given the arduous
process they went through. I still believe that, though there is work
published in the series that sometimes makes me doubt my perspective.

As of this point, that is still my plan as the co-chair ushering this
doc through the process. But if there is rough consensus for doing
something else, then that is what I will try to carry forward with. I
have not seen a rough consensus for a doc with a single main argument
and thus keep looking for balance.

> *I did it once and not again because of the arguments between authors.

I personally think it is important to continue trying to do so. Might be
wrong. Perhaps my perspective comes from never having started work with
the same perspective as my fellow authors and editors, but having been
part of works that eventually achieved a balance.

thanks

avri

>
> [1]
> http://people.csail.mit.edu/wlehr/Lehr-Papers_files/Lehr%20Lear%20Vest%20TPRC08%20Internet%20Address%20Running%20on%20Empty.pdf
> <http://people.csail.mit.edu/wlehr/Lehr-Papers_files/Lehr Lear Vest
> TPRC08 Internet Address Running on Empty.pdf>
>
>
>

---

From: Eric Rescorla <[email-masked]>
Date: Tue, 15 Oct 2019 09:17:47 -0700

On Tue, Oct 15, 2019 at 7:11 AM Eliot Lear <[email-masked]> wrote:

> Hi Eric
>
> > I would want there to be some level of consensus that the we had
> confidence in the statements in the document (just as I would want if I
> were to be on the PC of a conference which was considering this as a
> submission).
>
>
> Yes.  My point is simply that it is possible for two or more papers to be
> well written that rely on different data sets or different assumptions to
> derive different analysis and reach differing conclusions, and so long as I
> am not being asked to endorse one over the other, and so long as I have
> learned something interesting, then I would want the work to be published.
>

Yes, I agree with this. However, I do not think that this document as
currently written is anywhere near strong enough to serve that purpose.

-Ekr