Subject: Re: Alternative decision process in RTCWeb

From: "Scott O. Bradner" <[email-masked]>
Date: Sun, 1 Dec 2013 18:43:16 -0500

+1

i.e., maybe N=0 in this case

changing the basic genetics of the IETF is not warranted in this case (imo)

in the past (e.g., multicast routing) we formed two WGs and the problem worded itself out 
(one WG produced a result)

Scott

On Nov 28, 2013, at 10:35 AM, Sam Hartman <[email-masked]> wrote:

>>>>>> "Gonzalo" == Gonzalo Camarillo <[email-masked]> writes:
> 
> 
> Hi.  I've thought about the general issue of alternative decision making
> a lot, and it did come up a number of times while I was an AD, although
> none with such significant stakeholder interest as this.
> 
> The issue I'm most concerned about is confirming that there is rough
> consensus to make a decision.  I think any variance from rough consensus
> for decision making must itself meet a fairly high consensus bar.
> 
> "But we can't make a decision; we have to make a decision, so of course
> we're going to do something alternative," you might say.
> 
> You don't have to make a decision though.  You can decide the IETF is
> not the right forum for your decision.  You can decide not to
> standardize.  You can decide to run experiments, get  data, try to
> implement.  You can give up.
> You can redefine the problem.  You can for example decide to have two
> mandatory-to-implement options, or require receivers to implement both
> and not mandate which senders must pick.
> 
> In many cases it does turn out to be easier to get consensus that a
> decision must be made than it is to get consensus on a decision.
> However without chairs being able to clearly show that consensus to
> decide has been reached I'd expect an appeal and expect that appeal to
> be successful.
> 
> Next, it's strongly desirable to get consensus on your process for
> making the decision.
> If you can get that, or lacking that, get consensus  on someone who will
> decide the process (chairs or AD), I think you are coming along
> reasonably.
> 
> Obviously concerns of fairness are important.  A process that inherently
> advantaged one company or viewpoint would be suspect.
> 
> I'd personally favor coin-flips, external arbitors or similar over
> voting.  We don't want to encourage voting because we could get into
> situations where people block the consensus process in order to force a
> vote.  Absent things like well-defined membership and procedures to
> avoid one company stuffing the ballot box, going there seems
> problematic.
> 
> Coin flips are nice.  They really create pressure among anyone who
> actually cares about the outcome to compromise, to explore whether a
> consensus can be built.  However for the frequent situations where a
> decision is critical but it really doesn't matter (even if some people
> think it does), they get the job done.

---

From: "Joel M. Halpern" <[email-masked]>
Date: Sun, 01 Dec 2013 22:52:26 -0500

For Inter-Domain Multicast Routing, there was one working group.  When 
it could not converge, both protocols were published as Experimental. 
Several years later, well after we knew what had eventuated, we moved 
one to the standards track.   (Part of the delay was that the winner 
badly needed a rewrite for readability.)

Yours,
Joel

---

From: Jari Arkko <[email-masked]>
Date: Mon, 2 Dec 2013 14:28:39 +0200

I'm not an expert on webrtc or codecs, but for what it is worth, I have some opinions on the general issue of selecting technology in a WG. For full disclosure, in my day job my colleagues have a stake in the matter, but I'm just representing my own opinions.

Back to basic principles: Community decides. Interoperability. Running code. Best technology. 

Community decides: I think it is important that the WG and the IETF community decides what to do in this case. The WG is in a deadlock, and they should be able to decide - by consensus - if they want to attempt resolving this in a particular way. Including alternate decision mechanisms. Of course, we all may have opinions about how that resolution should happen. I'm personally in the "coin toss" camp (but see further below). 

And I'm reluctant for us in the management to attempt to override the decision in any way. Lets learn what the community wants to do. If you do not like the proposal on the table, make your opinion known or propose an alternative. Also, I've heard a couple of arguments saying that we should be worried about appeals on this matter. I'd say we should just try to the right thing. 

However, the WG process is allowed to "fail" (if you can call it that) and not standardise something that we do not find consensus on. We've been through this many times in the past, and took various actions. Sometime it did lead to considerable market split and lack of interoperability, sometimes a practical unified reality emerged from what the vendors did, sometimes we learned to live with multiple choices, sometimes we returned to the question sometime later and found a standard. It has usually not been the end of the world. 

Interoperability: This is one of the core values of the IETF. The reason why I think it makes sense to try hard to come to a decision. Now, it may not be possible, but having a common codec is very desirable. But in attempting to come to a decision, it is important that we all try to understand the real-world impact on interoperability. An agreement only on paper but not in what products actually do is not so useful. Understanding the real willingness of the parties to conform to what the RFC will say is important, no matter whether we are talking about consensus, coin toss, or voting.

Running code: (Or running silicon, as it may be.) Another core value of the IETF.

Best technology: Again, a core value. We should be careful about using non-technical arguments in making a decision about technology standards. Of course we all take all information into account. But technical arguments usually have better longevity. Good trumps bad even tomorrow. Other arguments are more susceptible to being changed, e.g., due to contracts, licenses, passage of time, and behaviour of parties who might not even be involved in the IETF today. Pretty much the only reliable information for the other factors is past performance, which leads us back to running code and experience. But it is indeed possible that on all objective criteria, two alternatives come out equal. As an Internet user, in that situation I'd prefer the WG to make a choice, *if* that choice actually leads the various players to be more unified than us not making a choice. 

Jari

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 2 Dec 2013 06:40:06 -0800

I am not active in RTCWeb, but I have been discussing the deadlock on a mandatory-to-implement video codec with a few people.

> However, the WG process is allowed to "fail" (if you can call it that) and not standardise something that we do not find consensus on. We've been through this many times in the past, and took vario…

A big +1 to this part of Jari's message. The IETF has seen more situations like this than we have seen "we'll vote, but just this once".

--Paul Hoffman

---

From: Avri Doria <[email-masked]>
Date: Thu, 28 Nov 2013 20:22:19 +0000

Hi,

Is it possible to reach rough consensus on MTI of one from (H.264 ,VP8 )  leaving the choice of which, to each implementation?

Avri

Yoav Nir <[email-masked]> wrote:
>On 28/11/13 8:10 PM, Tim Bray wrote:
>> It seems to me that if there are sane strongly-held objections to
>both 
>> of the alternatives on the table, then neither alternative is
>suitable 
>> for a standards-track RFC.
>So, does that mean that there should not be a standardized codec for 
>WebRTC, or that we should come up with with a new alternative that 
>nobody wants?

~~~
avri

---

From: "cb.list6" <[email-masked]>
Date: Mon, 2 Dec 2013 08:10:56 -0800

On Dec 2, 2013 8:08 AM, "Avri Doria" <[email-masked]> wrote:
>
> Hi,
>
> Is it possible to reach rough consensus on MTI of one from (H.264 ,VP8 )
leaving the choice of which, to each implementation?
> Avri

That does not achieve the goal of ensuring 2 or more peers have a common
codec to avoid codec negotiation failure

CB

>>> It seems to me that if there are sane strongly-held objections to both
>>> of the alternatives on the table, then neither alternative is suitable

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 2 Dec 2013 18:17:29 +0100

On 30 Nov 2013, at 16:08, Dave Crocker <[email-masked]> wrote:

> On 11/28/2013 7:27 PM, Carsten Bormann wrote:
>> In a similar vein, can anyone point out what we get if the IETF were to agree on a single MTI video codec for WebRTC?
>> What is the upside to making this herculean effort?
> 
> 
> basic interoperability, without prior agreement.

If making something MTI in a spec guaranteed that interoperability, I wouldn’t have asked.

This is not a bikeshed decision.  For an implementer, there appear to be significant commercial/legal implications that overshadow any technical considerations, or any spec “compliance” for that matter.  

> but perhaps you are asking a deeper question?

Yes, I was interested in the objective for continuing to insist on a decision, given that the above guarantee doesn’t seem to hold at all under the current circumstances, and there is likely to be significant collateral damage from any decision.

Right now it seems to me that focusing on this decision mostly serves to slow down WebRTC.
But then I’m just an interested bystander.  That’s why I’m asking.

At a minimum, I’d expect continued work on such a decision to be entirely decoupled from the technical work.  (I also tend to believe Eric’s Solomonic proposal hits the mark, but getting rid of the process block is more important.)

Grüße, Carsten

---

From: Pete Resnick <[email-masked]>
Date: Mon, 2 Dec 2013 14:25:56 -0600

> And I'm reluctant for us in the management to attempt to override the decision in any way. Lets learn what the community wants to do. If you do not like the proposal on the table, make your opinion…

Jari, I think we should all be clear that "community decides" is not 
carte blanche. For example, a WG can't simply decide on its own that it 
will no longer take decisions on the mailing list, but that all 
decisions will be taken face-to-face and simply announced on the mailing 
list. That would be directly against a documented process on which we 
have IETF consensus, and it would take a conscious change of that 
process by the entire community to do that. If such a process were being 
suggested, I think the IESG would have an obligation to step in.

In this case, the waters are much murkier (there's nothing that 
specifically says we can't vote), so I think it's reasonable to let the 
community take a stab at how it wants proceed. If the WG can come up 
with a process that involves sticking hands up in the air or paper 
ballots or whatever that still protects (to quote a certain document) 
"the rights of all parties in a fair and open Internet Standards 
Process", then we're doing "the right thing". But they do have to come 
up with that fair and open process. I for one am willing to wait and see 
if the WG and the larger IETF community can come to consensus on how to 
do that.

pr

-- 
Pete Resnick<http://www.qualcomm.com/~presnick/>
Qualcomm Technologies, Inc. - +1 (858)651-4478

---

From: Jari Arkko <[email-masked]>
Date: Mon, 2 Dec 2013 22:31:04 +0200

> In this case, the waters are much murkier (there's nothing that specifically says we can't vote), so I think it's reasonable to let the community take a stab at how it wants proceed. If the WG can come up with a process that involves sticking hands up in the air or paper ballots or whatever that still protects (to quote a certain document) "the rights of all parties in a fair and open Internet Standards Process", then we're doing "the right thing". But they do have to come up with that fair and open process. I for one am willing to wait and see if the WG and the larger IETF community can come to consensus on how to do that.

Yes - agreed.

Jari

---

From: Sam Hartman <[email-masked]>
Date: Mon, 02 Dec 2013 17:04:56 -0500

>>>>> "Carsten" == Carsten Bormann <[email-masked]> writes:

    Carsten> On 30 Nov 2013, at 16:08, Dave Crocker <[email-masked]> wrote:
    >>> In a similar vein, can anyone point out what we get if the IETF
    >>> were to agree on a single MTI video codec for WebRTC?  What is
    >>> the upside to making this herculean effort?
    >> 
    >> 

    Carsten> If making something MTI in a spec guaranteed that
    Carsten> interoperability, I wouldn’t have asked.

>From Cullen's message it sounds like a number of people believe that the
MTI  mechanism in the spec will influence what some implementations do.

For me that has real value and seems like a reason to spend significant
effort trying to come to a decision.

---

From: Ofer Inbar <[email-masked]>
Date: Mon, 2 Dec 2013 18:10:22 -0500

Avri Doria <[email-masked]> wrote:
> Is it possible to reach rough consensus on MTI of one from
> (H.264, VP8) leaving the choice of which, to each implementation?

It sounds like that is the default if the working group does *not*
pick a mandatory.  Many implementations will do both of those, some
will do only one or the other - and some of those may end up with
no codec in comment and will not interoperate.  The whole point of
this excercise is to try to avoid that situation.

It may be unavoidable, in which case it's what will likely happen.
  -- Cos

---

From: Gonzalo Camarillo <[email-masked]>
Date: Fri, 6 Dec 2013 12:57:11 +0200

Hi,

I want to thank everybody for all the input on this issue. Let me
address a few of the points that have been brought up during the
discussions.

People seem to have different views on which processes MUST be used at
the IETF when developing documents, which ones MAY be used, and which
ones MUST NOT be used.

Given the strong disagreements we have seen, it may be useful to agree
on a set of guidelines and document them at some point. In the mean
time, let me clarify how the RTCWeb WG (or at least some people in that
community) interpret the IETF process.

The view a part of the RTCWeb community shares is the following:

Documents MUST go through both a WGLC and an IETF LC where the community
needs to reach rough consensus on them. That is, per standard IETF
processes, documents that reach rough consensus at those stages are
progressed while documents that don't reach rough consensus are *not*
progressed.

In summary: proposals are evaluated first by the WG and then by the IETF
community as a whole using a consensus mechanism.

Where people seem to disagree, often strongly, is on how proposals that
will be put forward for such evaluation by the WG and the IETF community
can be generated.

Some people think that processes that are somewhat related to voting
MUST NOT be used within a WG at all. Other people think that using
processes such as straw polls or some types of voting can help the WG
understand better the situation at hand and help build consensus, which
will be *ultimately* evaluated in the WGLC and IETF LCs on the document.

In short, some people think that the important fact is that documents
need to reach consensus at the WGLC and IETF LC stages. Before that, one
could use different processes if they are likely to help building that
consensus. Other people disagree.

My personal opinion is that both interpretations above have some merit
and should be discussed within the IETF community, which will sure face
difficult situations like this one in the future again. Future WG chairs
will benefit from more clarity on these issues.

On a related note, there have been some comments about the RTCWeb chairs
effectively attacking the IETF principles. I think those comments are
unfair. We are talking about three former ADs in different areas all of
whom have made significant contributions to the IETF community along
many years. They understand the situation at hand and are trying to
actively propose alternatives to resolve a difficult and important
issue. Let's discuss which interpretation of the IETF process the
community agrees more with (per the points above) instead of judging
people for interpreting the rules differently in an area that is,
arguably, not completely clear.

With respect to the decision about the MTI video codec at the RTCWeb WG,
the chairs will be proposing a way forward (taking into account all the
input received) shortly.

Cheers,

Gonzalo

On 28/11/2013 11:17 AM, Gonzalo Camarillo wrote:
> Folks,
> 
> as you may know, the RTCWeb WG is trying to select a
> mandatory-to-implement video codec. So far, the WG has been unable to
> reach consensus using traditional consensus calls. Now, the WG is
> considering alternative options to make that decision.
> 
> If you are interested in following that discussion on the RTCWeb list,
> this would be a good place to start:
> 
> http://www.ietf.org/mail-archive/web/rtcweb/current/msg09909.html
> 
> Cheers,
> 
> Gonzalo
>

---

From: Eliot Lear <[email-masked]>
Date: Fri, 06 Dec 2013 14:50:35 +0000

Gonzalo,

Thanks for this really well written and good summary.  Again, the chairs
– and the area directors – as well as the whole RTCWeb group have been
working really hard on what many of us think will be critical
technology.  As one of the whiners about voting, let me say first that
*do* believe that strictly speaking the rules can be interpreted to
allow voting.  That doesn't make it a wise choice or a *substitute* for
rough consensus.   Therefore I subscribe to the following interpretation:

On 12/6/13 10:57 AM, Gonzalo Camarillo wrote:
> Other people think that using
And I would go further, and suggest that RFC 2418 provides working group
chairs with very broad latitude to declare what rough consensus is in
such circumstances.  The document merely points out that it is > 51% and
it needn't be 99%.  That large range should be fair warning that the
ruff in this case could be quite rough.  The ultimate point, as Jari
mentioned in his note of 2 Dec 2013 is to aim for broad interoperability.

And again, as one of the whiners, I probably have a responsibility to
put forward some thoughts on how one might proceed.  I state this as an
outsider to the WG, and so perhaps these ideas have been already tried
or considered and discarded.

You've created a rather long list of options.  It seems to me that it
would be good to know what we are talking about in terms of "rough".  It
might be useful still for participants to be polled on whether or not
they could live with each of those options.  That is - have a positive
signal.  This can be used to establish a front runner or group of
popular choices.  Recalling Jari's message again that interoperability
is what we are aiming for, it might also inform the working group to
understand what user bases and implementations are impacted, at the same
time.  Stating which implementation one is developing or is planning to
deploy will hopefully provide some feel for interoperability.  This does
weight near term over long term, but I think that's a good approach (you
don't get to long term if you can't satisfy near term).

I offer these purely as suggestions to be discarded if they are not
found to be useful.

Again, I wish the WG and the chairs the very best in trying to untie
this knot, and I offer to be whatever use I can to assist.

Eliot

---

From: Harald Alvestrand <[email-masked]>
Date: Fri, 06 Dec 2013 15:59:15 +0100

> don't get to long term if you can't satisfy near term).

This is one aspect I like about the "vote" (distasteful as I find the term).
A full stack-ranking by each participant of the alternatives on the 
table gives
a clear idea of how they weigh different alternatives against each other 
- which
allows us to probe more deeply into what possible compromises might actually
be possible for participants to live with.

The chairs have promised that the actual votes (full ranking + names) will
be made public. That allows everyone to analyze the stated opinions and see
what they interpret from that information; I think that's a Good Thing.

---

From: Dave Crocker <[email-masked]>
Date: Fri, 06 Dec 2013 08:40:12 -0800

Gonzalo,

> will be put forward for such evaluation by the WG and the IETF community

That's odd.  I hadn't even noticed that that was part of the proposal 
for voting circulated to the community, nor that it was a focus of 
responses.

 From Magnus' original posting of the proposal:

      "A large number of documents, over an extended period of time, 
with nothing published, suggests some deep and serious problems for an 
effort in the IETF."

That's not about generating proposals but about choosing among them.

> many years.

As financial reports often note: "Past performance is not a guarantee of 
future returns..."

More significantly, Gonzalo, you've just invoked an ad hominem argument 
as a defense (or justification.)  It's no more legitimate as a defense 
than as an attack.  Stated simply:  the nature of the people who made 
the proposal is irrelevant.  What matters is the nature of the proposal.

And my own reading of the criticisms of the proposal that was circulated 
was that they did primarily focus on the nature of the proposal, rather 
than on the nature of the proposal's authors.

But as long as you've made this personal, what happened to the general 
preference in the IETF -- especially for efforts that are complex or 
otherwise difficult -- to have working group chairs /not/ be document 
authors, so that the chairs can focus on /neutral/ efforts at managing 
the process?

While there are never guarantees about the progress of an IETF working 
group, such a separation might have had strategic benefit for this 
effort.  I note a number of points of broader concern about this working 
group:

    1.  Chartered 1.5 years ago.

    2.  11 working group drafts, with 13 related drafts.

    3.  Nothing yet published.  No overviews, architectures,
     use cases or anything else foundational, nevermind actual
     specifications.

    4.  WG can't even resolve choice of a component technology

Frankly it does not help that the effort already seems to have excellent 
market and IETF mindshare as the 'future' of Internet 'rich 
communications'.  Even the recent IAOC request for a volunteer is 
calling for "exposure" to the technology -- although the technology 
isn't stable.

A large number of documents, over an extended period of time, with 
nothing published, suggests some deep and serious problems for an effort 
in the IETF..

What am I mis-understanding?

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Gonzalo Camarillo <[email-masked]>
Date: Sat, 7 Dec 2013 11:14:54 +0200

Hi Dave,

to be clear and somewhat more concise, the WG does *not* intend to
replace any part of the consensus process in use in the IETF. The next
step for the WG is to work out what statement about codecs it is
reasonable to include in the next revision of the document (an Internet
Draft).

That document will then go through the typical WG discussions, WGLC,
IETF LC, IESG evaluation, and approval. As usual, at any stage the
document will be discussed and possibly modified.

So, what the chairs are proposing is a way for the working group to work
out a statement about codecs around which they can build consensus. Of
course, nothing guarantees that consensus will be actually achieved.

With respect to the actual process being proposed, the chairs have
listened to all the feedback received from the RTCWeb community and the
IETF community as a whole. Taking all that feedback into consideration,
they are going to be proposing next steps shortly. They have not done so
yet simply because the flu hit them hard last week.

Cheers,

Gonzalo

On 06/12/2013 6:40 PM, Dave Crocker wrote:
>      "A large number of documents, over an extended period of time, with
> in the IETF."

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Sat, 7 Dec 2013 21:16:00 -0600

Gonzalo, thank you for your posts in this thread. They were very helpful.

Best wishes to the RAI ADs and RTCWEB co-chairs, of course.

Spencer