Subject: DISPATCH and strategies for standards maintenance

From: Harald Alvestrand <[email-masked]>
Date: Fri, 22 Jan 2016 08:33:01 +0100

Recently, in a response to a Last Call on
<draft-campbell-art-rfc5727-update-02>, I wrote the following on the
IETF list:

  Objection:

  I find the DISPATCH style of review to be a horribly broken idea.

  I also find the use of the term "working group", and the procedures of
  working groups, for what is effectively a permanent review panel and a
  standing BOF venue to be counterproductive and destructive of getting
  work done.

  This document proposes recommending this method as a generic tool that
  can be used in areas outside of the limited scope of SIP extensions -
  something I think is taking a bad idea and making it even more harmful.
  (RFC 5727 already said it would do that, but the RAI area has not
  followed up on this particular bad idea from RFC 5727, letting
  initiatives like WEBRTC flourish outside of the DISPATCH process, so the
  damage done by DISPATCH has been limited.)

  I therefore object to this document being published as a BCP.

These words were rather harsh, and were commented on as such. It seems
best for the community that I spend some words on describing why I came
to this conclusion.

When I came to deal with WebRTC and the assocated areas, I believed (not
having examined the area too closely) that with the SIP suite, which
uses SDP and RTP, we had achieved a reasonably functional and
architecturally competent real time communications suite.

I was wrong.

It turned out that a number of fundamental problems existed, including:

* A lack of clarity on the connection between SDP and RTP - the word
“session” was used at both layers, but represented different things.
* A lack of consensus on how one negotiated session properties that
applied in one direction only.
* An SDP architectural misfeature that led to a completely bogus
requirement that differing media types be carried on different UDP ports
* A number of issues with use of addresses in signalling - including the
formulation of rules like the “SDP patch-up offer/answer” for making one
set of addresses (in m-lines) match up with another set of addresses (in
ICE)
* A general acknowledgement that certain fairly absolute rules, like
“always use RTCP”, were widely ignored in the industry and couldn’t be
depended on
* A number of rules around SSRCs that were based on an operating
environment that doesn’t exist in practice (multicast groups) - in
particular SSRC collisions - which have made these identifiers much less
powerful than they might have been.
* A number of long-standardized extension features (CAPNEG in
particular) that are largely ignored by industry due to complexity, but
are still used to block other efforts by saying “why can’t you use X
instead”.
* A number of unfinished products (EKT in particular) that seemed to be
a good fit for some problem - but somehow never seemed to cross the
finish line.

This list is far from exhaustive. We have been chipping away at the
issues that mattered to the WebRTC effort, while attempting to make sure
the issues we were not addressing did not affect the effort (by not
using the features they referenced).

I observed the DISPATCH process for much of the time of my engagement.
The DISPATCH process is reasonably efficient …. at solving the wrong
problem.
The ideal DISPATCH task is a proposal to address a small problem (or
paper over some deficency in the protocol for a certain set of cases).
It allows the ADs to delay such a proposal for 3 months without any
controversy (“until the next IETF meeting”), and for 6 months if it
turns out to need a WG (“first DISPATCH, then you write a charter, then
you meet”). (Sometimes such delay is a good thing; some bad ideas just
need the time to die.)
Once the proposal reaches a DISPATCH meeting agenda, it gets some
review, it gets some comments, it gets some recommendation (maybe) - and
then it can move on, if the proposers still have energy for the push,
and if any of the experts in the area have bought into the idea enough
to give it the guidance it needs to achieve non-conflict with all the
other small patches people are working on.

What the DISPATCH process fails to achieve is what a good directorate
should have been doing: Taking a view of the whole area of the protocol
suite, and noticing things like:

* SIP is largely non-interoperable. It’s useful to connect islands of a
single product while claiming standards conformance, and it’s easier to
wrangle into some semblance of interoperability than proprietary
protocols - but it’s not “plug and play”.
* SDP is a world picture that is not capable of representing a
significant subset of the interesting ways in which people want to
interconnect. This means that solutions that use SDP have to work around
SDP, not with it.
* RTP is showing its age. Security is a bolt-on, and RTCP is more of an
extension platform than it is an useful set of features in the base spec.

It’s clear that the industry needs a robust, interoperable, secure
protocol suite for real time communication. It’s also clear that the
IETF is the logical venue for that to happen.

But DISPATCH isn’t the way to get there.

--
Surveillance is pervasive. Go Dark.

---

From: "John Levine" <[email-masked]>
Date: 22 Jan 2016 16:00:52 -0000

>paper over some deficency in the protocol for a certain set of cases).

I wouldn't disagree with anything you said, but sort of ironically,
this model is fine for the apps area stuff that's now been added to
DISPATCH.

R's,
John

---

From: Alissa Cooper <[email-masked]>
Date: Thu, 28 Jan 2016 09:22:13 -0800

Hi Harald,

> 
> What the DISPATCH process fails to achieve is what a good directorate should have been doing: Taking a view of the whole area of the protocol suite, and noticing things like:
> 
> * SIP is largely non-interoperable. It’s useful to connect islands of a single product while claiming standards conformance, and it’s easier to wrangle into some semblance of interoperability than proprietary protocols - but it’s not “plug and play”.
> * SDP is a world picture that is not capable of representing a significant subset of the interesting ways in which people want to interconnect. This means that solutions that use SDP have to work around SDP, not with it.
> * RTP is showing its age. Security is a bolt-on, and RTCP is more of an extension platform than it is an useful set of features in the base spec.

Could you point to IETF directorates that have fixed problems of a similar caliber in other areas in the recent past? It would help to understand the keys to their success.

> 
> It’s clear that the industry needs a robust, interoperable, secure protocol suite for real time communication. It’s also clear that the IETF is the logical venue for that to happen.
> 
> 

I don’t think anyone is claiming such lofty ambitions for DISPATCH, or recommending it as a model to accomplish what you describe above in other areas. I agree with you that DISPATCH is not a mechanism for re-designing or replacing whole protocol suites. We don’t have a mechanism to do that efficiently. DISPATCH is a mechanism for organizing work that IETF participants want to bring into the area. 

It strikes me that re-designing and replacing whole protocol suites is a challenge that pervades pretty much every area of networking, that no area has particularly great solutions for, and that requires activity both inside and outside the IETF. Personally I’m totally open to ideas for how we can improve upon the status quo in ART, but I think pointing the finger at DISPATCH for the cruft in the real-time protocol suite is misplacing blame.

Alissa

---

From: Adam Roach <[email-masked]>
Date: Thu, 28 Jan 2016 15:05:56 -0600

On 1/22/16 01:33, Harald Alvestrand wrote:
> When I came to deal with WebRTC and the assocated areas, I believed 
> (not having examined the area too closely) that with the SIP suite, 
> which uses SDP and RTP, we had achieved a reasonably functional and 
> It turned out that a number of fundamental problems existed...

I agree with many of your criticisms of the result of a quarter century 
of evolving requirements and feature sets, added incrementally to a 
protocol suite, with the intention of being backwards compatible at each 
step.

On the other hand, I'm not sure anything more architecturally pure could 
have emerged from such an environment, either. Long-term protocol 
evolution tends to be messy. Time for SIP/3.0, anyone? ;-)

> The DISPATCH process is reasonably efficient .... at solving the wrong 
> problem.

DISPATCH has done a good job of perfoming the task it was put forth to 
perform, which was administrative in nature.

Where you see a procedural bump on the way to working group formation, I 
see a vast improvement over what we had in the areas that historically 
homed real-time work.

Traditionally, proponents of new work that didn't clearly belong to an 
existing WG would start a mailing list, try to attract a community of 
interest, spend significant time figuring out how to get their work 
noticed by the right parties in the IETF, and ultimately -- if they're 
lucky -- get enough attention from an AD to get them to sponsor a BOF. 
Then, they needed to write and complete a draft charter in preparation 
for the BOF. This would take many rounds back and forth with the AD. 
Then, many months (and lots of work) into this process, they would 
finally meet at a face-to-face meeting and present their charter.

If they continue to be lucky, the feedback they get is actionable and 
concrete enough that they'd be able to fix the charter and work with the 
AD to get a WG for the next IETF. In many cases, however, either the 
changes indicated by charter feedback would be substantial enough that 
another BOF was called for, or the process of spinning up a WG took so 
long that they didn't get WG status the next time around. So, the time 
from wanting to bring new work to the IETF to actually having a WG was 
frequently on the order of a year or longer.

That's not a technical problem; it's a process problem. And *that's* the 
issue that DISPATCH was formed to address.

As a case study for DISPATCH, let's look at MARTINI.

In September of 2009, Richard Shockey brought forth the requirement to 
allow bulk registration of SIP AoRs with a registrar. It was discussed 
on the DISPATCH list, and dispatched to its own ad hoc meeting at the 
November 2009 IETF meeting. By December 2009, the MARTINI working group 
was chartered to define mechanism for this work. Despite non-trivial 
disagreement around the proper technical solution, the WG finished its 
work and requested publication of the final mechanism in October of 2010.

So, in less time than many pre-DISPATCH working groups took to have 
their first meeting, we went from problem statement to IETF last call.

I concede that not all DISPATCH efforts are going to proceed as smoothly 
as MARTINI did; but prior to DISPATCH, this level of time efficiency 
would have been strictly impossible.

If you want a more recent example, the first proposal for standardizing 
lossless codecs reached DISPATCH in May of 2015
(and note that the proposal was quite nascent; its proponent framed the 
conversation with: "Our preparations are not yet at the stage where we 
have created a formal draft or submission for the IETF. Before we 
proceed, we'd like to engage the IETF community in discussion of the 
specifications and suitability for IETF"). This proposal was discussed 
in July in Prague, and the CELLAR WG was chartered in November.

I like the example of CELLAR in particular, for two reasons. First, it 
shows how low the barrier is before someone can bring their work to a 
large, receptive audience. Previously, you'd need a charter mostly done 
before anyone started paying attention. Secondly, it's not clear to me 
that this work would have ever found an appropriate audience in the IETF 
without the kind of onboarding provided by DISPATCH. I suspect that, 
absent such a venue, the CELLAR work would have never received 
sufficient attention to be taken on by the IETF.

> should have been doing

Yep, because that's not what it is.

You're complaining that my pipe wrench makes for a lousy hammer. I 
agree. Please leave my pipe wrench alone and find some more suitable 
tool for pounding nails.

/a

---

From: "Ben Campbell" <[email-masked]>
Date: Thu, 28 Jan 2016 17:44:26 -0600

Hi, Harald.

First, thanks for your clarifications. I have some comments inline:

On 22 Jan 2016, at 1:33, Harald Alvestrand wrote:

[...]

> controversy (“until the next IETF meeting”), and for 6 months if 
> it turns out to need a WG (“first DISPATCH, then you write a 
> charter, then you meet”). (Sometimes such delay is a good thing; 
> some bad ideas just need the time to die.)
> review, it gets some comments, it gets some recommendation (maybe) - 
> and then it can move on, if the proposers still have energy for the 
> push, and if any of the experts in the area have bought into the idea 
> enough to give it the guidance it needs to achieve non-conflict with 
> all the other small patches people are working on.
>

I don't dispute that there are many issues within our suite of real-time 
communication technologies. I take the basis of your argument to be that 
DISPATCH has either hindered attempts to solve those issues, or at least 
hasn't helped.

I think the real issue is that the community has not had the will to do 
massive protocol fixes and replacements in this area--probably for both 
good and bad reasons. There's a lot of inertia in the need for backwards 
compatibility, and there's lots of legacy products out there. I think 
conversations on ways to improve that would be interesting, and maybe 
even useful.

But I don't think DISPATCH (either as a working group or a process) is 
the problem here. You argued that DISPATCH is only suited for solving 
small problems. As a test case, let's look at the RTCWEB working group. 
This is not a small effort. It effectively replaces much of the RTC 
signaling architecture. It also requires tight cross-SDO coordination 
with the W3C. I think rtcweb falls into the category of "ambitious 
changes to modernize IETF technologies" if anything does.

As far as I can tell from the mailing list archives, discussion of that 
work started in Nov 2010 (probably at IETF79) when you submitted drafts 
to DISPATCH. There was a BoF at IETF80, and a working group formed in 
May 2011. I recognize that the rest of the world might think that was 
slow, but it was pretty quick by historical IETF standards. I think it 
likely that DISPATCH shortened the process, possibly cutting out the 
need for at least one additional BoF.

So, my question is: What would you like to be different? Do you think 
the conventional (non-DISPATCH) IETF approach to large new efforts would 
have worked better? Do you think we need something different than either 
of them? Can you describe how you would have expected the chartering of 
RTCWEB to have gone without DISPATCH?

Thanks!

Ben.