Subject: RE: Voting (again)

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Fri, 15 Apr 2005 09:32:44 -0700

> These are not mutually exclusive, and the last point is more 
> dubious than the first two.  While deployment is a necessary 
> condition for success, so is technical soundness.  Our 
> broader purpose is not just to create new protocols - it's to 
> keep the Internet working well. 

Technical soundness can be dealt with in Darwinian terms.

Design by committee tends to wreck as much as it improves.

Your concern is to avoid mistakes, my concern is the paralysis that
affects the standards process. I don't worry very much about mistakes
because most are self correcting, people do not usually implement broken
specs. If the standards process is responsive you can discover the
mistakes quickly and move on.

> On the contrary, there are very many occasions where the way 
> to understand the solution space to a problem is to involve a 
> large number of people with varied concerns related to that 
> problem.  

If I want folk with varied concerns I talk to users, not engineers.

> You don't want so many people involved in the 
> actual design - but you do want them in on the problem 
> definition and to provide feedback to design proposals. 
> I've seen very  few occasions where a design that was done in 
> isolation
> - even by very intelligent people - held up well on the Internet.

That's is a good point but why do you think that an elite 
engineering forum would be a sensible place to achieve that?

> IETF's imprimatur does have an effect, in the sense that if there's a 
> perceived need for a solution to some problem an IETF 
> document is more 
> likely to be seen as _the_ solution than a competing solution 
> from, say, a vendor.  

Only once deployment has reached the critical mass point. 

The market already understands the open standards issue to the point of
obsession. But the issue is really one of change control. The first
question people ask is 'does this work for me', not 'is it a standard'. 

It takes so long to obtain the IETF imprimartaur that the success
question is settled by the time the imprimataur is received. Perhaps
HTTP will be recognized as a standard sometime, but I doubt it. It
should take the IESG what, five seconds to agree that HTTP is a
standard?

To give a concrete example, does anyone believe that CSV+IETF
imprimataur has a hope of displacing SPF/Sender-ID with no imprimataur?

> > The real point of a working group process is to establish the 
> > coalition of support you need to get the work deployed.
> 
> I strongly disagree.  And treating this as the real point of 
> a WG is a good way to produce garbage.  What is really needed 
> is a balance - get the right people on board to produce a 
> good solution that meets a wide variety of needs, AND who 
> will help get the result deployed. That's not the same set of 
> people, but often there is some overlap.

I don't have any problem with either group you mention. The problem in a
WG is when you get visited by rock throwers or zealots who have a
completely different agenda, or some other group that wants to get their
standard deployed by making it a precondition for a spec that has a lot
of public momentum.

We still have WGs being told that they have to support BEEP, despite the
fact that it is inadequately engineered as a web services transport, has
been decisively rejected by the market as a web services transport and
is based on obsolete technology. SACRED was bounced into making BEEP
their transport despite the fact that the spec only makes sense as an
adjunct to SAML and XKMS which are SOAP based. The RID spec was made
worse by the insistence on support for BEEP.

Why is a 'standards' organization enforcing this Not Invented Here
attitude? If you look at what every Web Services platform supports, both
commercial and FOSS the only conclusion that is possible is that BEEP
should be moved out of standards track and considered obsolete.

The SOAP work won because it is supported by a much wider coalition of
vendors and developers. It established the necessary support. BEEP was
fast tracked through IETF in a transparent attempt to try to pre-empt
SOAP. It is a failure because the WG did not have any of the Web
Services platform developers participating.

> People will say to 
> themselves, if the FemtoSoft or TransCo guy is that opposed to this 
> proposal, it doesn't have a chance of being deployed, so I'm 
> going to withhold my support also.  I've seen it happen lots 
> of times.  I'm 
> very opposed to giving FemtoSoft and TransCo more votes than 
> anyone else but I don't have a problem with individuals 
> considering that certain contributors input carries more 
> weight than others.

Which is the same process I see.

Yes there are occasions in which you get a company that has some evil
vendor scheme going on, I have not seen many cases where this has
succeeded. 

But I have seen many cases where a group of rock throwers or zealots
have decided to visit a group with the sole purpose of 'sticking it' to
some vendor they dislike for some reason.

> > In my book people who actually write code and deploy code have a 
> > rather bigger say in the typical decision than those who do not. If 
> > someone makes a proposal and the authors of the six major 
> > implementations and all the ISPs in the room vote against 
> it then in 
> > my view the proposal isn't happening regardless of what the 
> > 'consensus' might be.
> 
> And in all my IETF experience I've never seen anyone declare 
> consensus 
> when the authors of six major implementations and all of the ISPs in 
> the room were opposed to it.  

We have both seen cases where the chair has refuse to acknowledge the
need to change a spec in order to support real world constraints that
affect key stakeholders.

IPSEC should have supported NAT from day one. IPSEC makes a dreadful,
lousy VPN protocol because it does not support NAT properly. But it was
only after several years of complaints and a revolt by the vendors that
anything was done about this.

I remember very clearly Steve Bellovin saying to the IPSEC WG that the
IPSEC proposal as then defined would not support NAT well but that some
folk would consider that to be desirable.

Five years ago you were one of the people who argued that it would be
better not to deploy DNSSEC than make the changes I was proposing. You
were successful in sticking it to a vendor you dislike. And the Internet
is the worse as a result. If my advice had been taken then DNSSEC would
be deployed now and people would be adopting it to avoid the DNS based
phishing attacks that are taking place.

> When chairs do abuse their positions, there's a process for 
> that.  I do think that we have too many cases where chairs 
> are also document authors so that there is an almost inherent 
> conflict of interest.

The process is to complain to the unelected AD who may not be too happy
challenging a WG chair who is also an AD and whose support he might
need.

> > Why can't we elect the WG chairs? Why can't we elect the ADs?
> 
> Say for the purpose of argument you're running a business, or 
> maybe a large non-profit organization.  Would you let your 
> employees elect 
> their managers?  Do you think that would be a good way of 
> choosing people who would carry out the organization's 
> strategic goals?

I don't consider the ADs managers. 

In OASIS, Apache, most FOSS projects we elect the oversight boards, in
W3C we elect most of the oversight board and it is pretty clear that any
director that succeeds TBL is almost certain to lose the appointment
privs that the director currently has.

The IETF is the only institution I have ever encountered that has such a
deficit of accountability. Even the US universities have moved to having
student representatives on most boards.

> The last thing that iETF needs is to let organizations buy votes by 
> having their employees become members or sending them to meetings.  

There are about 1000 people who qualify for Nomcon. To influence a vote
a company would need at least 300 people and plan their approach a year
ahead. The cost of each vote is 2 * ($600 + airfare + 1 weeks pay +
overhead). That's at least $5000 per vote if they send secretaries
along, $10K for an engineer. So to have an effect it would cost about $3
million.

I don't think that vote packing would work here. The only time I have
seen it in other standards orgs is when patents have been involved and
someone is trying to get a patent embedded in a standard. I see that as
a function of weak IP controls.

> I could see a place for voting in some things - like choosing 
> people who decide or oversee how the organization's money is 
> spent, and 
> maybe in choosing ombudsmen who would help in dispute 
> resolution. I don't think voting has a place in IETF's 
> technical decision-making.

I don't think that you vote over whether the earth is round or flat.

But it is useful sometimes to have a vote to say that it is not worth
discussing the round/flat issue any further.

My problem with WG processes is where you have the flat earth faction
coming back to the table again and again to argue the case long after
the time has passed.

If a group of loosers get together in a room they will produce lossage.
I don't worry about that because lossage has a short lifespan. Even in
the security area the evidence of SSL 2.0 and WEP demonstrate
conclusively that it is much better to deploy a broken spec than wait
for perfection. Mistakes are bad but if the D-Team do manage to get
something out into the market and used the A-Team will be along soon
enough to fix it.

> If you had tried to identify the parties whose buyin were 
> needed to deploy IPsec and IPv6 you'd have gotten it wrong.

My first meeting might not have the right people at the table but my
third always has.

> In both of those cases a big part of the reason for the delay 
> is that the people developing the protocols didn't understand 
> their target market. (this may still be the case to a large 
> degree)  And it's not because the presumed-to-be major 
> players  weren't involved, it's because the participants 
> didn't fully understand and anticipate how the Internet was 
> changing.  Voting wouldn't solve that problem.

The major players are not the vendors, they are the backbone carriers
and the ISPs.

> > The way the system works in OASIS is that there is a con call every 
> > week or two weeks and members of the group have to attend the con 
> > calls to maintain voting rights.
> 
> That's a really lousy way to ensure broad input and a really 
> good way to make sure that the group works in isolation and 
> produces irrelevant output.

SAML was produced in less than a year and has been considerably more
successful than most IETF specs produced in that period.

Actually the process has the reverse effect. People attend WG meetings
to keep their voting rights. Instead of four people carrying the WG
there are 40 or more.

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Keith Moore <[email-masked]>
Date: Fri, 15 Apr 2005 14:16:16 -0400

> Technical soundness can be dealt with in Darwinian terms.

That is wishful thinking on your part, not reality.

> Design by committee tends to wreck as much as it improves.

You want design by individuals or small committes with input and
feedback from larger communities.  

> because most are self correcting, people do not usually implement
> broken specs. 

No, my concern is producing good designs, which is not the same thing
as an absence of mistakes.  Mistakes in good designs can be corrected.
Mistakes in poor designs are not self-correcting because the design
is too broken to correct them and if the design gets widely deployed,
there's too much investment in the design to start over.

> If the standards process is responsive you can discover the

The standards process that we have is fairly good at fixing small
mistakes, not so good at fixing design flaws.
 
> If I want folk with varied concerns I talk to users, not engineers.

You want to talk to both.  Experienced engineers understand what it
takes to make a design successful and if you have engineers with 
a variety of experience it's more likely to be successful in a 
variety of scenarios.  Users have a different perspective which is
also valuable - provided, of course, that there are already users
of something resembling whatever you're trying to create.

> engineering forum would be a sensible place to achieve that?

IETF is not an elite engineering forum.  It's a self-selecting
engineering forum.  We have some good people here (still) but
we're not nearly as good at attracting especially talented 
engineers as we need to be.

Of course the reason you want good engineers is so you can draw
on their experience.

> > IETF's imprimatur does have an effect, in the sense that if there's
> > a  perceived need for a solution to some problem an IETF 
> > document is more likely to be seen as _the_ solution than a
> > competing solution from, say, a vendor.  
> 

That's a circular argument, and it's also incorrect. IETF's imprimatur 
affects early deployment as least as much as late deployment.  Once 
there's critical mass behind some protocol, it's very difficult for
IETF or anyone else to change the direction the market is going -
even given tremendous technical advantages for the new direction.
IETF can try to adopt or revise that protocol but it can't always
fix inherent flaws in that protocol.

> The market already understands the open standards issue to the point
> of obsession. But the issue is really one of change control. The first

Yup, and that's the right order.   But in the absence of critical mass
behind a single solution, of several competing solutions that claim to 
work, one that is a standard will typically be favored, and for sound 
reasons.

> should take the IESG what, five seconds to agree that HTTP is a

HTTP is a draft standard.  In practice that seems to be good enough
for almost everyone.  Hardly anyone inside or outside of IETF is 
really concerned that HTTP isn't "full" standard - in practice, that's 
just an ego thing.  And the process that got HTTP to draft standard
was valuable - as messy as HTTP/1.1 is, it isn't nearly as bad
as what "the market" was doing to HTTP without review.  The reason 
HTTP isn't full standard is that the perceived gain isn't worth the 
perceived effort. Are there significant improvements that need to 
be made to the specification (that wouldn't require a recycle at 
proposed or draft)?  Will making HTTP a full standard increase 
the market acceptance? No and no.  As I said, it's just an ego thing.

> imprimataur has a hope of displacing SPF/Sender-ID with no

It's a mistake to view this as an either-or question.  The thing to
do is to allow multiple schemes to coexist.  Layered defense.
The market might try to deploy either or both  of these, but neither
one is going to be very useful by itself in its current form (or 
I should say, as of the last time I reviewed them, which was 
a few months ago).

> I don't have any problem with either group you mention. The problem in
> a WG is when you get visited by rock throwers or zealots who have a
> completely different agenda, or some other group that wants to get
> their standard deployed by making it a precondition for a spec that
> has a lot of public momentum.

Yep, those are problems.  Dealing with them is an art.  Voting will not 
help.

> We still have WGs being told that they have to support BEEP, despite
> the fact that it is inadequately engineered as a web services
> transport, has been decisively rejected by the market as a web
> services transport and is based on obsolete technology. SACRED was
> bounced into making BEEP their transport despite the fact that the
> spec only makes sense as an adjunct to SAML and XKMS which are SOAP
> based. The RID spec was made worse by the insistence on support for
> BEEP.

Not being familiar with the particulars of such mandates, I can't
comment on them.  But those are process issues, and we have ways of
dealing with them.

Also, I'm not very impressed with SOAP, and only slightly more impressed
with BEEP.  But then again, I think XML in general is highly overrated. 
Useful yes, but overrated.

> have decided to visit a group with the sole purpose of 'sticking it'
> to some vendor they dislike for some reason.

I haven't seen cases like that.  I've actually seen a lot more cases
of single-vendor coercion.  Everybody's experience is different, I
suppose.

> need to change a spec in order to support real world constraints that

I've also seen cases where the chairs refuse to acknowledge technical 
realities because the self-appointed stakeholders were applying 
pressure.  Chairs make mistakes.  And the last thing we need is a
structure where self-appointed stakeholders can impose mandates.

One person's stakeholder is another person's oppressive monopoly.

> IPSEC should have supported NAT from day one.  

If memory serves, IPsec (or at least the idea of IPsec) predated NAT
by many years.  And as far as I can tell, almost everything IETF has
done to support NAT has been harmful.  I do think that IPsec got 
stuck in the mode of thinking of "IP address = host identity" and
that in hindsight this was a bad idea for lots of reasons which have
nothing to do with NAT.  But it was how nearly everyone thought back 
then.

> IPSEC proposal as then defined would not support NAT well but that
> some folk would consider that to be desirable.

It _was_ desirable to suppress NAT.  NAT is a disaster and support
for NAT was correctly seen as shortsighted.  IETF's mistake,
in hindsight, was not making transition issues paramount in the
design of IPv6.  In hindsight, IPAE looks a lot better now than it 
did at the time.

> were successful in sticking it to a vendor you dislike. 

I would have had the same objections no matter who made the proposal,
and no matter what vendor was running the root or major TLDs.  If it
helps your ego to think this was a personal attack against you or your
employer, fine, you are welcome to your delusion.  But to make that
statement in a public forum is, if I'm not mistaken, slander.

> And the Internet
> is the worse as a result. If my advice had been taken then DNSSEC
> would be deployed now and people would be adopting it to avoid the DNS
> based phishing attacks that are taking place.

Sorry, I don't buy into your world view.

> The process is to complain to the unelected AD who may not be too
> happy challenging a WG chair who is also an AD and whose support he
> might need.

That's right.  But if you don't get relief you go to the IAB.
 
> > employees elect their managers?  Do you think that would be a good
> > way of  choosing people who would carry out the organization's 
> > strategic goals?
> 

Then you don't understand what ADs do.

> The IETF is the only institution I have ever encountered that has such
> a deficit of accountability.

You seem to think that the IETF isn't accountable because they make 
decisions that you don't like.   I assure you from having been there
that there is significant pushback against IESG people who do things
that WG participants don't like.  Appeals are painful enough for 
everyone on IESG that even the implied threat of appeal is a
powerful incentive to craft a compromise.

> There are about 1000 people who qualify for Nomcon. To influence a
> vote a company would need at least 300 people

Nope, because nowhere nearly that many people would vote even in
IETF-wide elections.  Or if they did, their votes would be meaningless.
there are far fewer than 1000 people who actually would know who
they were voting for or why they should pick one candidate over
another.  Under those conditions, it's easy to manipulate
an election, and you're not likely to get good selections even if
the election isn't manipulated.

>  and plan their approach
> a year ahead. The cost of each vote is 2 * ($600 + airfare + 1 weeks
> pay + overhead). That's at least $5000 per vote if they send
> secretaries along, $10K for an engineer. So to have an effect it would
> cost about $3 million.

And there are a few companies that would pay a significant
fraction of that if they could dictate the outcome.
 
> > spent, and maybe in choosing ombudsmen who would help in dispute 
> > resolution. I don't think voting has a place in IETF's technical
> > decision-making.
> 

This is one of those "reality" arguments again, isn't it?  Meaning
that you get to claim that your version of "reality" dictates some
poor technical decision?  I've got news for you - "reality" is
neither objective nor immalleable.
 
> If a group of loosers get together in a room they will produce
> lossage. I don't worry about that because lossage has a short
> lifespan. Even in the security area the evidence of SSL 2.0 and WEP
> demonstrate conclusively that it is much better to deploy a broken
> spec than wait for perfection. 

It's even better to deploy a good spec, one which is well designed
and adaptable without disruption.  Of course, part of the problem with
trying to make TLS sane was trying to be backward compatible with
the broken spec that was already out there and which didn't have
a reasonable upgrade path.   I'm not saying that SSL should have 
waited until it was standardized before people started deploying it,
I'm saying that standards can't always fix things that are broken
once those things are widely deployed.

> The major players are not the vendors, they are the backbone carriers

That's a very narrow view.

> > > The way the system works in OASIS is that there is a con call
> > > every  week or two weeks and members of the group have to attend
> > > the con  calls to maintain voting rights.
> > 

I suspect your notion of success doesn't coincide with mine.  At any 
rate it's meaningless to compare success of one standard with another
unless those standards more-or-less serve the same purpose.

> to keep their voting rights. Instead of four people carrying the WG

And it's a really good way to screw those who will be adversely 
affected by the WG's output but who can't afford to attend every 
meeting of very WG that will affect them.  

Keith

---

From: Sam Hartman <[email-masked]>
Date: Fri, 15 Apr 2005 17:40:31 -0400

>>>>> "Keith" == Keith Moore <[email-masked]> writes:

    Keith> You seem to think that the IETF isn't accountable because
    Keith> they make decisions that you don't like.  I assure you from
    Keith> having been there that there is significant pushback
    Keith> against IESG people who do things that WG participants
    Keith> don't like.  Appeals are painful enough for everyone on
    Keith> IESG that even the implied threat of appeal is a powerful
    Keith> incentive to craft a compromise.

I'd like to second this.  There are several accountability mechanisms.
ADs get significant pressure when they make a decision that people
don't like from the people effected, from the working group chairs,
from the rest of the IESG and in some cases from the IAB.

I also believe the nomcom process does provide accountability.  I
think that the nomcom interview process was more comprehensive than
any job interview process I've gone through.

I suspect the nomcom does get a lot of feedback about sitting IESG/IAB
members and it seems likely that on average they seriously consider
this feedback.

--Sam