Subject: RE: accusations of cluelessness

From: "Christian Huitema" <[email-masked]>
Date: Sat, 11 Oct 2003 22:58:55 -0700

> > It is perfectly fine to review a specification, understand the
intent of
> > the original designer, and suggest ways to better achieve the same
> > result. That is exactly what working groups are supposed to do. It
is
> > also perfectly fine, if the original designer won't change their
design,
> > to publish an alternative design that hopefully works better, and
then
> > rely on market forces to sort it out. But it is not fine to try to
> > prevent the original designer from actually shipping products,
either by
> > preventing publication of the specification or by trying to prevent
> > deployment.
> 
> On the contrary, it is our duty to do all of these things.
> 
> What is not fine is for participants to expect IETF to lend its
support to
> bad designs.

Well, who made us kings? It is one thing to work and publish designs
that hopefully will be good. It is quite another to judge someone else's
design and brand it bad. It is far better to let the market be judge.

-- Christian Huitema

---

From: [email-masked]
Date: Sun, 12 Oct 2003 03:24:37 -0400

On Sat, 11 Oct 2003 22:58:55 PDT, Christian Huitema said:
> design and brand it bad. 

On the other hand, the same skills that allow us to evaluate our own
designs as workable should also be able to evaluate other's designs for
workability.  And if we find fatal flaws in a design, it is our responsibility
to say so, rather than let it be deployed in the real world.

This applies whether we are civil engineers, aeronautic designers, or
software creators.

>                    It is far better to let the market be judge.

If letting the market be judge is a good idea, why is such a large percentage
of my bandwidth chewed up by emails bearing viruses?

---

From: Leif Johansson <[email-masked]>
Date: Sun, 12 Oct 2003 11:24:32 +0200

|
| Well, who made us kings? It is one thing to work and publish designs
| that hopefully will be good. It is quite another to judge someone else's
| design and brand it bad. It is far better to let the market be judge.
|

I agree. But should we not as a community try to produce results
which attempt to be consistent? If a feature in the network layer
break assumptions in the application layer then (setting aside the
discussion of whose fault the inconsistency is) if we want to be
consistent one has to give. I put it to you that the difference
between 'keeping' and 'removing' in this case is just in our heads.

Eg. keeping SL in the original form (I agree this is a hypothetical
situation) would have implied removing features from several protocols.
Did we remove SL or did we keep those protocols intact?

	Cheers Leif

---

From: grenville armitage <[email-masked]>
Date: Sun, 12 Oct 2003 20:58:12 +1000

Christian Huitema wrote:
	[..]
> Well, who made us kings?

a track record of being decent engineers. which is why most of
us are ignored when we offer legal advice, and some of us are
acknowledged when we offer clues on networking. (true, others of
us are merely princes, pages or court jesters, but still...)

> It is one thing to work and publish designs
> that hopefully will be good.

that's a good way to be crowned a king, yes.

> It is quite another to judge someone else's
> design and brand it bad.

it is called critique. even vendors go through internal processes
designed to weed out bad product ideas, poorly thought out features,
nasty implementation trade-offs....

(academics have a similar, imperfect process called peer review.
nothing new here.)

> It is far better to let the market be judge.

who blessed "the market" with engineering insight?

the market has sight. market analysts have hind-sight.
engineering is about fore-sight. therein lies a world of
difference in roles and responsibilities.

cheers,
gja
-- 
Grenville Armitage
http://caia.swin.edu.au
I come from a LAN downunder.

---

From: Keith Moore <[email-masked]>
Date: Sun, 12 Oct 2003 07:42:12 -0400

> design and brand it bad. It is far better to let the market be judge.

You're not insisting that the market be judge; you're insisting that
IETF lend its support to bad designs in order to help mislead the market.

Also, you're simply wrong that it's better to let the market judge things -
that is, if you want things to actually work.  

When vendors insist on a level playing field - what they mean is that they
want things slanted in a way that gives them an advantage.

---

From: Mark Seery <[email-masked]>
Date: Sun, 12 Oct 2003 08:02:09 -0700 (PDT)

--- grenville armitage <[email-masked]> wrote:
<snip>
> difference in roles and responsibilities.

The fore-sight of any role is constrained by assumptions. An engineer who
designs a building to withstand a collision with a 707 suddenly has a lot more
hindsight (than fore-sight) when it is hit with a 767. So let's acknowledge
there is no such thing as perfect fore-sight for any role, only better or worse
given a set of assumptions.

The ethos of running code is all about establishing a proof that something
works. Challenging design assumptions is worthwhile, especially if you can see
the 767 coming, but this institution appears to give weight to running "code".
So in the context of running code, the market can be the judge.

--- [email-masked] wrote:
<snip>
> If letting the market be judge is a good idea, why is such a large percentage
of my bandwidth chewed up by emails bearing viruses? <

A large percentage of your bandwidth is chewed up because there is no market -
no cost associated with sending an email. A large percentage of those emails
have viruses because a) viruses are to cyber-assets what WMD are to physical
assets - i.e. widespread damage from one device b) the crimial justice system
has not kept pace with technological change and c) there is resitance to
allowing the crimial justice system to keep pace (which may or may not be good
thing depending on your view). Put another way, if the criminal justice system
had the same level of effectiveness at protecting physical assets, I wonder
whether civilization as we know it would exist.

---

From: [email-masked]
Date: Sun, 12 Oct 2003 11:43:53 -0400

> whether civilization as we know it would exist.

If you want to take it in that direction, we have a case where "the market" decided
that locks weren't needed on doors in a high-crime area of town.  OK, so maybe
the crime rate was still low when the door was installed - there's been a lot of
delay and negligence in installing said locks.

---

From: mark seery <[email-masked]>
Date: Sun, 12 Oct 2003 08:57:54 -0700

That is a fair point, but I would observe that there are plenty of more 
cases where locks were not sufficient.

If there are no tradeoffs to actions/feedback loops, then agents in a 
system can not make optimization/fitness decisions and evolution does 
not occur.

---

From: Keith Moore <[email-masked]>
Date: Sun, 12 Oct 2003 12:31:12 -0400

> designs a building to withstand a collision with a 707 suddenly has a lot
> more hindsight (than fore-sight) when it is hit with a 767. So let's
> acknowledge there is no such thing as perfect fore-sight for any role, only
> better or worse given a set of assumptions.

there is no perfect foresight.  there is no perfect hindsight either.
nor is the market either efficient or reliable at choosing good solutions.

but we are probably better off using some combination of foresight,
hindsight, and market forces than by excluding any of these or 
relying on any of these exclusively.

---

From: mark seery <[email-masked]>
Date: Sun, 12 Oct 2003 18:35:59 -0700

>nor is the market either efficient or reliable at choosing good solutions.
It appears the word "market" is overloaded with lots of social/political 
ideology so rather than go down that road, let me redirect the point by 
simply observing that running code allows the designer(s) to get 
feedback about a design and its implementation implications, which is 
the point about "letting the market decide". Many can imagine a "better" 
design for some problem space, according to some fitness bias we have, 
but imagining a "better" design doesn't help unless it can be run and 
produce results, against the fitness bias the consumers of a solution have.

There is nothing wrong with basing an implementation on previous 
experience and foresight, but there are many things which are more 
complex than what we can readily understand and therefore our foresight 
is limited - our implementation experience helps here. This is the age 
old debate about waterfall vs iteration - probably another dead end 
discussion point.

>relying on any of these exclusively.
ack. so as usual, there are no absolutes and a fair amount of subjective 
judgement, which I guess is the reason why discussion occurs.

---

From: Keith Moore <[email-masked]>
Date: Sun, 12 Oct 2003 22:01:37 -0400

> the point about "letting the market decide". 

Within IETF, "running code" has usually meant interoperability tests of
multiple implementations of a single protocol.  These days we find we need
to be more concerned about the interaction between protocols than we used to
be.  The notion that protocols can be examined in isolation no longer holds. 
So for instance: we expect non-TCP-based protocols to share bandwidth fairly
with TCP-based protocols; and we expect all protocols to be reasonably secure
because when even a single protocol is compromised it can disrupt operation of
the network, other protocols, and hosts that weren't directly compromised.

The "market" is not in a good position to evaluate these characteristics
because many of these effects do not become visible to the "market" until
the "market" has already substantially invested in a particular path.
Indeed, the very discipline of "engineering" exists to minimize such
catastrophes. If you make design decisions based on guesswork, that's
guesswork; if you make design decisions based on analysis of how well a
solution meets well-defined criteria, that's engineering.

These days, a lot of people seem to be arguing that IETF shouldn't do
engineering.

---

From: mark seery <[email-masked]>
Date: Sun, 12 Oct 2003 19:25:25 -0700

>These days, a lot of people seem to be arguing that IETF shouldn't do
Hopefully not - not I for sure, I mean that would require a name change 
;-) What I think exists is a difference of opinion about what the 
definition of engineering is. Another well-known institution seems also 
to be stuggling with this same question:

"....*where numbers count for everything and hunches are scorned...."

*http://www.cnn.com/2003/TECH/space/10/12/nasa.reformers.ap/index.html

---

From: Keith Moore <[email-masked]>
Date: Sun, 12 Oct 2003 22:55:38 -0400

> What I think exists is a difference of opinion about what the 
> definition of engineering is. 

another way to put it is - after ~17 years of IETF, we need to start
defining what "Internet Engineering" means.

> Another well-known institution seems also to be stuggling with this same
> question:
> 

ignoring hunches didn't cause the demise of Challenger and Columbia.
ignoring hard data did.  of course, paying attention to intuition might
be useful in those inevitable (hopefully rare) cases where you do ignore
hard data.

---

From: "Al Arsenault" <[email-masked]>
Date: Mon, 13 Oct 2003 08:57:57 -0400

I have a feeling I'm gonna regret getting involved in this rat-hole, but
here goes anyway...

I mostly agree with the "let the market decide" philosophy, but there have
to be some limits. Following this strictly leads to pathological cases.  To
point out incidents from the IETF PKIX group over the years that demonstrate
what can happen:

	1 - a gentleman (I'll be polite and call him that) asked PKIX to adopt as a
work item a time-stamping technology he developed.  This gentleman was given
an opportunity (at Adelaide and I believe at one other meeting, as well as
on the list) to present his material to the working group. In the end, PKIX
didn't pick up his work as a work item because it didn't fall under the
charter of the WG. A total of only about 3 people even thought it was
interesting, and they weren't sure it was within the working group's
purview.

	This gentleman has devoted much energy since then to disparaging the IETF,
PKIX, and the PKIX WG chair personally, for this.  His view, stated in
public numerous times, is that the way the IETF should work is that any item
which is properly formatted must be published as an RFC.  That is, any
individual or group can create a document.  If it meets the formatting
requirements, it must be published as an RFC.  It is not the job of the WG,
the IESG, IAB or any other group to approve/disapprove, evaluate, analyze or
otherwise pass judgement on the technical contents of the document.

	Furthermore, all RFCs must be given the same status. It is not appropriate
for the IETF to favor one RFC or the technology contained in it over any
other RFC or technology.

	Any other strategy constitutes restraint of trade, interference in the
market place, and basically a violation of all that a "standards group"
should do.

	Now, fortunately, not too many people agree with this gentleman, but it
does represent a pathological case of "let the market decide".

	2 - the PKIX WG published two competing, non-interoperable protocols for
the same function. One is referred to as CMP; it's in RFC 2510.  The other
is referred to as CMC; it's specified in RFC 2797.  There are a lot of
reasons why we did this; but it boiled down to a schism.  One of the two
largest PKI players at that time preferred CMP, and its allies refused to
yield.  The other of the two largest players preferred CMC, and its allies
refused to yield.  This was because those protocols represented what their
products did, and nobody wanted to change his product.  So, to prevent
bogging down and making no progress at all, the WG decided to progress both
protocols and "let the market decide".  Most people now agree that this was
a mistake.  The strategy of publishing both protocols as "equals" did not
lead to interworking in the Internet; it arguably did the opposite.  It was
also horribly misunderstood by those not in the know; it was believed in
some quarters that one protocol was the interim strategy and the other was
the long-range target. (The supporters of each side did nothing to
discourage this misunderstanding.)

 	While this is not the biggest factor in the failure of PKI to become
ubiquitous, it didn't help.

	My bottom line on this: while I'm not a strong believer in having the
cognoscenti dictate to the unwashed masses the "one true way to do things",
it helps a lot if the Internet Engineering Task Force actually does some
"Internet Engineering".

				Al Arsenault

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
Christian Huitema
Sent: Sunday, October 12, 2003 1:59 AM
To: Keith Moore
Cc: [email-masked]
Subject: RE: accusations of cluelessness

intent of
is
design,
then
either by
support to

-- Christian Huitema

---

From: grenville armitage <[email-masked]>
Date: Tue, 14 Oct 2003 00:18:38 +1000

Mark Seery wrote:
	[..]
> works.

I never said otherwise. "Running code" has been a useful means for reducing the
solution space from which the IETF publishes. It has never been a hard and fast
metric of good design. (Indeed, if anything the IETF's history has shown a number
of protocol designs that were running code and yet still only waypoints towards
better designs in the future.)

But that's really not the point here. The role of an _engineering_ taskforce
is to act like engineers, not a vanity press. Our output should be educated
guidance to the wider community - created with diligence and offered with humility.
We can do no more and should do no less.

(Arguments that the IETF prevents deployment by preventing publication are a
red-herring and should be discarded as such. The "market" has shown a remarkable
affinity, at times, to protocols that have either never been fully published or
published outside the IETF.)

cheers,
gja