Subject: Re: on the value of "running code" (was Re: Do you want to have more meetings outside US ?)

From: [email-masked] (Jun-ichiro itojun Hagino)
Date: Wed, 1 Aug 2007 13:56:08 +0900 (JST)

> IMHO, "running code" gets more credit than is warranted.  While it is
> certainly useful as both proof of concept and proof of implementability,
> mere existence of running code says nothing about the quality of the
> design, its security, scalability, breadth of applicability, and so
> forth.  "running code" was perhaps sufficient in ARPAnet days when there
> were only a few hundred hosts and a few thousand users of the network. 
> It's not sufficient for global mission critical infrastructure.

	tend to agree.  how about "multiple interoperable implementations"?

itojun

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Keith Moore <[email-masked]>
Date: Wed, 01 Aug 2007 02:34:17 -0400

> 	tend to agree.  how about "multiple interoperable implementations"?
that's certainly better than one implementation, especially if
implemented on multiple platforms.  though still, I think, this is not
sufficient in general.

again, I'm biased because I've heard too many arguments of the form "we
have running code for <deficient protocol>, and it's already (somewhat)
deployed so we have to approve it as a standard without changing it".

Keith

---

From: Douglas Otis <[email-masked]>
Date: Wed, 01 Aug 2007 01:20:39 -0700

On Tue, 2007-07-31 at 17:24 -0400, Keith Moore wrote:
> certainly useful as both proof of concept and proof of
> implementability, mere existence of running code says nothing about
> the quality of the design, its security, scalability, breadth of
> applicability, and so forth.  "running code" was perhaps sufficient in
> ARPAnet days when there were only a few hundred hosts and a few
> thousand users of the network. It's not sufficient for global mission
> critical infrastructure.

A simple axiom "Do not execute scripts from strangers" is often
violated.  The Noscript plugin for Firefox represents an excellent (and
highly recommended) example of this principle in action.  Unfortunately,
a mouse-click is often not required for a computer to become 0wned. : (

When coping with spam, security issues related to DNS are often ignored.
Concerns raised in the draft-otis-spf-dos-exploit were dismissed by
suggesting list of bogus NS records are not limited to the same extent
anyway.  Many libraries implementing SPF do not impose limits on the MX
record, or the number of NXDOMAIN, suggested as fixes in the OpenSPF
group's rebuttal.

http://www.openspf.org/draft-otis-spf-dos-exploit_Analysis

Ironically, the rebuttal points out a bogus NS record method that
worsens a DDoS barrage that can be caused by SPF.  SPF remains a
significant risk, even when limited to just 10 SPF record transactions
per email-address evaluated.  With local-part macro expansion, these DNS
transactions represent a gift of a recipient's resources given at no
cost to the spammer.  DDoS attacks made absolutely free and unstoppable!

Offering a method to macro expand elements of the email-address
local-part, when used in a spam campaign, allows a _single_ cached SPF
record to cause an _unlimited_ number of DNS transactions from any
remote DNS resolver servicing SPF libraries.  Uncached targeted DDoS
attacks are not tracked by email logs and can not be mitigated.  The
gain of this attack can be highly destructive, while remaining virtually
free to spammers wishing to also stage the attack.

In addition to offering a means for staging a DDoS attack on
authoritative servers, unfettered access afforded to remote recursive
DNS servers by SPF scripts permits brute force DNS poisoning.  Even
knowing whether SPF related exploits are ongoing is not easily
discerned.  With the current state of affairs related to web browsers,
the axiom "Do not execute scripts from strangers" is a concept that
should be seriously taken to heart.

-Doug

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 01 Aug 2007 06:04:14 -0600

Keith Moore wrote:

>>The danger here is that when people bring work to IETF, they might
>>refuse to change protocols which are already deployed.
>>    
>>
>This already happens to far too great a degree.  People keep arguing
>that because they have running/deployed code, IETF has to standardize
>exactly what they have already produced.  In many cases things that are
>deployed before they get widespread design review are very poorly designed.
>  
>
Indeed. And I have to admit that I have been in situations like this 
myself. There were several cases when I was reluctant to upgrade my code 
to the latest draft when I've implemented a previous version.

>>>I think we've seen several examples of where the IETF has spent
>>>significant amount of energy, ranging from heated discussions to
>>>specification work, on solutions that simply won't fly.  It would be
>>>useful if that energy waste could be reduced.  Having 'running code' as
>>>a barrier for serious consideration within the IETF may be one approach.
>>>      
>>>
>>I agree that running code should be given extra weight, but I am not
>>sure that running code should be a requirement for something which is
>>not well understood yet (some Lemonade WG documents come to mind).
>>    
>>
Yes. I found that implementing a spec before WG/IETF LC pretty much 
always improves the spec.

So although I see many benefits in running code, I don't think it should 
be given more weight that it deserves.

>forth.
Agree.

>"running code" was perhaps sufficient in ARPAnet days when there

---

From: "Steve Silverman" <[email-masked]>
Date: Wed, 1 Aug 2007 10:37:10 -0400

If we simply charged a high fee for the I-Ds and a very high meeting fee
($10k) we wouldn't have all these documents to read.  The attendance would
drop so the meeting could be held in a small inexpensive room.  The IETF
could also charge for the email lists, cutting back on all the messages we
have to read.  This would also minimize those pesky new RFCs that make
things more complicated for the equipment vendors.  

Alternatively the IETF could NOT use the meeting fees to subsidize the other
IETF activities.  They would need to find another funding source.  
Perhaps they could charge for address space? 

Another option would be to charge several dollars per cookie.  I estimate
this would generate enough income to solve all financial problems and
possibly extend the lives of the attendees.  

I can't believe this thread has continued this long. 

Steve Silverman

-----Original Message-----
From: Suresh Krishnan [mailto:[email-masked]] 
Sent: Tuesday, July 31, 2007 3:05 PM
To: [email-masked]
Subject: Funding (was Re: Charging I-Ds)

Charging for IDs will kill innovation. I use IDs to float ideas which 
may or may not bear fruition. I would not work on these if I had to pay. 
  I also work on things at the IETF than my employer does not sponsor. 
These things will get thrown out as well.

Since we have started slaughtering the sacred cows (free {ids,mailing 
lists, rfcs ...}), I might as well suggest few more.

* Make remote participants share the costs. i.e. charge for live 
audio/video feeds. I believe this is fairer than charging for ids, rfcs 
or mailing lists. People who want to participate, but are unable to 
travel can get these for a lower meeting fee (25% maybe).

* Get some kind of corporate sponsorships for supporting free documents 
(kind of like IEEE get 802)

* Cut back on the food and beverages

I would be unhappy with some of these things, but at least they still 
retain the openness of the IETF and are reasonably fair.

Cheers
Suresh

---

From: Suresh Krishnan <[email-masked]>
Date: Wed, 01 Aug 2007 10:41:08 -0400

Hi Itojun,
   How would you write documents which warn against people doing funny 
things? I wrote a draft about the issues with hop-by-hop options in IPv6 
and cautioning against their use. I see that there are still proposals 
coming out which depend on new hbh options? What should I do instead of 
writing a draft?

Cheers
Suresh

Jun-ichiro itojun Hagino wrote:
> 	I assume i-d to be a proposal for a new protocol, which is
> 	implementable with a reasonable efforts and costs.  i think your
> 	view and my view are opposite.
> 
> 	i'd like to see the following:
> 	- submission of i-d requires an implementation
> 	- to become a RFC requires two independent interoperable implementation
> 
> itojun
>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 1 Aug 2007 16:49:46 +0200

Suresh Krishnan writes:
>   things? I wrote a draft about the issues with hop-by-hop options in 
>   IPv6 and cautioning against their use. I see that there are still 
>   proposals coming out which depend on new hbh options? What should I 
>   do instead of writing a draft?

Well.... read RFC 1925 and attain zen?

Arnt

---

From: Jun-ichiro itojun Hagino <[email-masked]>
Date: Thu, 02 Aug 2007 00:04:28 +0900

>coming out which depend on new hbh options? What should I do instead of 

	you can implement test tools which would scan and probe vulnerabilities.
	like "dsniff".

itojun
http://www.youtube.com/profile_videos?user=itojun

---

From: Eric Burger <[email-masked]>
Date: Wed, 01 Aug 2007 10:35:10 -0500

My faulty recollection is that in our game of rock-paper-scissors, Running
Code beats Untested Idea, but Well Reviewed Architecture and Protocol beats
Running Code.

> deployed so we have to approve it as a standard without changing it".

Notice:  This email message, together with any attachments, may contain information  of  BEA Systems,  Inc.,  its subsidiaries  and  affiliated entities,  that may be confidential,  proprietary,  copyrighted  and/or legally privileged, and is intended solely for the use of the individual or entity named in this message. If you are not the intended recipient, and have received this message in error, please immediately return this by email and then delete it.

---

From: David Conrad <[email-masked]>
Date: Wed, 1 Aug 2007 13:42:20 -0700

I'd offer that the OSI protocol stack was probably significantly more  
reviewed than the TCP/IP stack.

At the very least, running code is an empirical proof that an  
architecture _can_ work.

Rgds,
-drc

On Aug 1, 2007, at 8:35 AM, Eric Burger wrote:
>>>> implementability,
> Notice:  This email message, together with any attachments, may  
> contain information  of  BEA Systems,  Inc.,  its subsidiaries   
> and  affiliated entities,  that may be confidential,  proprietary,   
> copyrighted  and/or legally privileged, and is intended solely for  
> the use of the individual or entity named in this message. If you  
> are not the intended recipient, and have received this message in  
> error, please immediately return this by email and then delete it.
>

---

From: Cyrus Daboo <[email-masked]>
Date: Tue, 31 Jul 2007 12:07:08 -0400

Hi Peter,

--On July 30, 2007 2:11:38 PM -0600 Peter Saint-Andre <[email-masked]> 
wrote:

> Further, in-person meetings are so second-millennium. How about greater
> use of text chat, voice chat, and video chat for interim meetings? Are
> three in-person meetings a year really necessary if we make use of
> collaborative technologies that have become common in the last 15 years?

All that will do is shift the discussion from "where shall we hold the 
meeting" to "what time/timezone shall we have for our conf call". Given 
that we have people participating from across the globe, trying to arrange 
a time that is acceptable for all participants will be just as hard as 
trying to find a meeting venue acceptable to all.

Unfortunately we don't have scheduling tools that will help resolve issues 
like that (or at least make it easier than a multiple party email 
exchange/negotiation) - but some of us are working on that!

-- 
Cyrus Daboo

---

From: Lixia Zhang <[email-masked]>
Date: Thu, 2 Aug 2007 08:57:04 -0700

>>> ......

forgive me for jumping into the middle of a discussion (and I did not  
know which of the lemonade doc's the above referred to), but my past  
experience seems suggesting that an attempt to implement a "not well  
understood" idea is a good way towards a better understanding of how  
to make the idea work, or what can be potential issues.

> It's not sufficient for global mission critical infrastructure.

it seems to me the above argues that running code is necessary, but  
not sufficient as evidence of a sound design.
(well, that is the interpretation; I have not seen anywhere a claim  
that running code is sufficient, but rather simply to filter out the  
weed)

Lixia

---

From: Andy Bierman <[email-masked]>
Date: Thu, 02 Aug 2007 10:47:33 -0700

Lixia Zhang wrote:
> understood" idea is a good way towards a better understanding of how to 
> make the idea work, or what can be potential issues.
> 

yes!
I tried to resist the 47th rehash of this thread, but... too late...

Within a commercial environment, the organization has to be
fairly convinced that their better mousetrap is going to work,
in order to fund it, productize it, document it, sell it, and support it.

This process will always find more bugs in the mousetrap than
simply documenting it and skipping all the other steps.

If a vendor bothers to do all this, and multiple IETFers can say in a BoF
that they have used the mousetrap and it really does work,
that is worth a whole lot more than "I read the draft and
it looks pretty good".

There is a certain amount of healthy risk that the IESG
can take when chartering new standards-track work.
Prior implementations should not be a gating factor, but
it makes their decision much easier when there is objective
evidence the mousetrap actually works and it is already being
used by the industry.

But implementation and deployment are not enough alone.
There also needs to be some pre-existing consensus that
a standard version could be written and approved by the IETF,
and people are willing to work on it.

The slogan says "rough consensus and running code".
It doesn't say "rough consensus, then running code".
Without running code, there is no deployment.
Without deployment, there is no point to this exercise.

Andy

> it seems to me the above argues that running code is necessary, but not 
> sufficient as evidence of a sound design.
> (well, that is the interpretation; I have not seen anywhere a claim that 
> running code is sufficient, but rather simply to filter out the weed)
> 
> Lixia
>

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 02 Aug 2007 11:59:54 -0600

> to make the idea work, or what can be potential issues.

Yes, but this is only useful once one understands what is actually 
needed in a spec to begin with ;-).
I found running code most useful when the spec is nearly ready for 
publication.

> not sufficient as evidence of a sound design.

Agreed. Running code is useful to identify things that are difficult to 
implement or unclear.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 02 Aug 2007 11:22:31 -0700

> reviewed than the TCP/IP stack.

Depends what you mean by "more reviewed".

More eyes looking at the specs?  Probably yes.  More critical analysis by 
senior technical architects?  Probably not.

 > At the very least, running code is an empirical proof that an

Again, depends upon what one means by running code.

Certainly there were early prototypes of OSI modules, and even running 
products.  Clearly, that was not enough.  In contrast, the Internet code was 
deployed and used in a running service, with increasing scale.  So the 
distinction between prototype and production is probably of fundamental 
importance.  (I think that Dave Clark really meant "running service" when he 
said "running code".)

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Keith Moore <[email-masked]>
Date: Thu, 02 Aug 2007 15:01:49 -0400

> that is worth a whole lot more than "I read the draft and
yes.  but then again, vendors are insensitive to certain kinds of bugs. 
the myriad bugs produced by introduction of NAT are good examples.  a
little bit of analysis should have convinced any responsible vendor to
either not sell NAT products, or to be honest in marketing them and to
accompany them with rather strong disclaimers.

(not to attack NATs specifically, they're just the most obvious of many
examples and the easiest ones to cite)
> evidence the mousetrap actually works and it is already being
again, being used by the industry is no indicator of soundness.  and
being used by the industry in the absence of public protocol review is
highly correlated with poor design.

Keith

---

From: "Tom.Petch" <[email-masked]>
Date: Fri, 3 Aug 2007 18:33:53 +0200

----- Original Message -----
From: "Dave Crocker" <[email-masked]>
To: "David Conrad" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, August 02, 2007 8:22 PM
Subject: Re: on the value of "running code" (was Re: Do you want to have more
meetings outside US ?)

> importance.  (I think that Dave Clark really meant "running service" when he
OSI got well beyond the prototype stage.  Major manufacturers produced products
and I was involved with their implementation.  From c.1990 to c.1995 we all knew
that, with such a weight of political pressure behind it, OSI was bound to sweep
all before it.  By 1995, the tide had turned, but it was not the lack of
interoperable, production software that did the turning.

Tom Petch

---

From: Dave Crocker <[email-masked]>
Date: Fri, 03 Aug 2007 11:22:54 -0700

Tom.Petch wrote:
>> products. 
...
> and I was involved with their implementation.  

So did minor manufacturers.  We (Wollongong) developed and sold a full OSI 
stack, from clnp up through x.400.  that's why I said "running products". At 
base, however, such products were purchased more as a checklist item than for 
real use.

My point was about the failure to make sure there was large-scale, 
multi-vendor, in-the-wild *service*.  Anything that constraint what can go 
wrong will limit the ability to make the technology robust and usable.

It is the focus on pragmatic steps that make the technology usable that I 
believe Clark was referring to, by saying "running code".

d/
-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Fri, 03 Aug 2007 11:24:07 -0700

...

So did minor manufacturers.  We (Wollongong) developed and sold a full OSI
stack, from clnp up through x.400.  That's why I said "running products". At
base, however, such products were purchased more as a checklist item than for
real use.

d/
-- 

   Dave Crocker

---

From: Douglas Otis <[email-masked]>
Date: Fri, 3 Aug 2007 14:12:49 -0700

On Aug 3, 2007, at 11:24 AM, Dave Crocker wrote:
> multi-vendor, in-the-wild *service*.  Anything that constraint [in]  
> what can go

There are currently millions of unconstrained large-scale, in-the- 
wild services being manipulated and controlled by criminals.   
Constraints that must be taken seriously are related to the economies  
limiting the staging of DDoS attacks.  Criminals often utilize tens  
of thousands of 0wned systems.  These systems often send large email  
campaigns.  Any scheme that expects receipt of a message to invoke a  
process that initiates additional traffic must be carefully considered.

Expecting recipients to employ the local-part of a purported  
originator's email-address to then construct dozens or even hundreds  
of DNS transactions wholly unrelated to the actual source of the  
message is a prime example of how economies related to DDoS attacks  
are being gravely shifted in the wrong direction.

Spammer are already spamming, either directly or through an ISP's  
outbound server.  Lacking a reasonable constraint on the recipient  
process, criminals can attack without expending their resources and  
not expose the location of their systems.  Using SPF/Sender-ID as an  
example, just one DNS resource record is able to source an attack  
comprised of millions of recipient generated DNS transactions.  The  
lack of receipt process constraint in this case is an example of  
either negligence or incompetence.  Here an attack may employ several  
levels of indirection and yet nothing germane to the attack will be  
found within email logs.

Not imposing reasonable constraints does not make the Internet either  
more robust or usable.

-Doug