Subject: Re: A proposal for a scientific approach to this question [was Re: I'm struggling with 2219 language again]

From: Abdussalam Baryun <[email-masked]>
Date: Sun, 6 Jan 2013 09:41:03 +0100

Hi Marc Petit-Huguenin ,

>I read the responses so far, and what can be said today is that there is 2
philosophies, with supporters in both camps.  The goal of the IETF is to make
the Internet work better, and I do believe that RFC 2119 is one of the
fundamental tool to reach this goal, but having two ways to use it does not
help this goal.

I like the approach, and agree with you that we need a solution in
IETF which still is not solved or ignored by participants fo some
reasons. However, I agree of a survey or an experiment what ever we
call it, that makes IETF reflects to the RFC2119 performance on the
end-user of such product implementation of RFC protocols. I think many
old participants already have good experience to inform us of some
reality of IETF standards' end-user production.

AB

---

From: John Day <[email-masked]>
Date: Sun, 6 Jan 2013 23:49:52 -0500

Strictly speaking, the language of 2119 should be followed wherever 
necessary in order for the text to be normative and make it mandatory 
that a conforming implementation meets some requirement.  Otherwise, 
someone could build an implementation and claim it was correct and 
possibly cause legal problems. However, in the IETF there is also a 
requirement that there be two independent but communicating 
implementations for an RFC to standards-track. Correct?

For all practical purposes, this requirement makes being able to 
communicate with one of the existing implementations the formal and 
normative definition of the RFC.  Any debate over the content of the 
RFC text is resolved by what the implementations do.  It would seem 
to be at the discretion of the authors of the implementations to 
determine whether or not any problems that are raised are bugs or not.

Then it would seem that regardless of whether 2119 is followed, the 
RFCs are merely informative guides.

So while the comments are valid that RFC 2119 should be followed, 
they are also irrelevant.

Given that any natural language description is going to be ambiguous, 
this is probably for the best.

Take care,
John Day

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 07 Jan 2013 10:53:48 +0000

Speaking as both a reviewer and an author, I would like
to ground this thread to some form of reality.

Can anyone point to specific cases where absence or over
use of an RFC2119 key word caused an interoperability failure,
or excessive development time?

- Stewart

---

From: John Day <[email-masked]>
Date: Mon, 7 Jan 2013 07:22:17 -0500

As you are guessing that is unlikely, however, the more pertinent 
question is whether it has prevented some innovative approach to 
implementations.  This would be the more interesting question.

We tend to think of these as state machines and describe them 
accordingly.  There are other approaches which might be prevented if 
using a MUST when it wasn't needed.

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 07 Jan 2013 12:42:43 +0000

Indeed an interesting additional question.

My view is that you MUST NOT use RFC2119 language, unless you MUST use 
it, for exactly that reason. What is important is "on the wire" (a term 
that from experience is very difficult to define) inter-operation, and 
implementers need to be free to achieve that though any means that suits 
them.

- Stewart

On 07/01/2013 12:22, John Day wrote:
> .

-- 
For corporate legal information go to:

http://www.cisco.com/web/about/doing_business/legal/cri/index.html

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 07 Jan 2013 13:31:15 +0000

> implementers need to be free to achieve that though any means that suits

Agreed. Imagine the effect if the TCP standard had said that a particular
congestion control algorithm was mandatory. Oh, wait...

... RFC 1122 section 4.2.2.15 says that a TCP MUST implement reference [TCP:7]
which is Van's SIGCOMM'88 paper. So apparently any TCP that uses a more recent
congestion control algorithm is non-conformant. Oh, wait...

... RFC 2001 is a proposed standard defining congestion control algorithms,
but it doesn't update RFC 1122, and it uses lower-case. Oh, wait...

RFC 2001 is obsoleted by RFC 2581 which obsoleted by RFC 5681. These both
use RFC 2119 keywords, but they still don't update RFC 1122.

This is such a rat's nest that it has a guidebook (RFC 5783, "Congestion
Control in the RFC Series") and of course it's still an open research topic.

Attempting to validate TCP implementations on the basis of conformance
with RFC 2119 keywords would be, well, missing the point.

I know this is an extreme case, but I believe it shows the futility of
trying to be either legalistic or mathematical in this area.

   Brian

---

From: John Day <[email-masked]>
Date: Mon, 7 Jan 2013 09:45:43 -0500

Let me get this straight, Brian.  It would seem you are pointing out 
that the IETF does not have a clear idea of what it is doing?  ;-)  I 
could believe that.

No, your example is not an example of what I suggested at all.

Yours is an example of not specifying the conditions that a 
congestion control algorithm must have rather than the congestion 
control algorithm itself.

What I was suggesting (and it is very easy trap to fall into) was 
defining a spec with one implementation environment in mind and not 
realizing you are constraining things unnecessarily. Consider the 
difference between defining TCP as a state machine with that sort of 
implementation in mind and building an implementation in LISP. (I 
know someone who did it.)  It would be very easy to make assumptions 
about how something was described that made a LISP implementation 
unduly messy, or missed an opportunity for a major simplification.

It is quite easy to do some thing mathematical in this area (not 
necessarily alluding to formal specification), but you do have to 
have a clear concept of the levels of abstraction.  Of course, once 
you do, you still have the question whether there is a higher 
probability of errors in the math or the program.

Yes, programming is just math of a different kind, which of course is 
the point.

Take care,
John

---

From: John Scudder <[email-masked]>
Date: Mon, 7 Jan 2013 15:50:16 +0000

On Jan 6, 2013, at 11:50 PM, "John Day" <[email-masked]> wrote:

> However, in the IETF there is also a requirement that there be two independent but communicating implementations for an RFC to standards-track. Correct?

Alas, no. 

--John

---

From: [email-masked]
Date: Mon, 07 Jan 2013 07:41:55 -0800 (PST)

> trying to be either legalistic or mathematical in this area.

Exactly. Looking for cases where the use/non-use of capitalized terms caused an
interoperability failure is a bit silly, because the use/non-use of such terms
doesn't carry that sort of weight.

What does happen is that implementation and therefore interoperability quality
can suffer when standards emphasize the wrong points of compliance. Things
work, but not as well as they should or could.

A fairly common case of this in application protocols is an emphasis on
low-level limits and restrictions while ignoring higher-level requirements. For
example, our email standards talk a fair bit about so-called "minimim maximums"
that in practice are rarely an issue, all the while failing to specify a
mandatory minimum set of semantics all agents must support. This has led to
lack of interoperable functionality in the long term.

Capitalized terms are both a blessing and a curse in this regard. They make it
easy to point out the really important stuff. But in doing so, they also
make it easy to put the emphasis in the wrong places.

tl;dr: Capitulized terms are a tool, and like any tool they can be misused.

				Ned

---

From: John Day <[email-masked]>
Date: Mon, 7 Jan 2013 11:12:56 -0500

Alas, indeed.  ;-)

At 3:50 PM +0000 1/7/13, John Scudder wrote:
>>  However, in the IETF there is also a requirement that there be two 
>>independent but communicating implementations for an RFC to 
>>standards-track. Correct?
>
>Alas, no.
>
>--John

---

From: Thomas Narten <[email-masked]>
Date: Mon, 07 Jan 2013 16:46:33 -0500

> implementers need to be free to achieve that though any means that suits 

The latter goes without saying. It's one of the "obvious" assumptions
that underlies all IETF protocols. It may not be written down, but its
always been an underlying principle.

E.g., from RFC 1971:

   A host maintains a number of data structures and flags related to
   autoconfiguration. In the following, we present conceptual variables
   and show how they are used to perform autoconfiguration. The specific
   variables are used for demonstration purposes only, and an
   implementation is not required to have them, so long as its external
   behavior is consistent with that described in this document.

Other document (that I've long forgotten) say similar things.   

That sort of language was put into specific documents specifically
because some individuals sometimes would raise the concern that a spec
was trying to "restrict" an implementation.
   
IETF specs have always been about describing external behavior
(i.e. what you can see "on the wire"), and how someone implements
internally to produce the required external behavior is none of the
IETF's business (and never has been).

(Someone earlier on this thread seemed to maybe think the above is not
a given, but it really always has been.)

Thomas

---

From: John Day <[email-masked]>
Date: Mon, 7 Jan 2013 21:01:05 -0500

All standards groups that I am aware of have had the same view.  This 
is not uncommon.

Although, I would point out that the TCP specification nor do most 
protocols specifications of this type follow this rule.  State 
transitions are not visible on the wire.  The rules for sliding 
window are not described entirely in terms of the behavior seen on 
the line, etc.

I have seen specifications that attempted this and the 
implementations built from them were very different and did not come 
close to interoperating or in some cases of even doing the same thing.

In fact, I remember that we thought the new Telnet spec (1973) was a 
paragon of clarity until a new site joined the Net that had not been 
part of the commuity and came up with an implementation that bore no 
relation to what anyone else had done.

This problem is a lot more subtle than you imagine.

Take care,
John Day

---

From: Abdussalam Baryun <[email-masked]>
Date: Tue, 8 Jan 2013 12:53:54 +0100

Hi John,

thank you for your reply (i.e. learned alot), so I understand that a
RFC standard track may have more than one implementation but same
behavior enough not to make an error. Regarding following 2119, I
understand most text follow it only when there are normative actions.
Regarding implementer claiming following a RFC, but the question of
error in process is does the RFC lack communication requirement with
the community?

AB

---

From: Abdussalam Baryun <[email-masked]>
Date: Tue, 8 Jan 2013 19:57:46 +0100

>but the question of
> error in process is; does the RFC lack communication requirement with
> the community?
>

Sorry if not clear. I mean that as some participant are requesting a
scientific approach to struggling with 2119 (i.e. thread-subject),
does that mean in some RFCs the use or not use (i.e. we see that
participant use different approaches to follow the 2119) that it may
add communication confuse with some of community?

AB

---

From: Abdussalam Baryun <[email-masked]>
Date: Tue, 8 Jan 2013 20:01:35 +0100

Why not participant follow one approach to use 2119 in IDs and done,
and if not/another, then please specify in the language section.

AB

---

From: Dean Willis <[email-masked]>
Date: Tue, 8 Jan 2013 17:39:23 -0600

> or excessive development time?

I'm anecdotally familiar with some early pre-RFC 2543 SIP implementations where the implementors ignored everything that didn't say MUST and got something that didn't work. At all. But it was apparently really easy to develop, as the spec only had a few dozen MUST clauses, and the developers didn't include-by-reference any of the cited specs, such as SDP.

When we were trying to decide whether to make RFC 3261 (the replacement of RFC 2543) a "draft standards" instead of a "proposed standard", I recall Robert Sparks and some others attempting to define a "fully interoperable implementation" test that tabulated all of the RFC 2119 invocations that had sprouted in FC 3261. They then immediately gave up the idea as impractical, we recycled at "proposed", and gave up on every making "full". The testing methodology has greatly improved since then, and it makes lithe use of RFC 2119 language for test definition or construction. 

--
Dean

---

From: Dean Willis <[email-masked]>
Date: Tue, 8 Jan 2013 17:54:35 -0600

> add communication confuse with some of community?

I'm absolutely certain that some of our community is confused about something related to this thread. Given the absence of information that would help in a decision, might preference would be just to pick one, and provide a stick for hitting those who do it the other way.

--
Dean

---

From: Abdussalam Baryun <[email-masked]>
Date: Wed, 9 Jan 2013 08:54:21 +0100

>This is what I have been talking about. The human mind's ability to believe that the whole world sees everything the same way they do. It really is quite amazing.

>These so-called gaps often arise because they were unstated assumptions or things that the author believed were patently obvious and didn't need to be stated. Actually didn't know needed to be stated. From his point of view, no one would do it differently. Nothing had been left out and he didn't make the "mistake." What the other guys did was a bug.

That is why I think the IETF names the RFC or standards or
specifications as *Request For Comments*, if there is a gap or the
human mind failed, then it should communicate to the world and report
such gap/bug.

AB

---

From: Abdussalam Baryun <[email-masked]>
Date: Wed, 9 Jan 2013 09:09:28 +0100

>might preference would be just to pick one, and
> provide a stick for hitting those who do it the other way.

I think that IESG is already using that stick :)

AB

On 1/9/13, Dean Willis <[email-masked]> wrote:
> I'm absolutely certain that some of our community is confused about
> something related to this thread. Given the absence of information that
> would help in a decision, might preference would be just to pick one, and
> provide a stick for hitting those who do it the other way.
>
> --
> Dean
>
>
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 09 Jan 2013 09:56:38 -0800

On 1/7/2013 6:01 PM, John Day wrote:
> All standards groups that I am aware of have had the same view.  This is
> not uncommon.
>
> transitions are not visible on the wire.  The rules for sliding window
> are not described entirely in terms of the behavior seen on the line, etc.
>
> I have seen specifications that attempted this and the implementations
> built from them were very different and did not come close to
> interoperating or in some cases of even doing the same thing.
>

+1

A protocol *is*:

	- the states at the endpoints
	- the messages on the wire
	- a description of input events (message arrival, upper-layer
		interface requests, timer expiration) that indicates
		the subsequent change of state and output event
		(message departure, upper layer indication,
		or timers to set)

(i.e., a Mealy machine, attaching events to arcs)

The "wire" is the second of these, and entirely insufficient as a 
protocol spec.

Yes, there are two ways to try to write a protocol spec:
	- procedural
		defining the above explicitly
	- behavioral
		defining a protocol only from its external
		behavior

The difference between these is easy to see for sort algorithms:

	procedural:
		quicksort
		heapsort
		etc.

	behavioral:
		sort

AFAICT, "on the wire" often implies behavioral equivalence, but it's a 
lot more complicated than just the on-wire messages. A behavioral 
description of a protocol would treat the protocol as a black box and 
explain its behavior under every possible input.

I'll take procedural descriptions of protocols over behavioral any day.

Joe