Subject: Re: Making RFC2119 key language easier to Protocol Readers

From: Abdussalam Baryun <[email-masked]>
Date: Wed, 9 Jan 2013 09:03:23 +0100

Hi Martin,

I agree with your points below, if any one finds an error in IETF
standards then should report it on this list or write an ID about that
to community,

AB
++++++++++++++++++++++++++

Sub: Re: A proposal for a scientific approach to this question [was
Re: I'm struggling with 2219 language again]

http://www.ietf.org/mail-archive/web/ietf/current/msg76710.html

>I believe that the problem with the formal logic is that it is difficult
to both write as well as read/understand, and to verify that the chosen
"axioms" actually reflect (and lead to) the desired behaviour/outcome,
the relative of scarcity of suitable tools, the complexity of the
underlying theory and tools, and the tools' resulting lack of
"intuitive usability".

>> Once you have both, there is still the problem that if a bug or ambiguity
>> shows up,

For any formal proofing system worth its dime, this can be 100% ruled out,
since the proofing system will emit a 100% bugfree implementation of the
spec in the programming language of your choice as a result/byproduct of the
formal proofing process.

>>
>>  neither the code nor the formal spec nor a prose spec can be taken
>> be the right one.  What is right is still in the heads of the authors.

But maybe you're not really thinking of a defect in the implementation
(commonly called "bug") but rather a "gap" in the specification that
leads to unintended or undesired behaviour of a 100% bug-free
implementation of your spec.

(see Section 4 of this paper for an example:
  http://digbib.ubka.uni-karlsruhe.de/volltexte/documents/1827
)

Donald Eastlake wrote:
>
> Another problem is maintenance. Protocols change. Having to maintain a
> formal specification is commonly at least an order of magnitude more
> effort than maintaining a prose description.

Definitely.  Discussing protocols at the level of formal specification
language would be quite challenging (typically impossible in an open
forum, I believe it is even be difficult in a small and well-trained
design team).

>
> And, as has been mentioned before, I'd like to emphasize that the IETF
> experience and principal is that, if you want interoperation,
> compliance testing is useless.

Ouch.  I believe this message is misleading or wrong.

Compliance testing is VERY important, rather than useless,

Compliance testing would actually be perfectly sufficient iff the spec
was formally proven to be free of conflicts and ambiguities among
specified protocol elements -- plus a significant effort spent on
ensuring there were no "gaps" in the specification.

As it turns out, however, a significant amount of implementations will
be created by humans interpreting natural language specifications,
rather than implemenations created as 100% bug-free by-products of a
formal proof tool, and often, interop with such buggy, and often
spec-incompliant implementations is desirable and necessary since
they make up a significant part of the installed base.

>
> The way to interoperation is interoperation testing between
> implementations and, to a first approximation, the more and the
> earlier you do interoperation testing, the better.

The #1 reason for interop problems and road-block to evolution of
protocols is the wide-spread lack of compliance testing (on the flawed
assumption that it is useless) and focus on black-box interop testing
of the intersection of two subsets of protocol features.

The problem with interop testing is that it really doesn't provide
much of a clue, and the results can _not_ be extrapolated to features
and areas that were not interop tested (typically the other 90% of the
specification, many optional features and most of the extensibility).

The near complete breakage of TLS protocol version negotiation is a
result of the narrow-minded interop testing in companion with a complete
lack of compliance testing.

If done right, pure compliance testing can go a long way to providing
interop.  The only area where real interop testing is necessary is
those protocol areas where the spec contains conflicts/inconsistencies
that implementors didn't notice or ignored, or where there is a
widespread inconsistencies of implementations with the specification,
of created by careless premature shipping of a defective implementation
and factual creation of an installed base that is too large to ignore.

-Martin

---

From: Abdussalam Baryun <[email-masked]>
Date: Tue, 15 Jan 2013 13:54:06 +0100

Hi Marc Petit-Huguenin,

I agree that we need to be able to make complex protocol's readable in IETF,
That is why I am doing an update ID for the RFC2119, I know many don't
think it is a right thing to do, but I think maybe in future while
making new versions of the update draft I will get to better
discussions,

AB
++++++++++++++++++++++++++++++++

Date: Tue, 15 Jan 2013 03:21:41 -0800

> I think there are cases of standards of extreme complexity, such as SIP,
> where such profiles may be useful, because otherwise interoperability
> cannot be achieved.

I would not call SIP a standard of extreme complexity, but anyway there is
more and more protocols on a similar complexity - just two protocols that I am
working with currently - RSTP 2.0 and RELOAD - are of similar complexity.

> But perhaps the lesson for the IETF here is slightly different - don't
> design standards which allow that degree of complexity in the first place.

There is no simple solution to a complex problem, so as the problems we try to
solve increase in complexity, so are our solutions to them.  But perhaps you
are right in a way.  Perhaps the problem is simply that RFC 2119, and the
issues I and other see with the approach in using as little of the keywords as
possible, was designed for a time when problems - and solutions - were
simpler.  Perhaps RFC 2119 imposes an upper limit on the complexity that a
protocol developed with it can reach, and we are just hitting this threshold
more and more often.

I am not saying that it is a bad thing - I certainly like simple protocols,
but perhaps the IETF is simply the wrong place for developing complex protocols.

- --
Marc Petit-Huguenin
Email: marc at petit-huguenin.org

---

From: Marc Petit-Huguenin <[email-masked]>
Date: Tue, 15 Jan 2013 05:20:17 -0800

On 01/15/2013 04:54 AM, Abdussalam Baryun wrote:
> Hi Marc Petit-Huguenin,
> 
> I agree that we need to be able to make complex protocol's readable in
> IETF, That is why I am doing an update ID for the RFC2119, I know many
> don't think it is a right thing to do, but I think maybe in future while 
> making new versions of the update draft I will get to better discussions,

Changing RFC 2119 would perhaps permit to manage more complex protocols, but
this will delay the problem, not fix it.  Perhaps there is no other solution
than using formal languages when going beyond a threshold of complexity, and
that definitively not something that will be done at the IETF.

But perhaps there could be enough interest to create an IRTF working group, on
the subject of writing specifications for complex protocols.

> 
> AB ++++++++++++++++++++++++++++++++
> 
> more and more protocols on a similar complexity - just two protocols that I
> am working with currently - RSTP 2.0 and RELOAD - are of similar
> complexity.
> 
> There is no simple solution to a complex problem, so as the problems we try
> to solve increase in complexity, so are our solutions to them.  But perhaps
> you are right in a way.  Perhaps the problem is simply that RFC 2119, and
> the issues I and other see with the approach in using as little of the
> keywords as possible, was designed for a time when problems - and solutions
> - were simpler.  Perhaps RFC 2119 imposes an upper limit on the complexity
> that a protocol developed with it can reach, and we are just hitting this
> threshold more and more often.
> 
> I am not saying that it is a bad thing - I certainly like simple
> protocols, but perhaps the IETF is simply the wrong place for developing
> complex protocols.

- -- 
Marc Petit-Huguenin
Email: [email-masked]
Blog: http://blog.marc.petit-huguenin.org
Profile: http://www.linkedin.com/in/petithug

---

From: Abdussalam Baryun <[email-masked]>
Date: Tue, 15 Jan 2013 15:03:22 +0100

I don't think there is a general level of simple or complex protocol,
it always will depends on a point of view a machine,

AB

On 1/15/13, Marc Petit-Huguenin <[email-masked]> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
>
> iQIcBAEBCAAGBQJQ9VeLAAoJECnERZXWan7E+u0P/1howuEDFI+h9FSV77Cv2YBW
> u5+Kfj9fhPp0nhqLr8z/1Hc4YAK/+YPmQ0gDzO5SVtnMTZwE5N3RgPx+yglOZ11X
> peZdwNgftMTSZOwc2r1B/gdHdyeVgYtZnEFJicbVgxbW6VirRml8qtYrOGijeh5B
> CAneVVqjpF1ou3BXjvGoxCXyP0Tw/XCPt+6+vGYpZ8GApfVizECViYljyZrOeDEG
> ZW6vgGnBrV6G+vqRXJRRZELZu2WMnJaC2ADHfCpIdAKCTBKX6Lj7v+7Ni4IAjEYC
> Z6hEDynBwaNVODMAuglbmVfC5I2kB45M5JwLoVu8sbfhOrGEJi7CnHDTbi1jW1N/
> 70ZdsQvHbDj0uJL4s6mulwnJOYCh3d7uqAeMEFM5uyMYA+U5/5Dr7RYOhrP/xD0Y
> U+Y/YlFz+TLncsio7aYnzEpdjfFDYnbeLLyE0TdjM1kIGPkeKA3GPAdJbW4mDXpg
> VAbxGgiS89mSXGOjhlUKjfMEmmF7R21e8kAcubonOp/chu/nqO821qE1lfAYNZ5i
> WGHUPPTB/p5iXmhkt9cHESlOB1suW5r5y4Qq5qkz376FraIHIQgflSX4HuE0BBx1
> DEbxoOhr9QgHU6INEt1GBuwxzJW076tD22hkprkl/w90KtbHP2HvjpK5bkNN/MA5
> owExvo14qd9YEq0/SKhk
> =YyMX
> -----END PGP SIGNATURE-----
>