Subject: Re: letting IETF build on top of Open Source technology

From: Miles Fidelman <[email-masked]>
Date: Wed, 1 Nov 2017 12:14:42 -0400

Let us not forget that a number of significant Internet protocols are 
based on open-source work (though not always called that).

We can start with the early TCP/IP reference implementations.  And then 
there's the web - HTTP pretty much co-evolved with early browsers & daemons.

Isn't the issue more one of underlying design philosophy & intent?  
Which comes first - the architecture & the protocol or the software?

Personally, I don't think it's an issue of the IETF "building on top of 
Open Source technology," it's a matter of educating developers to think 
in terms of protocol architectures & standards. (Something that I think 
we, as network & protocol developers have woefully failed to do - and 
that our professional organizations, like ISOC, should be doing more of.)

Miles Fidelman

-- 
In theory, there is no difference between theory and practice.
In practice, there is.  .... Yogi Berra

---

From: Behcet Sarikaya <[email-masked]>
Date: Wed, 1 Nov 2017 11:33:12 -0500

Hi Alia,

One member mentioned that the open source code does serve as the protocol
description and hence it could be referenced as dependency in RFCs.
I think we should question that.

I worked many years on formal description techniques for protocols. I think
that such specs could serve as a reference.
Source code in some specific programming language is very long and
difficult to follow for someone interested in a high level view of the
protocol.
Also many people do not know the specific programming language it was
written.

My 2 cents.

Regards,
Behcet

On Tue, Oct 31, 2017 at 1:29 PM, Alia Atlas <[email-masked]> wrote:

> Hi Mehemet:
>
> below in-line
>
> On Tue, Oct 31, 2017 at 1:14 PM, Mehmet Ersue <[email-masked]> wrote:
>
>> See ME:
>>
>>
>>
>> Thanks,
>>
>> Mehmet
>>
>>
>>
>> *From:* Alia Atlas [mailto:[email-masked]]
>> *Sent:* Tuesday, October 31, 2017 5:13 PM
>> *To:* Mehmet Ersue <[email-masked]>
>> *Cc:* Brian E Carpenter <[email-masked]>; Martin Thomson <
>> [email-masked]>; IETF Discussion Mailing List <[email-masked]>
>> *Subject:* Re: letting IETF build on top of Open Source technology
>>
>>
>>
>> Hi Mehmet,
>>
>>
>>
>> On Tue, Oct 31, 2017 at 12:05 PM, Mehmet Ersue <[email-masked]> wrote:
>>
>> Hi Alia,
>>
>> > >  1.  Is it stable, mature, and immutable (except for errata)?
>> >
>> > Those are not well known characteristics of open source projects.
>> > Even if we tie a reference down to a very specific version, I'm unclear
>> how
>> > that can truly act as a stable reference. It may be stable in a
>> mathematical
>> > sense, but it may also be seven years behind deployed reality.
>>
>> I agree with Brian. There are different type of non-SDO organizations
>> having documents with diverse maturity levels and stability. Many of them
>> don't qualify as normative references.
>>
>>
>>
>> Of course - and that's why it is necessary for the item to meet the
>> questions posed in the draft - which are the requirements (and we'll
>> rephrase more clearly as Russ suggested).
>>
>>
>>
>> Obvious examples for normative references are usually from well-known
>> SDOs like IEEE or W3C where the new standard-track RFC uses it as its basis
>> or builds on-top of it (e.g.  [W3C.REC-xml-20001006])
>>
>> I also don't see an issue with MD5 and HMAC as these are Informational
>> RFCs at IETF with a well-known stability.
>>
>>
>>
>> Sure - and that was one of the motivations for RFC 3967.
>>
>>
>>
>> Looking at section 1.1. of RFC 3967 and the statement: "a normative
>> reference specifies a document that must be read to fully understand or
>> implement the subject matter in the new RFC, or whose contents are
>> effectively part of the new RFC, as its omission would leave the new RFC
>> incompletely specified",
>> can you please give examples of external documents from non-DOSs, which
>> clearly qualify as normative references for a standard-track RFC at IETF?
>> I think primarily on non-SDO documents where the new RFC would build
>> on-top of it.
>>
>>
>>
>> An example would be potentially thrift, I think, for protocol encodings.
>> I have to hunt down the detailed reference for it.
>>
>> I know that protobufs have good documentation also.  I am sure that there
>> are others and would be happy to hear folks chime in with more.
>>
>>
>>
>> It's unclear whether there is a set of requirements acceptable to the
>> IETF Community that would work for WHATWG work.
>>
>>
>>
>> Is your concern that you don't see useful open source work that would be
>> suitable to reference? Or that none would meet our set of requirements?
>>
>> Or that it is risky to consider referencing non-SDO external work?
>>
>>
>>
>> ME: Generally I don’t see an issue with referencing useful open source
>> work. Though only in rare cases an IETF RFC has a dependency on or builds
>> indeed on-top of such a document normatively. I think the main criteria for
>> a normative reference is still based on the quoted text from RFC 3967 above.
>>
>>
>
> [Alia] Agreed - and I pulled the requirements around stability, maturity,
> confidentiality, and IPR considerations largely from 2026.
> My question is whether it is rare than an IETF RFC builds on top of open
> source work because the path to do so is seen as too
> challenging/long/expensive or simply not relevant. This draft is proposing
> a policy assuming the former applies at least some of the time.
>
>
>> I believe use cases and examples will help to define concrete
>> requirements.
>>
>> Once we have the requirements together such documents need to be looked
>> at in a case by case basis and taken over into a registry.
>>
>
> The requirements, I think, stem from our (the IETF community's)
> understanding of what is necessary for a dependency of a standards-track
> RFC and I am not proposing that those should change.  I'm trying to capture
> what they are and how to evaluate a piece of work.
>
> Rather than looking at the set of documents and creating a registry, the
> draft suggests that when a WG wants to use such an item as a dependency,
> then after that use has IESG Approval, then the item can be added to a
> registry - so other WGs and other documents in that WG can benefit from the
> evaluation that was done and feel more confident about using it.
>
> Regards,
> Alia
>
>
>> Regards,
>>
>> Alia
>>
>>
>>
>>
>>
>> Thanks,
>> Mehmet
>>
>>
>> > -----Original Message-----
>> > From: ietf [mailto:[email-masked]] On Behalf Of Brian E
>> Carpenter
>> > Sent: Monday, October 30, 2017 4:49 AM
>> > To: Martin Thomson <[email-masked]>; Alia Atlas
>> > <[email-masked]>
>> > Cc: IETF Discussion Mailing List <[email-masked]>
>> > Subject: Re: letting IETF build on top of Open Source technology
>> >
>> > On 30/10/2017 16:27, Martin Thomson wrote:
>> > > On Mon, Oct 30, 2017 at 2:18 PM, Alia Atlas <[email-masked]>
>> wrote:
>> > >> The existing downref process (RFC 3967) doesn't apply. That is to
>> > >> handle references to an RFC that is at a lower maturity - such as a
>> > >> normative reference to an Informational or Experimental RFC - or from
>> > >> an Internet Standard to a Proposed Standard.
>> > >
>> > > I couldn't find anything in 3967 to support that view (other than the
>> > > parenthetical in the abstract).  Maybe it's implicit.
>> >
>> > The quote from RFC2026 in section 1 of RFC3967 is pretty unambiguous
>> when
>> > read in context - throughout RFC2026, "standards track" refers to the
>> IETF
>> > standards track, and "standards track specifications"
>> > refers to IETF documents. And in any case the Abstract of RFC3967 sets
>> the
>> > scope thus: "Exceptions to this rule are sometimes needed as the IETF
>> uses
>> > informational RFCs to describe non-IETF standards or IETF-specific
>> modes of
>> > use of such standards."
>> >
>> > My main concern with the new proposal is other:
>> >
>> > >  1.  Is it stable, mature, and immutable (except for errata)?
>> >
>> > Those are not well known characteristics of open source projects.
>> > Even if we tie a reference down to a very specific version, I'm unclear
>> how
>> > that can truly act as a stable reference. It may be stable in a
>> mathematical
>> > sense, but it may also be seven years behind deployed reality.
>> >
>> >     Brian
>>
>>
>>
>
>

---

From: "Mehmet Ersue" <[email-masked]>
Date: Wed, 1 Nov 2017 20:41:19 +0100

Hi Alia, All,

during the discussion it kind of gets lost what kind of reference is sufficient and whether a normative reference is explicitly required.

If possible we should avoid such external dependency.

IMO only in very rare cases an IETF RFC would use an external document normatively (i.e. use the external document as its basis or build on-top of it) and a normative reference would be required.

To be able to understand in which cases a normative reference is imperative I think the requirements should be discussed and specified clearly giving real-life use cases.

Mehmet

From: Behcet Sarikaya [mailto:[email-masked]] 
Sent: Wednesday, November 1, 2017 5:33 PM
To: Alia Atlas <[email-masked]>
Cc: Mehmet Ersue <[email-masked]>; IETF Discussion Mailing List <[email-masked]>
Subject: Re: letting IETF build on top of Open Source technology

Hi Alia,

One member mentioned that the open source code does serve as the protocol description and hence it could be referenced as dependency in RFCs.

I worked many years on formal description techniques for protocols. I think that such specs could serve as a reference.

Source code in some specific programming language is very long and difficult to follow for someone interested in a high level view of the protocol.

Also many people do not know the specific programming language it was written.

My 2 cents.

Regards,

Behcet

On Tue, Oct 31, 2017 at 1:29 PM, Alia Atlas <[email-masked] <mailto:[email-masked]> > wrote:

Hi Mehemet:

below in-line 

On Tue, Oct 31, 2017 at 1:14 PM, Mehmet Ersue <[email-masked] <mailto:[email-masked]> > wrote:

See ME:

Thanks,

Mehmet

From: Alia Atlas [mailto:[email-masked] <mailto:[email-masked]> ] 
Sent: Tuesday, October 31, 2017 5:13 PM
To: Mehmet Ersue <[email-masked] <mailto:[email-masked]> >
Cc: Brian E Carpenter <[email-masked] <mailto:[email-masked]> >; Martin Thomson <[email-masked] <mailto:[email-masked]> >; IETF Discussion Mailing List <[email-masked] <mailto:[email-masked]> >
Subject: Re: letting IETF build on top of Open Source technology

Hi Mehmet,

On Tue, Oct 31, 2017 at 12:05 PM, Mehmet Ersue <[email-masked] <mailto:[email-masked]> > wrote:

Hi Alia,

> Even if we tie a reference down to a very specific version, I'm unclear how
> that can truly act as a stable reference. It may be stable in a mathematical

I agree with Brian. There are different type of non-SDO organizations having documents with diverse maturity levels and stability. Many of them don't qualify as normative references.

Of course - and that's why it is necessary for the item to meet the questions posed in the draft - which are the requirements (and we'll rephrase more clearly as Russ suggested). 

Obvious examples for normative references are usually from well-known SDOs like IEEE or W3C where the new standard-track RFC uses it as its basis or builds on-top of it (e.g.  [W3C.REC-xml-20001006])

I also don't see an issue with MD5 and HMAC as these are Informational RFCs at IETF with a well-known stability.

Looking at section 1.1. of RFC 3967 and the statement: "a normative reference specifies a document that must be read to fully understand or implement the subject matter in the new RFC, or whose contents are effectively part of the new RFC, as its omission would leave the new RFC incompletely specified",
can you please give examples of external documents from non-DOSs, which clearly qualify as normative references for a standard-track RFC at IETF?
I think primarily on non-SDO documents where the new RFC would build on-top of it.

An example would be potentially thrift, I think, for protocol encodings. I have to hunt down the detailed reference for it.

I know that protobufs have good documentation also.  I am sure that there are others and would be happy to hear folks chime in with more.

It's unclear whether there is a set of requirements acceptable to the IETF Community that would work for WHATWG work.

Is your concern that you don't see useful open source work that would be suitable to reference? Or that none would meet our set of requirements?

ME: Generally I don’t see an issue with referencing useful open source work. Though only in rare cases an IETF RFC has a dependency on or builds indeed on-top of such a document normatively. I think the main criteria for a normative reference is still based on the quoted text from RFC 3967 above. 

[Alia] Agreed - and I pulled the requirements around stability, maturity, confidentiality, and IPR considerations largely from 2026.

My question is whether it is rare than an IETF RFC builds on top of open source work because the path to do so is seen as too challenging/long/expensive or simply not relevant. This draft is proposing a policy assuming the former applies at least some of the time.

I believe use cases and examples will help to define concrete requirements.

Once we have the requirements together such documents need to be looked at in a case by case basis and taken over into a registry.

The requirements, I think, stem from our (the IETF community's) understanding of what is necessary for a dependency of a standards-track RFC and I am not proposing that those should change.  I'm trying to capture what they are and how to evaluate a piece of work.

Rather than looking at the set of documents and creating a registry, the draft suggests that when a WG wants to use such an item as a dependency, then after that use has IESG Approval, then the item can be added to a registry - so other WGs and other documents in that WG can benefit from the evaluation that was done and feel more confident about using it.

Regards,

Alia

Regards,

Alia

Thanks,
Mehmet

> From: ietf [mailto:[email-masked] <mailto:[email-masked]> ] On Behalf Of Brian E Carpenter
> To: Martin Thomson <[email-masked] <mailto:[email-masked]> >; Alia Atlas
> <[email-masked] <mailto:[email-masked]> >
> Cc: IETF Discussion Mailing List <[email-masked] <mailto:[email-masked]> >
> > On Mon, Oct 30, 2017 at 2:18 PM, Alia Atlas <[email-masked] <mailto:[email-masked]> > wrote:
> The quote from RFC2026 in section 1 of RFC3967 is pretty unambiguous when
> read in context - throughout RFC2026, "standards track" refers to the IETF
> refers to IETF documents. And in any case the Abstract of RFC3967 sets the
> scope thus: "Exceptions to this rule are sometimes needed as the IETF uses
> informational RFCs to describe non-IETF standards or IETF-specific modes of
> use of such standards."
>
> Even if we tie a reference down to a very specific version, I'm unclear how
> that can truly act as a stable reference. It may be stable in a mathematical

---

From: S Moonesamy <[email-masked]>
Date: Wed, 01 Nov 2017 13:34:55 -0700

Hi Alia,
At 08:02 PM 29-10-2017, Alia Atlas wrote:
>After some discussion, my co-authors and I have written down a 
>proposed policy that would clearly and specifically allow 
>standards-track RFCs to have external non-SDO normative 
>references.  It's not disallowed now - but determining early when it 
>is ok is unclear.

I read draft-atlas-external-normref-00.  "Opus" was an exception.  I 
don't think that it is a good example of referencing external code.

What is an "Open Source Organization"?  Are there examples of such 
organizations documenting IPR?

Regards,
S. Moonesamy

---

From: Alia Atlas <[email-masked]>
Date: Wed, 1 Nov 2017 17:14:11 -0400

Hi S.

On Wed, Nov 1, 2017 at 4:34 PM, S Moonesamy <[email-masked]> wrote:

>> After some discussion, my co-authors and I have written down a proposed
>> policy that would clearly and specifically allow standards-track RFCs to
>> have external non-SDO normative references.  It's not disallowed now - but
>> determining early when it is ok is unclear.
>>
>

Agreed - the point was supposed to be that while referencing external code
might be reasonable in extremely limited situations (an algorithm or codec
and Opus was such an example), usually a clear specification is needed.

> organizations documenting IPR?

There are a number of projects under the Linux Foundation.  Taking a quick
glance at OpenNFV, I find
https://www.opnfv.org/about/bylaws-and-policies/ip-policy
which binds contributions to the Apache License.

While the motivation that I see for this proposed policy is to help make it
easier for the IETF and Open Source to collaboration, nothing in the policy
is intended to be restricted to Open Source Organizations or require a
clear definition of one.

Thanks for reading the draft!  I really do appreciate all the feedback :-)

Cheers,
Alia

Regards,

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 1 Nov 2017 22:35:08 -0400

My specifications are my code.

If you look at any of the protocol specifications for the Mesh, you
will find a reference section in the back that is generated directly
from the schema and examples taken from running code generated from
the same schema. And the running code used to generate the examples is
the same as the reference code.

http://www.prismproof.org/Resources/

The tool set is also open source though documenting this has been a
lower priority for me than finishing the Mesh.

http://www.prismproof.org/Code/standard/html/12cd4543-7aeb-41b3-81cf-4f828b2f76d7.htm

The original point of doing this was accuracy rather than saving time.
But it certainly saves me a lot of time now I have the system working.
I am currently working on generating examples for the Web site as
well.

At present, the code generators create C# code but they have generated
C in the past. I will probably return to that after the current code
set is complete.

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 2 Nov 2017 07:01:23 +0100

How do i know which behavior of your code is a bug and which behavior is a feature ?

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 2 Nov 2017 12:25:39 -0400

> How do i know which behavior of your code is a bug and which behavior is a feature ?

The point of this exercise is to make the implementation and
specification match as closely as is feasible without the extreme
overhead of formal proof.

There is still the possibility/possibility that the specification is
broken. Hence the reason to participate in a standards review process.

---

From: Miles Fidelman <[email-masked]>
Date: Thu, 2 Nov 2017 13:01:26 -0400

> the same as the reference code.
That's a little bit disingenuous, isn't it - particularly since you've 
written quite a bit of documentation beyond your code.

Beyond that, the point of a specification is to DRIVE implementation & 
testing of multiple, interoperable implementations.  A reference 
implementation is one thing, but as soon as one specific piece of code 
(warts & all) becomes the "spec" - it's not really a specification anymore.

Miles Fidelman

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 3 Nov 2017 08:07:07 -0400

> written quite a bit of documentation beyond your code.

I did not say my code is my specification. Which is what a lot of
people have done in the past.

I develop the documentation and the code at the same time. Of course
there are parts of the documentation that are not code, the user
guide, requirements, etc. But every specification has normative and
non-normative sections.

> (warts & all) becomes the "spec" - it's not really a specification anymore.

Reference code is not necessarily the same as production. Production
code should be conservative in what it sends and liberal in what it
accepts. Reference code should be the other way round. It should
complain when the sender fails to abide by the strict specification.

We have always considered rough consensus and running code to be
important. Building code from specifications is the next logical step.

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Sun, 5 Nov 2017 02:18:37 +0000

-----Original Message-----
From: ietf <[email-masked]> on behalf of Toerless Eckert <[email-masked]>
Date: Wednesday, November 1, 2017 at 7:55 AM
To: "[email-masked]" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: letting IETF build on top of Open Source technology

    So, whats the right terminology here for at least the two classes
    that i think are easy to destinguish:
    
    a) In one case i have FOSS that implements a new protocol like rsync does,
       and interop means that you want two independent implementations support
       the protocol.
    
    b) In another case, which i think are all your examples, you have some
       form of "service" which is providing some form of "API" and interop
       really means that only one implementation is required to show how
       it works with multiple clients of the service.
    
    I also think that b) is easier to adopt because 
    - interop does not require you to find other implementations of the
      same protocol/service
    - An API that MUST be used by a third party client is more naturally
      documented than a protocol that was meant to be used between multiple
      instances of a single FOSS software (typical a) case).

[cue] I agree.When I think about opens source software that would stand up well in the face of the 5 requirements in section 2 of the draft, your class b would certainly stand a better chance. When I read the draft, I saw it as targeting that class of open source projects. 

Cheers,
Charles
    
    Cheers
        Toerless
    
    On Mon, Oct 30, 2017 at 11:19:19PM +0100, Eliot Lear wrote:
    > The obvious example would be the vast quantities of YANG that are
    > cropping up everywhere. Another one that many reference outside of the
    > IETF context is ArcSight CEF, which is an overlay on SYSLOG that is
    > frequently used.  When there's active development on a project, this
    > isn't a good fit, due to feature velocity.  Even there, though, some may
    > wish to declare certain interfaces both public and stable.  I could
    > imagine the git team doing that, for instance, given the number of front
    > ends there are with it.
    > 
    > Eliot

---

From: "Charles Eckel (eckelcu)" <[email-masked]>
Date: Sun, 5 Nov 2017 02:18:37 +0000

This is true in some cases, but not in others. There has been and continues to be a big push in open source software and in software in general to create applications that are compose of many small pieces that interact with each other through well defined interfaces. This leads to more modular software and better code/component reuse. This practice results in code that is more appropriate as a normative reference. I would expect the normative reference to be to a specific version or release of the software and its corresponding API(s). Those APIs could change with future versions of the software, but as with standards, we tend to try pretty hard to avoid non-backward compatible APIs changes after the software has reached a state of maturity that others on building on top of it and relying on those APIs.
  
Cheers,
Charles

-----Original Message-----
From: ietf <[email-masked]> on behalf of Keith Moore <[email-masked]>
Date: Wednesday, November 1, 2017 at 4:10 PM
To: Alia Atlas <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: letting IETF build on top of Open Source technology

    It seems to me that a fundamental problem with referencing Open Source software is that in much (not all) of the Open Source world, rapid innovation and evolution is valued more than stable protocol interfaces.   There is a tendency to rely on users to frequently update their software, which is seen as giving developers freedom to change interfaces on relatively short notice.  
    
    Of course that's an oversimplification.  In practice good open source developers are careful to avoid abrupt changes to interfaces; instead they create new interfaces and deprecate old ones.   But since one of the oft-touted benefits of Open Source is to allow software systems to evolve rapidly, this sometimes also results in new protocol features being deployed rapidly to the point that it impairs interoperability between new and old implementations.   This causes a great many problems other than just protocol interoperability failures, including dependency hell.
    
    So part of the effort to improve cooperation between IETF and Open Source developers - who tend to have similar motivations even if their methods differ - might be in convincing them that they would benefit from more stable interfaces - at least on the wire - or at least encouraging them to define protocols so that they are cleanly extensible, and defining a baseline for interoperability even as they extend protocols to accommodate new features.   And actually it might be the case that IETF could learn something about protocol extensibility from people who have struggled to accommodate rapid evolution of their software.
    
    Keith

---

From: Ramki Krishnan <[email-masked]>
Date: Sun, 5 Nov 2017 15:17:43 -0800

I would agree with Charles, this is what we have in mind for a large open
source project such as ONAP.

Thanks,
Ramki

On Sat, Nov 4, 2017 at 7:18 PM, Charles Eckel (eckelcu) <[email-masked]>
wrote:

> This is true in some cases, but not in others. There has been and
> continues to be a big push in open source software and in software in
> general to create applications that are compose of many small pieces that
> interact with each other through well defined interfaces. This leads to
> more modular software and better code/component reuse. This practice
> results in code that is more appropriate as a normative reference. I would
> expect the normative reference to be to a specific version or release of
> the software and its corresponding API(s). Those APIs could change with
> future versions of the software, but as with standards, we tend to try
> pretty hard to avoid non-backward compatible APIs changes after the
> software has reached a state of maturity that others on building on top of
> it and relying on those APIs.
>
> Cheers,
> Charles
>
>     It seems to me that a fundamental problem with referencing Open Source
> software is that in much (not all) of the Open Source world, rapid
> innovation and evolution is valued more than stable protocol interfaces.
>  There is a tendency to rely on users to frequently update their software,
> which is seen as giving developers freedom to change interfaces on
> relatively short notice.
>
>     Of course that's an oversimplification.  In practice good open source
> developers are careful to avoid abrupt changes to interfaces; instead they
> create new interfaces and deprecate old ones.   But since one of the
> oft-touted benefits of Open Source is to allow software systems to evolve
> rapidly, this sometimes also results in new protocol features being
> deployed rapidly to the point that it impairs interoperability between new
> and old implementations.   This causes a great many problems other than
> just protocol interoperability failures, including dependency hell.
>
>     So part of the effort to improve cooperation between IETF and Open
> Source developers - who tend to have similar motivations even if their
> methods differ - might be in convincing them that they would benefit from
> more stable interfaces - at least on the wire - or at least encouraging
> them to define protocols so that they are cleanly extensible, and defining
> a baseline for interoperability even as they extend protocols to
> accommodate new features.   And actually it might be the case that IETF
> could learn something about protocol extensibility from people who have
> struggled to accommodate rapid evolution of their software.
>
>     Keith
>
>
>
>

-- 
Thanks,
Ramki

---

From: Miles Fidelman <[email-masked]>
Date: Sun, 5 Nov 2017 18:43:48 -0700

Meant to reply earlier - on a trip...

> guide, requirements, etc. But every specification has normative and

More precisely, can you elaborate just a tad on HOW your "specifications 
are your code?"

I've taken a look at some of the documentation you've pointed at, but 
have yet to find any that have "a reference section in the back that is 
generated directly from the schema and examples taken from running code 
generated from the same schema" - can you perhaps give a specific reference?

And can you say just a tad more about the representation of the schema, 
your tool chain, and process?  (Without taking a really deep dive, what 
I surmise is that the Goedel tools can ingest ASN and generate code - am 
I reading that correctly?)

> complain when the sender fails to abide by the strict specification.

Now that is a very good point - well worth remembering.

Miles Fidelman