Subject: RE: Last Call: <draft-housley-implementer-obligations-01.txt> (Expectations of Implementers of IETF Protocols) to Informational RFC

From: "Adrian Farrel" <[email-masked]>
Date: Sat, 10 May 2014 19:34:22 +0100

While I appreciate the effort the author put in to resolve previous
discussions, I do not support the publication of this document.

The thrust of my previous comments were to say "It is all platitude, 
but probably harmless" and "at whom is this document aimed". This 
review is in a little more detail. I still find the whole document to
be one big platitude that does not need to be published, but I have also
found a number of things that I think need to be fixed.

---

This document significantly conflates advice to implementers wishing to
ensure interoperability, best practice for people claiming to have
implemented (i.e., claiming conformance to) a specification, and 
constraints to the freedom for implementers of IETF specifications.

As the text notes, IETF specifications (please don't call them standards
unless that is what you really mean) are not mandatory to implement. So
the text really must not tell people what they must or must not do. For
example, the Abstract says "By choosing to implement..." This is
nonsense! I can choose to do what I like. If I choose to implement stuff
and tweak it and make it better, that is entirely my choice. 
Maybe you could have said "By claiming to have implemented..." But even
that is marginal. We are not the Internet Police and we have no 
influence in the world of advertising or marketing. Nor do we run 
conformance labs. This attempt at constraining implementations is bogus
and needs to be removed from the document.

As far as interoperability is concerned, it is great to give concrete
advice. When the Introduction says
   This document provides advice to implementers of IETF protocols to
   improve interoperability of their implementations.
it would be wonderful if the document lived up to the claim. But this
claim seems to be at odds with the document Title and Abstract and all
I find in the document is effectively "If you want two implementations
to interoperate, they need to implement the same thing." Well, if that
comes as a surprise to anyone perhaps they are in the wrong business.

So what is the document actually trying to do and say, and to whom?

---

The document also contains a lot of passive voice that hides the
motivation for the text. For example, the Abstract says "...one is
expected..." Expected by whom? Why? Perhaps you can attribute the 
expectation to make this meaningful.

---

Introduction
"IETF protocols foster interoperability."
I don't believe this is true. A protocol cannot of itself achieve this.
Possibly the clear specification of a consensus-based protocol can do
that. Possibly.

---

Introduction
   Yet, IETF standards are
   voluntary standards.  No one is required to implement them.
   Implementation is a choice.  By making this choice, an implementer is
   expected to:

      (1) Follow the protocol specification;

Please clarify "IETF standards" since you almost certainly don't mean 
what you have written.

But note that this text is circular. Implementation of a specification 
is, by definition, following the specification. So this text doesn't
say anything! It is the *claim* of implementation that has an 
expectation attached to it. What I do in the darkness of my own room is
not a matter for anyone's expectations.

---

Introduction

   When implementers meet these expectations, protocols interoperate as
   intended by the IETF.

This is a mixed message. Are you trying to set out the expectations on
people who implement (or claim implementation), or are you giving advice
on how to achieve interoperability?

---

Section 2

   An implementer needs to maintain their implementation
   into the future.  It is not sufficient to do an initial
   implementation of the protocol.  One needs to apply changes as they
   come out.

While the example given is highly desirable, it does not go as far as 
"needs to". An implementation is a snapshot, a moment in specification
history, and is correctly described as such. There is no moral or other
binding on the implementer to make a change, just as there was no 
requirement that the implementer select a particular specification to
implement.

When I implement a protocol as specified in an RFC I am not making a 
commitment to update my implementation to fix bugs in the specification
or to add features.

Furthermore, not all protocol extensions are desirable in all 
environments. There is no requirement for an implementer to add a widget
as there is no requirement for an implementer to implement version 2 of
a protocol or to pick up fixes to version 1.

---

Section 3

I am not comfortable with this attempt to define the purpose and meaning
of BCPs over and above RFC 2026. The statements in this section might 
reflect how BCPs have been used in the past, but this text is too strong
in the way it looks to the future. It might be appropriate to put this 
into the past tense...

   Best Current Practices (BCPs) about IETF protocols (not the BCPs that
   define IETF processes and procedures) have often been used to 
   document IETF consensus about operational or implementation practices
   pertaining to IETF protocols.

By *your* definition of BCP, why is this document not a BCP?

---

Section 3

   By following the BCPs,
   implementers, operators, and administrators are able to provide a
   common experience when using the protocol, regardless of their point
   of attachment to the Internet.

Do you mean "provide" or "obtain". If "provide" then provide to whom?

---

Section 3

   Sometimes BCPs are referenced in the protocol specification.  Often
   the implementer needs to look through the BCP index to find related
   BCPs.

The implication here might be that by checking the list of BCPs an
implementer will find all of the relevant advice and guidance outside the
specification itself. This is not true. There are plenty of 
Informational RFCs describing ways to build and deploy protocols. And 
there is this (under-used) thing called an Applicability Statement
[RFC 2026 - section 3.2].

Adrian

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 11 May 2014 08:31:45 +1200

Hi Adrian,

On 11/05/2014 06:34, Adrian Farrel wrote:
> be one big platitude that does not need to be published, 

I can certainly see how you could reach that view, but I think you're
looking at it from the viewpoint of somebody who's been around the
IETF for a long time and knows what we do here and why we do it.

I like to think of somebody else: a young programmer working far,
far away, who will probably never attend an IETF meeting or join
an IETF mailing list. For this person, we need to state things that
are obvious to us. For example:

"It is not sufficient to do an initial implementation of the protocol.
 Maintenance is needed to apply changes as the come out in the future,
 especially to fix security issues that are found after the initial
 publication of a protocol specification."

That isn't a trivial statement. It says that a protocol design may have
zero-day vulnerabilities and if you implement it, it's your job to
watch out for future updates and apply them. Is that going to be obvious
to someone starting a garage company in Africa?

I think the document is valuable.

One specific comment: I think it should be mentioned that implementers
should always read the references cited in a specification, especially
but not only the normative references, and apply them as relevant.

I also suggest mentioning implementation of extensions. Faulty extensions
are harmful to interoperability and security. We have a couple of RFCs
about that too (4775 and 6709).

   Brian

> but I have also

---

From: Abdussalam Baryun <[email-masked]>
Date: Sun, 11 May 2014 11:17:38 +0100

I agree with most of Adrian comments.  However, Brian's opinion is
reasonable and the author's intentions are understood, but the draft should
change its approach in solving the problem and its sources. I think more
work needs to be done in the title, Abstract, and body. The draft has a
narrow viewpoint and does not consider the problems in users' expectations
of IETF.

The draft needs to identify the problem clearly, then design the approach
to solve such problem as informational work or as best practice work (it is
better to make it an informational). The problem is not only in the user
side or the implementor side but also includes the IETF specification
author, the problem domain includes IETF process that should consider
expectations of both sides.

More comments below, and I hope the author to consider my suggestions and
comments in this input.

> > discussions, I do not support the publication of this document.

Agree, because more work is needed.

> > be one big platitude that does not need to be published,

I think the requirement was to enhance the relationship between
IETF specification author and  IETF user, their both expectations are not
matching, so the draft needs to fix both sides not only one, while the
source of problem may be the IETF or the IETF expectations.

> IETF for a long time and knows what we do here and why we do it.

You Brain are the same as Adrian, both been around for long time. However,
the relationship with diversified users and their expectations is still
needed in IETF.

> are obvious to us.

> I agree, but also we need to see if our IETF process and its expectations
are involving such small firm diverse users.

> For example:

I agree, but does IETF have relationships with African users. We cannot
make our expectations of others if we don't know them or we don't involve
them in this draft process. I recommend many diverse users to review this
draft and we get their feedback and document it to the acknowledgement
section.

> I think the document is valuable.

Valuable only for IETF and for who were around for long, but not much for
the diverse implementers.   The draft title should state expectation of the
relationship between IETF and users/implementers.

> but not only the normative references, and apply them as relevant.

So that is good suggest, because you are fixing the author side or the IETF
side, so any author/IESG-member needs to consider this draft while the IETF
processes.

> about that too (4775 and 6709).

That is important for both sides authors and users. Does WGs consider all
RFCs within IETF or does IETF WGs just look into their published work? We
need more cross Area within IETF process, that confuses the users very very
much. Users are expecting that IETF is one body and not separated by Areas
or WGs, this statement needs to be added to the draft as well.

AB

---

From: S Moonesamy <[email-masked]>
Date: Sun, 11 May 2014 02:48:41 -0700

At 12:18 09-05-2014, The IESG wrote:
>The IESG has received a request from an individual submitter to consider
>the following document:
>- 'Expectations of Implementers of IETF Protocols'
>   <draft-housley-implementer-obligations-01.txt> as Informational RFC
>
>The IESG plans to make a decision in the next few weeks, and solicits
>final comments on this action. Please send substantive comments to the
>[email-masked] mailing lists by 2014-06-06. Exceptionally, comments may be

The document defines three expectations which can be summed up as 
"follow".  I don't think that's a good idea.

The Introduction Section of the draft states that IETF protocols are 
about interoperability and defines what is expected of 
implementers.  In that section:

   "IETF protocols foster interoperability.  This interoperability brings
    great benefits.  IETF protocols are building blocks for many products
    and services, and they enable innovation."

The above comes out as marketing IETF protocols to the world.  The 
implementers I am acquainted would be suspicious if I were to use 
words such as "great benefits" and "enable innovation".

   "Yet, IETF standards are voluntary standards.  No one is required to
    implement them."

Some IETF standards, e.g. TCP, are de facto standards.  When an IETF 
standard is widely used and widely implemented there isn't any 
practical alternative except to follow suit.  Arguing that it is a 
voluntary goes against describing an IETF protocol as the standard.

The real test of a protocol specification is the implementation.  An 
implementer (not speaking for anyone else) decides how to implement 
what is specified in the RFC instead of blindly following what is 
written in it.  I'll note that some IETF protocols are 
over-specified, e.g. they prescribe methods instead of specifying the 
interoperable elements.

Ideally, a BCP would describe best current practice.  In practice, a 
BCP states the beliefs of the proponents as current practice.  The 
expectation for implementers to follow associated BCPs could be seen 
as an imposition.

It is not clear what IANA registry practices has to do with this 
document.  There is a normative reference to RFC 2860.  That RFC is 
an agreement between IETF and ICANN.

Section 2 quote Postel's Law. It has been argued in an IEEE 
publication that several mistakes in Postel's principle lead to the 
opposite of robustness - unmanageble insecurity.  The RFC 1122 
discussion of that principle provides better guidance to the 
implementer.  Unfortunately, there is a tendency to quote the 
sentence instead of the RFC 1122 discussion.

Section 2 of the draft states that "many protocol specifications are 
living documents".  An implementer would expect that a specification 
has reached a level of maturity where it won't be changed 
significantly.  The rationale here is that there is a cost to making 
changes.  There are also other parties involved and the implementer 
does not have any control over them.  On a somewhat related note, the 
IETF has published a specification in 1998.  There are errata for 
that specification.  It seems that the IETF found it sufficient to 
publish a specification and not include technical changes which 
affect the specification.  Reading nine "updates" does not make the 
life of the implementer easier.  In simple terms, the IETF would be 
unable to follow its own advice.

Section 3 mentions that:

   "Often the implementer needs to look through the BCP index to find
    related BCPs."

The secret of Polichinelle is that there is some hand-waving in a BCP 
when nobody knows, or there isn't agreement, about what guidance to 
give.  That leaves the implementer without guidance.

Section 4 is quite lengthy compared to the sections about the first 
two expectations.  The section states that the Internet Assigned 
Numbers Authority is a central authority.  That section might be in 
contradiction with RFC 6220; the latter mentions a Registry Operator 
without stating that the Internet Assigned Numbers Authority will 
always be the central authority.

The following is odd:

   "Implementers are expected to follow the IANA registry practices
    associated with the protocol, especially in the assignment of new
    values.  By following these practices, other implementations will
    learn about new values and make the appropriate updates to handle
    them properly."

It looks like implementers are to follow registry practices so that 
other implementations will have to be updated.  It lessens the 
argument for interoperability by choice and makes a strong argument 
for forcing implementations to follow the registry practices.  A 
future effect might be to turn registry assignments into 
controversial discussions with the IETF acting as a gatekeeper.

The implementer is advised to follow the registry practices for 
"top-level DNS names in the rare cases where such values are 
needed".  A search turned up:

http://support.microsoft.com/gp/gp_namespace_master
http://support.apple.com/kb/ts3389
http://www.belkin.com/PYRAMID/AdvancedInfo/F5D7632uk4Aver7000/Interfaces/Interface/English/lan_main_0.stm
https://support.comodo.com/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=1295
http://www.enterprisenetworkingplanet.com/netsysm/article.php/991281
http://community.linuxmint.com/tutorial/view/159
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-hostname.html

The advice has not been followed previously.  It is unlikely that it 
will be followed.

It is better to have a technical discussion about the implementation 
details of a specification than to argue that "you must do what the 
RFC says".  Interoperability does not work without mutual 
consent.   An expectation that people will follow does not create a 
sense of collaboration.  The draft mentions that the "expectations 
reflect the fundamental philosophy of the IETF".  As much as it is 
appealing to know better, is that the philosophy that the IETF would 
like to encourage?

Regards,
S. Moonesamy

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Sun, 11 May 2014 21:55:17 +0200

> > discussions, I do not support the publication of this document.

I cannot see the purpose of this document. If it is published and an
implementor does not follow it, what does it happen? Does IETF revoke
his/her license to implement?

There are also phrases in the document that while they are
good-intentioned they make no sense, and an implementor will dismiss the
whole document because of it.

For example: "An implementer should plan to maintain their
implementation."

So if an implementor is under contract to implement RFCxxxx must he
continue updating his implementation even after the contract is
terminated? What does this phrase conveys? Will IETF take care of his
expenses?

At best I believe the document needs to re-purpose itself. 

>  publication of a protocol specification."

This document doesn't fill this purpose as it is written as a what-to-do
document rather than a document with advice to implementers. If somebody
has specific expectations from implementers then that should be
reflected in a contract with them.

If on the other hand this is written in purpose to introduce
IETF-certified or IETF-approved implementations it must be even more
precise than this document. As it is, it doesn't fill any obvious
purpose.

regards,
Nikos

---

From: Brian E Carpenter <[email-masked]>
Date: Mon, 12 May 2014 08:23:27 +1200

Nikos,

> On Sun, 2014-05-11 at 08:31 +1200, Brian E Carpenter wrote:

...
> reflected in a contract with them.

That's a straw man. You know very well that (precisely because IETF
standards are voluntary) there will never be such a contract between
the IETF and the implementer.

> precise than this document. As it is, it doesn't fill any obvious

The document is aspirational, not contractual. It seems perfectly reasonable
to ask implementers (whether a profit-making company, an open-source
community, or an individual) to accept ongoing responsibility for their
code. Isn't that exactly what GnuTLS does, for example?

I'm not sure the IETF has ever said this before, however, and the only
way we have to say things permanently is by publishing an RFC.

   Brian

---

From: Elwyn Davies <[email-masked]>
Date: Mon, 12 May 2014 00:53:24 +0100

Hi.

I have some sympathy with both Brian's and Adrian's points of view on
this document.

I am with Brian in that I think there is a useful document buried in
here and trying to get out.

I am with Adrian in that it is currently relatively content free and, to
my mind, has both the wrong 'tone of voice' (particularly the abstract,
but generally) and misrepresents what the IETF is trying to do.
Specifically I would not use 'expectation' but try to explain the
(three) strands that the IETF provides to smooth the implementers path
to a useful and interoperable implementation.

I also think there are some important points that could be added to make
it less platitudinous:
- We are assuming (I believe) that the implementer isn't first up with
their implementation - probably ought to make this clearer as the
intende audience - and so there will be one or more existing
implementations out there.  New implementers should look at them for
various reasons.
- RFCs contain requirements language:  MUST etc are clear, but what
about SHOULD/RECOMMEND and MAY?  The (naive) implementer deserves some
advice here.  S/he needs to think about any SHOULDs... Why would the
implementation not follow the SHOULD? Maybe because 'everybody else'
ignored it as well (in the prior implementations), but see the next
point.  MAYs also need to be considered... they can represent options.
What did the prior implemntations do? Is this the right choice for *this
implementation*?
- What are the implementer's aims? A tool for some target audience? The
best and most comprehensive implementation ever? A basis for pulling the
protocol to pieces and making a better protocol?  If the first, then the
audience needs to be asked what they are expecting... they may have
views on the SHOULDs and MAYs ... or they may hedge their bets and want
configuration options.  If its the second or third, then .. you're on
your own sport! However, the implementer still needs to think about
potential users.
- A reminder that RFCs only mandate the order and meaning of the bits on
the wire, together with message flow and state machines and possibly
some configuration options that need to be provided.
- If an RFC talks about data structures in nodes, they tell you what
probably ought to be stored and a abstraction of a storage structure; it
is *not* mandatory that the implementer follows this dogmatically - if
there is a better way s/he can think of, then go for it. 
- Not all prior implementations are right: It is probably worth the
implementer capturing some sample traffic and using it to verify whether
it matches the specification.  In a very few cases, we say through
gritted teeth that the implementation is right.  In some other cases
(TCP - RFC 793 - being a case in point) significant parts of the
specification have never been implemented or have fallen into disuse:
Regrettably this is not easy to cope with but is less true of more
recent specs.
- Many RFCs are *very complicated*: the captured traffic can also be
used to check whether the prospective implementer has understood the
specification correctly.
- The IETF doesn't provide text specs: Plan your testing as you go
along.
- Oh, and check the ERRATA!

Regards,
Elwyn

---

From: <[email-masked]>
Date: Mon, 12 May 2014 03:17:47 +0000

Even the title is ambiguous: Expectations *for* implementers...

Obligations (in the filename)? there are no obligations.

Discussion of Postel's law is simplistic; interop is often improved by exactness, and debugging when being liberal can be an exercise in frustration.

Not sure I see the point of this doc. Implementers often read specs in isolation, they won't see this unless it's a mandatory normative reference. And it's not good enough to be that or BCP. 'Follow the BCPs' could be read as 'follow the doc references', and this won't be one of them.

What Elwyn said.

Lloyd Wood
http://about.me/lloydwood
________________________________________
From: ietf <[email-masked]> on behalf of Elwyn Davies <[email-masked]>
Sent: Monday, 12 May 2014 9:53 AM
To: Brian E Carpenter
Cc: [email-masked]; [email-masked]
Subject: Re: Last Call: <draft-housley-implementer-obligations-01.txt> (Expectations of Implementers of IETF Protocols) to Informational RFC

Hi.

Regards,
Elwyn

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Mon, 12 May 2014 09:40:40 +0200

> the IETF and the implementer.

I believe that's to the point of the document. Unless there is a
contract between IETF and the implementer, such documents should be
published as advisory to the implementer rather than listing
expectations from the implementer. That's at minimum a matter of
courtesy to implementers.

> code. Isn't that exactly what GnuTLS does, for example?

The only expectations from an implementation are the ones you see in
written in the contract you have with the implementer. If you don't
have one, then you check the license of the implementation. On gnutls
that you refer to, if you got without any contract, you see: 'BECAUSE
THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE
LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER
EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS
WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION. '

So do you really think that publishing a document in IETF that adds
expectations to an implementation is going to change that? However, I
agree that there may be expectations that one may infer from the
activity of a community project, but that's orthogonal to the document
at hand.

regards,
Nikos

---

From: Eliot Lear <[email-masked]>
Date: Mon, 12 May 2014 10:00:19 +0200

While I am neutral on the need for such a document – perhaps Brian or
Adrian is correct – I have some concerns about its content.

When implementers vary from specification, it falls into two cases: a
bug or a feature.  I have varied from the spec when I thought it was
necessary to protect my customer base or to improve their experience in
some other way.

I consider it a more nuanced question as to when Postel's Law should be
applied, because it is often difficult to distinguish a "technical
error" from an attack.  It may be better to fast fail in these sorts of
circumstances, and I believe working groups have been taking this
attitude for some time in their guidance, starting with DRUMS way back when.

To address the above concerns, I would suggest that the author provide
more nuanced guidance.  One must balance being liberal in what one
accepts against the likeliehood of an attack or misinterpretation of
what the sender is trying to say.  Furthermore, for any protocol that
involves some sort of store and forward or proxy component, in as much
as intermediaries are able to interpret a malformed protocol element, if
the end point cannot, one must see that the error gets back to the right
place.

In addition, I concur with Elwyn about the tone of the document, in that
it borders on legalistic.  Fundamentally we argue that IETF standards
are voluntary, and I agree with Adrian on that point.

I have different concerns about the need to maintain implementations. 
This is a very important concern in the Internet of Many Tiny Little
Things that deserves attention.  Considerable research has been
undertaken as to how long it takes to get zero day exploits fixed in
browsers and phones.  But the world is changing and browsers and phones
are now joined by refrigerators[1] and toilets[2] (to name a few
examples).  I agree with Russ that this is a problem, and it may even be
the IETF's problem (see above about NOT being liberal, for instance),
but perhaps this area needs its own document.

Finally, it seems to me that much of this (revised) text could be
incorporated into the Tao.

Eliot
[1]
http://www.businessinsider.com/hackers-use-a-refridgerator-to-attack-businesses-2014-1
[2]
http://www.theverge.com/2013/8/3/4584980/inax-satis-bluetooth-toilet-android-app-vulnerability

---

From: John C Klensin <[email-masked]>
Date: Mon, 12 May 2014 09:20:02 -0400

--On Monday, May 12, 2014 08:23 +1200 Brian E Carpenter
<[email-masked]> wrote:

>...
>> This document doesn't fill this purpose as it is written as a
>> what-to-do document rather than a document with advice to
>> implementers. If somebody has specific expectations from
>> implementers then that should be reflected in a contract with
>> them.
> 
> That's a straw man. You know very well that (precisely because
> IETF standards are voluntary) there will never be such a
> contract between the IETF and the implementer.

But that is exactly the point, I think.  This document reads
like an attempt to make one because it leads immediately to "you
implemented that IETF spec, therefore you are/were obligated
to..." statements and arguments.

Also, let's be clear about "voluntary".  I hope we never
contemplate this, but suppose the IETF were to break its
protocol specifications into two parts, the requirements
statement, context, and overview and the actual protocol
specification.  And suppose we created a mechanism by which,
while the former was public, freely available, etc., getting to
the latter required a click-through license asking for agreement
to these sorts of expectations.  I'll leave it to the lawyers
(actual and amateur) to figure out something like that might
work and be enforceable, but it would be no worse than a lot of
commercial software licenses, arguably including the GPL.   The
standard would still be "voluntary" in the sense that no one was
required to implement it.  And _that_ is the sense in which
"voluntary standard" has been used, consistently, since I first
started doing standards work in the early 1970s.

Let me rephrase Nikos's comment (probably in a way he didn't
intend).  "If a particular protocol spec has or requires
specific expectations from implementers then those should be
reflected in conformance statements". [1]

>> IETF-certified or IETF-approved implementations it must be
>> even more precise than this document. As it is, it doesn't
>> fill any obvious purpose.
> 
> The document is aspirational, not contractual. It seems
> perfectly reasonable to ask implementers (whether a
> profit-making company, an open-source community, or an
> individual) to accept ongoing responsibility for their code.

And yet...

We know that many of the people who find an RFC, e.g., by a web
search looking for specifications in a particular area or by a
reference from elsewhere are extremely unlikely to browse the
RFC collection looking for a document about "Expectations of
implementers...".  The people who are active in the IETF
probably don't need this document.  If they do, and its content
is really necessary, it probably belongs in the Tao, as others
have suggested.

As it is and for an external audience, it is likely to be mostly
useful for whining, e.g., in conversations like:

(i) IETF: You and your implementation didn't live up to our
expectations.
    Implementer: What expectations?
    IETF: Read RFC 9999.
    Implementer: How was I expected to find that?

(ii) IETF: You and your implementation didn't live up to our
expectations.
    Implementer: So?

And perhaps worse (if the document is actually spotted):

(iii) IETF: Why did you create a completely new protocol for
that purpose rather than following our fine, high-quality,
consensus standard?
    Implementer: We, and our lawyers, looked through your
"expectations", decided they constituted an attempt at a
shrink-wrap style license that encumbered the IETF spec and that
we were therefore better going our own route.

Note that the third hurts interoperability.

A few additional observations if it is determined to publish
this anyway (which, as is probably obvious, I do not favor).
Note that several of these overlap with Adrian's comments, with
which I generally agree:

(1) The draft (-02) tells readers that they are expected to
conform to the BCPs but that they may need "to look through the
BCP index to find related
BCPs".   That makes another expectation of implementers that
they be able to perform a treasure hunt for materials we just
admitted require some deduction to find. using tools the
document specifies only by hand waving, and get it right.  Seems
unlikely to me.

(2) This is mostly a nit, but, while appealing to RFC 760 as an
authority is arguably non-normative because the text is quoted,
it is a bad choice of reference for several reasons including:
760 has been obsoleted by other documents, it has a status of
"unknown", and a present-day reference to a document whose title
describes it as a "DoD Standard" may not be wise in the
present-day political climate of which the author, as IAB Chair,
is only too aware.  FWIW, the rule is repeated, although stated
differently, in Section 1.2.2 of RFC 1122 and RFC 1123, both of
which are full Standard Applicability Statements without any of
those three issues.  As SM points out, the discussion in RFC
1122/1123, a discussion that is not present in 760, is very
important and should be identified.   In addition to the article
he indirectly cites, the IETF has had very significant
experience with sender-implementers claiming that they can do
astonishingly non-conforming (and stupid) things on the grounds
that the receiver is obligated to be robust and liberal in
acceptance.

(3) It is very likely that, with 1122/1123 as premier examples
for the technical specifications they interpret, the document
should call out relevant Applicability Statements as well as
BCPs. Indeed, paralleling discussions that I gather have been
going on in the IESG, unless BCPs really represent established
Best Practices for technical specs rather than BCP-author
instincts or preferences, this document should be citing those
Applicability Statements as exist in preference to BCPs.

(4) I note that several reference implementations that have been
very important to the development of the Internet and/or the
specific protocols involved would not have conformed to this set
of "expectations".   I note that several of them were funded by
various agencies specifically as one-shot (or one shot plus a
very restricted period for iterations) deals.  If the IETF's
stated expectation were really that people should not implement
unless they are committed to maintain, we would be discouraging
such implementations and the support for them.  Similar
observations could be made about a lot of other implementations
created in research settings: few, if any, research managers
would be willing to commit to something that carried obligations
for long-term maintenance.   That would, IMO, be a really nasty
side-effect of publishing a document like this.

I'll make some constructive suggestions about how to solve the
presumed problem to which this is addressed in a separate note.

best,
   john

[1] If the language of RFC 2119 prevents writing such
conformance statements then, IMO, 2119 needs to be fixed.

---

From: John C Klensin <[email-masked]>
Date: Mon, 12 May 2014 10:46:38 -0400

Suppose there is a problem here that needs solving, i.e., that
this document is not just part of some external agenda about
which the IETF has not been informed (given the normative
reference to RFC 2860, I can imagine it might be).  Then I
suggest that...

There are real requirements that we need to have people follow
to ensure interoperability.  It is possible that some of those
requirements, like the requirement to maintain, really are
generic although I argued in my prior note that the maintenance
one would likely reduce the number of implementations,
especially reference implementations, resulting in some harm and
little if any gains.  Others, including whatever is said in
Applicability Statements and BCPs that are based on consensus of
actual experience and practice, are much more document-specific.

If such requirements apply to a given technical specification,
then an offspring of this document should be created, given
normative status as a BCP or, preferably, as a generic
Applicability Statement, and then a normative reference to it
should be incorporated into the RFC boilerplate or 2119-like
language for technical specifications and other relevant
documents.  That would solve the problem of how implementers and
others are supposed to find this sort of statement.

For particular specifications, this document indirectly opens up
some longstanding (and interrelated) problems, e.g.,:

(i) When a standard (with or without an STD number) consists of
multiple technical specifications, maybe an applicability
statement or two (which may precede some of the technical
specs), and maybe one or more BCPs, what actually is the
standard?  What extensions are considered mandatory, which ones
optional, and, since we aren't consistent about deprecating old
cruft, which ones have quietly become Not Recommended. [1]

(ii) What, actually, is a conformance clause in our world?  RFC
2119 says that both SHOULD and MUST are used to describe things
that are important for interoperability.  The present document
adds a lot of generic requirements that are claimed to be
important for interoperability.  Other comments have identified
the importance to some protocols of a doctrine of "fail early"
to improve robustness against, e.g., careless implementations
even if the relevant provisions are not strictly needed for
interoperability.  Some of the normative language in RFCs 1122
and 1123 is consistent with that model, 2119's definitions
probably are not.  And this draft essentially makes all BCPs
that have impact on a particular technical spec mandatory to
implement/support.  My sense is that many BCPs, including some
that are about operations rather than implementation, have been
approved with no community intent that they be considered
mandatory and that, had that been the intent, the consensus (or
at least the level of scrutiny) would have been much different.
If we intend to make support of a BCP associated with a given
technical spec mandatory, it should, IMO, be done on a
case-by-case rough consensus basis, not as a side effect of a
document that appears to promote all relevant BCPs to mandatory
status [2].

The IETF has had many discussions about how to solve the above
problems, several of which would also provide authoritative
forward reference from earlier specs to later changes, advice,
and description.

One is represented exactly by RFC 1122 and 1123: publication of
Applicability Statements that draw the pieces of a conceptual
standard together and bind it to implementation and operational
advice.

A second (extended a bit from the last discussion I remember)
would have us use STD numbers more broadly and aggressively,
assigning them when (or before) Proposed Standards are approved
and using them to group supplemental documents, including
appropriate BCPs and Informational implementation reports, as
well as the technical specs themselves. [3]

THe third is to create new documents that actually describe what
is in a standard and what the associated implementation and
operational advice or norms are as appropriate.  Such a document
could also serve as a sort of AS because it could identify the
difference between mandatory, recommended, and optional
specification components as thinking about those evolved [4].

The IESG has declined to consider any of those ideas, at last in
part because all of them would involve additional work.  But I
think that the problems that presumably led to the current draft
make that work important enough that we should find the
resources to do something along those lines.   If the problems
are not important enough to justify the effort to do things at
least reasonably well, then I suggest that they aren't important
enough to justify publishing a generic document that is unlikely
to be seen by a large fraction of its most important audience
and that is likely to have bad side-effects.

   best,
     john

[1] I note in passing that the new IESG practice of documenting
decisions to move a technical specification to Historic may
require that the sort of document search the present (-02) draft
implies needs to extend, not just to the BCP Index, but to the
RFC Index/Database and the whole history of IESG actions on
Historic and, in many cases, updated or obsoleted, documents.

[2] This also raises interesting questions when a document is
proposed as a BCP whose associated technical specifications are
not IETF-stream documents that are appropriate for normative
references and over which IETF has change control.

[3] See draft-klensin-std-numbers and draft-otis-newtrk-rfc-set
for very different takes on this.

[4] This idea was developed by a WG in
draft-ietf-newtrk-repurposing-isd although a contemporary
version of the ideas would undoubtedly be different.

---

From: Avri Doria <[email-masked]>
Date: Mon, 12 May 2014 10:53:17 -0400

Hi,

I think it is useful, not only for the new protocol designer, but for
those outside of the IETF, who are trying to understand how the IETF
does what it does.

Having said that, I do think that offering a bit more specificity, as
suggested by Adrian and Brain, would be good.

I would like to see it published.

avri

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 13 May 2014 08:44:26 +1200

On 12/05/2014 19:40, Nikos Mavrogiannopoulos wrote:
> THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 

Of course. Most open source code comes with such a disclaimer,
and so do RFCs. (It's an indirect reference now, but for years all
RFCs contained text starting "This document and the information contained
herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES...").

We're not talking about legalities though. We're talking about
voluntary behavior.

    Brian

---

From: Eric Rosen <[email-masked]>
Date: Tue, 13 May 2014 12:59:40 -0400

I find this document a bit puzzling.

Obviously, the notion of "expectation" is not meant to be descriptive.  No
one really expects implementors to meet all the "expectations", and often
they do not.  

So the notion of "expectation" must be intended to be prescriptive, roughly
equivalent to "obligation".  However, there does not seem to be any legal
obligation to do all the things the document lists.  And it would be rather
far-fetched to claim that failing to follow a protocol specification is
immoral or unethical.

The document would be less puzzling if it gave up all talk of "expectations"
or obligations and simply said "if you want to maintain interoperability
with other implementations, here are things that would be wise to do".  If
that's all that is really meant.  Even so, it has the feel of the sort of
lecture that parents like to give their teenagers: "if you live in my house
you have to follow my rules".  And it will have the same effect (i.e., no
effect). 

I also think the document has a few serious mistakes.

For instance:

    "Follow the protocol specification"

Well, that sounds harmless enough, but sometimes the protocol specification
has errors (sometimes of omission), and the errors are known to the
implementors.  Are they supposed to implement the errors?

"Follow the standard" is ITU-T stuff, and leads to conformance testing; "do
what you have to do in order to interoperate" is much more in the IETF
spirit.  

     "IETF standards are voluntary standards.  No one is required to
     implement them.  Implementation is a choice."

Sorry, most implementors are not volunteers, but are being paid to implement
the protocols.  Often there is no real choice involved.

Or by "implementor", does the document mean "whoever is paying to have the
implementation done", rather than "whoever does the implementation"?  In
that case, "the implementor" would often be a company.  And the individuals
doing the implementation will do what is necessary to accomplish the
company's goals, even if this deviates from the advice given in this
document.  

   "An implementer should plan to maintain their
   implementation.  It is not sufficient to do an initial implementation
   of the protocol.  One needs to apply changes as they come out."

While this may be generally good advice for any computer programmer, it
obviously does not apply to individual implementors who move on to other
tasks or other companies.  As for whether a company needs to maintain its
implementation in this way in any particular case, well, that depends on
business considerations.  

There are also a few things that are not very clear.

   "The Internet is composed of networks operated by a great variety of
   organizations, with diverse goals and rules.  By following the BCPs,
   implementers, operators, and administrators are able to provide a
   common experience when using the protocol, regardless of their point
   of attachment to the Internet."

What does this even mean, "a common experience"?  And what does that have to
do with BCPs?  And what if a common experience is not desired?  And what if
changes in technology or business conditions make the BCPs less relevant
(assuming they were ever relevant to being with)?

This is a good example of the platitudinous nature of the document.    There
are numerous reasons why IANA practices are not always followed in practice:

- An implementor might not know the proper IANA processes.

- An implementor might know the processes, but think they are too
  cumbersome, when it's much easier just to define a constant in a header
  file.  

- If the registration policy is "standards action" or any other policy that
  requires IETF or IESG review, codepoint allocation can become enmeshed in
  politics and/or business rivalries.  There are circumstances in which one
  really has no other choice than to follow the rule that "it is better to
  apologize later than to ask permission first".

It's not that useful to say "just follow the process" without saying
anything about the problems that prevent the process from being followed in
many cases.

So overall I'd say that this document is not worth publishing, but if it is
published it should be made clear that it is about "what you have to do to
maximize the chances of interoperability" than it is about "what you have to
agree to before you are allowed to implement".