Subject: Criteria for removing "unused" features

From: Dave Crocker <[email-masked]>
Date: Mon, 16 Jul 2012 13:26:04 -0700

Howdy.

This note is a preliminary query about a continuing point of IETF 
debate, namely the criteria for removing features from a specification, 
when there is a -bis revision effort.

We have little documented guidance about the issue.  This encourages 
inconsistency or, worse, semantics debates.  I'm sending a note to 
wgchairs, before wider IETF circulation, to seek discussion in a 
smaller-but-experienced group.  The topic has currency for a number of 
us in this group.

I'm hoping folks can converge on a consistent view about criteria.

RFC 6410 moved the IETF to two standards levels.  It includes:

   "the change will result in the
    incorporation of lessons from implementation and deployment
    experience, and recognition that protocols are improved by removing
    complexity associated with unused features."

Note the word "unused".  While the text nicely cites the benefit of 
reducing complexity -- and since it's not the focus of the document, 
there isn't a discussion of why reduced complexity is a benefit -- it 
does not provide any concrete basis for determining that something is 
"unused".

An example of a semantics debate is whether its presence in a charter 
means that there must be literally no software supporting the feature or 
whether there must merely be an extremely small amount of software. It 
also raises the question of whether there the requirement is for 
absolutely no field operation that exercises the code or whether there 
can be only a small amount.

Absent clarification, the word invites debate about development versus 
operation, and mathematically perfect zero or statistically meaningful 
"little".

Thoughts?

d/
-- 
  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net

---

From: Leif Johansson <[email-masked]>
Date: Mon, 16 Jul 2012 22:52:58 +0200

On 07/16/2012 10:26 PM, Dave Crocker wrote:
> debate, namely the criteria for removing features from a
> specification, when there is a -bis revision effort.
> 
> We have little documented guidance about the issue.  This
> encourages inconsistency or, worse, semantics debates.  I'm sending
> a note to wgchairs, before wider IETF circulation, to seek
> discussion in a smaller-but-experienced group.  The topic has
> currency for a number of us in this group.
> 
> "the change will result in the incorporation of lessons from
> implementation and deployment experience, and recognition that
> protocols are improved by removing complexity associated with
> unused features."
> 
> Note the word "unused".  While the text nicely cites the benefit
> of reducing complexity -- and since it's not the focus of the
> document, there isn't a discussion of why reduced complexity is a
> benefit -- it does not provide any concrete basis for determining
> that something is "unused".
> 
> An example of a semantics debate is whether its presence in a
> charter means that there must be literally no software supporting
> the feature or whether there must merely be an extremely small
> amount of software. It also raises the question of whether there
> the requirement is for absolutely no field operation that exercises
> the code or whether there can be only a small amount.
> 
> Absent clarification, the word invites debate about development
> versus operation, and mathematically perfect zero or statistically
> meaningful "little".

Its often very hard to tell the difference between "very little" and
"widely deployed but only in small communities", especially when one
is wearing the "internetscale-spectacles" (as we tend to in the IETF).

Maybe a better yardstick is if the spec has lead to interoperable
implementations of the feature, rather than attempting to measure
its impact.

	Leif

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 16 Jul 2012 17:14:45 -0400

Dear colleagues, Dave:

On Mon, Jul 16, 2012 at 01:26:04PM -0700, Dave Crocker wrote:

> inconsistency or, worse, semantics debates.  

<petpeeve>I'm not actually sure why a semantic debate in this case
would be worse, since as a matter of fact this issue is one of
semantics.</petpeeve>  I take your mail to be an attempt to
resolve the semantics of "unused" once and for all.  

> amount of software. It also raises the question of whether there the
> requirement is for absolutely no field operation that exercises the
> code or whether there can be only a small amount.
> 

In one example with which I am familiar (SPFBIS), there is an
additional matter at work: the charter was, on purpose, scoped very
narrowly.  I therefore would like to suggest that any attempt to
answer the meaning of "unused" once and for all neglects the context
in which the term is employed.  

In the case I'm thinking of, the overall thrust of the charter is to
move a protocol from the experimental track to the standards track
with as few changes as possible.[1] Future work might be chartered to
advance the resulting protocol along the standards track, at which
time additional changes might be contemplated.  In that context, the
bar for showing a feature is "unused" is -- or in my opinion anyway
ought to be -- somewhat high.  Evidence of deployment of the feature
is _prima facie_ evidence that the feature is used, even if it is used
rarely.  An ugly feature for which we have evidence of actual use,
even if we think that use is stupid or foolish, is still a used
feature.  I don't see how it would help to pretend that it's not
really used, just because it's not used by our friends.

A similar bar might not exist in other cases.  Perhaps, for instance,
deployment of the protocol is narrow and recent, and it is practical
to ask everybody who's ever been involved with the protocol what has
happened.  Perhaps the experiment has actually run according to its
original estimates, and hasn't become a part of production code in
large sites on the Internet.  Perhaps the feature is obviously
redundant.  All of these seem to me to be factors that need to be
weighed by the WG in making its decision as to what is going to count
as "unused".  Unfortunately, that boils down to a semantic question.
Again in the case of SPFBIS, the debate about the meaning of "unused"
revealed an important feature of the use of RRTYPE 99, which was that
the protocol was actually broken as written.  This meant that, even if
the RRTYPE was used, it could not be used reliably and interoperably;
the WG made its determinations accordingly.  I'd be reluctant to
constrain the freedom of other WGs in attacking similar problems.

Best,

A

[1] One response to this state of affairs might be to object that the
original charter was in error: if changes are to be made to a
protocol, it seems that it would be preferable to introduce them when
moving from experimental to standards track, partly on the basis of
experience from the experiment.  While I see the merits of that
argument in the abstract, it appears to me that in the example I'm
thinking of the conclusion was otherwise.  In any case, it's
irrelevant to the meaning of "unused".

-- 
Andrew Sullivan
[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 16 Jul 2012 17:32:32 -0400

Trying again, from my subscribed address.  Sigh.

Dear colleagues, Dave:

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "DRAGE, Keith (Keith)" <[email-masked]>
Date: Tue, 17 Jul 2012 01:20:45 +0200

I'd offer two thoughts:

1)	In order to advance to the new level (which is I assume what you are trying to do) you need to test all the features for interoperability. If the featured is unused, then you will not be able to do the test, because no one will have implemented. So your search for implementations to cover the complete set of features may already identify this. If you cannot perform such tests, then you either have to assume that the document is not ready to advance, or that it is "unused".

2)	Is removing a feature effectively putting it in the same status as "historical" when applied to an entire RFC? RFC 2023 gives very little input on what criteria to apply to declare an RFC historical, but at least it has happened in the past, and may provide some input.

Keith

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Dave Crocker
> Sent: 16 July 2012 21:26
> To: Working Group Chairs
> Subject: Criteria for removing "unused" features
> 
> Howdy.
>

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 17 Jul 2012 09:49:23 +1000

In HTTPbis, we aren't removing features, but we will sometimes deprecate them (i.e., offer advice cautioning against their use). In something as widely deployed as HTTP, there aren't any completely unused features.

Cheers,

On 17/07/2012, at 6:26 AM, Dave Crocker wrote:

> Howdy.
> 
> This note is a preliminary query about a continuing point of IETF debate, namely the criteria for removing features from a specification, when there is a -bis revision effort.
> 
> We have little documented guidance about the issue.  This encourages inconsistency or, worse, semantics debates.  I'm sending a note to wgchairs, before wider IETF circulation, to seek discussion in a smaller-but-experienced group.  The topic has currency for a number of us in this group.
> 
> Note the word "unused".  While the text nicely cites the benefit of reducing complexity -- and since it's not the focus of the document, there isn't a discussion of why reduced complexity is a benefit -- it does not provide any concrete basis for determining that something is "unused".
> 
> An example of a semantics debate is whether its presence in a charter means that there must be literally no software supporting the feature or whether there must merely be an extremely small amount of software. It also raises the question of whether there the requirement is for absolutely no field operation that exercises the code or whether there can be only a small amount.
> 
> Absent clarification, the word invites debate about development versus operation, and mathematically perfect zero or statistically meaningful "little".
> 
> Thoughts?
> 
> d/
> -- 
> Dave Crocker

--
Mark Nottingham   http://www.mnot.net/

---

From: <[email-masked]>
Date: Mon, 16 Jul 2012 19:53:39 -0400

> 1)	In order to advance to the new level (which is I assume what you are
> trying to do) you need to test all the features for interoperability. If the
> featured is unused, then you will not be able to do the test, because no one
> will have implemented. So your search for implementations to cover the
> complete set of features may already identify this. If you cannot perform such
> tests, then you either have to assume that the document is not ready to
> advance, or that it is "unused".

Not exactly ... excerpting selectively from RFC 6410 that defines the
new level:

   The IESG, in an IETF-wide Last Call of at least four weeks, confirms
   that a document advances from Proposed Standard to Internet Standard.
   The request for reclassification is sent to the IESG along with an
   explanation of how the criteria have been met.  The criteria are:

   (3) There are no unused features in the specification that greatly
       increase implementation complexity.

3.2.  Requirement for Interoperability Testing Reporting

   Testing for interoperability is a long tradition in the development
   of Internet protocols and remains important for reliable deployment
   of services.  The IETF Standards Process no longer requires a formal
   interoperability report, recognizing that deployment and use is
   sufficient to show interoperability.

So, unused features are not a bar to advancement, but the IESG owns the
judgment call on whether they're ok vs. "greatly increase implementation
complexity."  OTOH, the burden of explaining why they should remain
(correctly, IMHO) falls on the WG or people making the request for
reclassification.

Thanks,
--David

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of DRAGE, Keith (Keith)
> Sent: Monday, July 16, 2012 7:21 PM
> To: Dave Crocker; Working Group Chairs
> Subject: RE: Criteria for removing "unused" features
> 
> I'd offer two thoughts:
> 
> 1)	In order to advance to the new level (which is I assume what you are
> trying to do) you need to test all the features for interoperability. If the
> featured is unused, then you will not be able to do the test, because no one
> will have implemented. So your search for implementations to cover the
> complete set of features may already identify this. If you cannot perform such
> tests, then you either have to assume that the document is not ready to
> advance, or that it is "unused".
> 
> 2)	Is removing a feature effectively putting it in the same status as
> "historical" when applied to an entire RFC? RFC 2023 gives very little input
> on what criteria to apply to declare an RFC historical, but at least it has
> happened in the past, and may provide some input.
> 
> Keith
>

---

From: Dave Crocker <[email-masked]>
Date: Tue, 17 Jul 2012 10:15:37 -0700

On 7/16/2012 1:52 PM, Leif Johansson wrote:
> implementations of the feature, rather than attempting to measure its
> impact.

A word like "unused" is relatively common for -bis working group
charters.  I believe that charters for such working groups are typically
worded to be relatively restrictive, to avoid a working group's possible
tendency towards too much change, ignoring effects on the installed base.

In querying the group, I am hoping that we can work with that type of
word, since it represents established IETF practice.  Rather than
considering a change to the basic practice of wording charters, I'm
hoping to get a stable interpretation of meaning.

I believe an essential point behind a word like "unused", and especially
for a -bis effort to revise a specification that has been deployed into
production service, is what the field /use/ of the feature has been.

Note that we no longer have Draft standards status.  While -bis efforts
have often been for deployed specifications, it is now essentially
guaranteed.  The case of revising a specification that has merely
undergone 'testing' is essentially removed from the IETF standards
process and, I think, therefore removed from likely wg chartering language.

When talking about possible changes for a specification that has not
received much or any field operation, there is greater flexibility.

When talking about possible changes for a specification that has an
installed base of Internet users, things are more challenging, primarily
from a concern about the effect on the installed base of users.  So
that's the case I meant to focus on with my query.

My own preference would be a group consensus that is not subtle or
contingent.  That is, one that interprets "unused" in a reasonably
consistent manner across a wide range of cases.  There might be
additional language the would constrain or expand the charter's scope
and flexibility, but I would hope that it would not change the IETF
definition of "unused"...

As a simple example, I would hope that "unused" is not confused with (or
by) "broken".  Something can be broken and used.  Something can be
entirely functional but unused.  It's really an orthogonal construct.

So I would hope that a charter intending to cover both concerns would
explicitly distinguish between features that do not function as intended
(broken) and those that have not gained traction (unused).

On 7/16/2012 4:49 PM, Mark Nottingham wrote:
> In HTTPbis, we aren't removing features, but we will sometimes
> deprecate them (i.e., offer advice cautioning against their use). In
> something as widely deployed as HTTP, there aren't any completely
> unused features.

I think that deprecating vs. removing is another independent issue.  It
arises once one has satisfied some criteria that classes a feature as a
candidate for removal or deprecation.

An example of a criterion would
be... that it is unused.

Again, I'm hoping we can agree on a definition of unused that neither
depends upon context nor upon what action will be taken.

d/

---

From: Dave Crocker <[email-masked]>
Date: Tue, 17 Jul 2012 10:20:07 -0700

> resolve the semantics of "unused" once and for all.

Indeed, I meant that qualifying text a bit ironically.

Since communications relies on shared meaning and semantics means 
meaning, I too consider it a core issue.

However what most people mean when they say that someone is arguing 
semantics is that they are quibbling about small differences.

That most people don't understand (or intend) the correct meaning of 
semantics is the irony.

As for why I meant "worse", I consider quibbling worse than worrying 
about specification inconsistency because worrying about inconsistency 
is productive.

d/

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 17 Jul 2012 13:50:03 -0400

> contingent.
 
> depends upon context nor upon what action will be taken.

The above is easily achieved.  We can do it by fiat.  If and only if
there is not at least one observed occasion of the deployment of a
feature on the Internet, then the feature is unused.  This meets all
your criteria above, though it has two problems:

    1.  It depends on the observers.  If nobody we know has looked
        (this definition doesn't require that anyone has), then the
        feature meets the test for "unused" even if it's in wide use.

    2.  It sets the bar for being used lower than your message
        yesterday suggested you wanted it to be set.

To solve (1), of course, we could appeal to good behaviour on the IETF
participants' part, the open standards process, and assumptions about
our ability to attract operators of standards into our processes.
Even if I thought those were good points (and, frankly, there are very
few operators at the IETF), that's all contingent.

To solve (2), of course, we could just change the value of "at least
one" to some other value.  But it seems unlikely that we are going to
come up with a number that will satisfy there, because the right
number surely needs to depend on judgements about actual
deployments. For instance, "at least one person" doing something with
an EPP extension probably qualifies as "use", because the EPP-using
population is small and narrow in diversity.  "At least one person"
doing something with DNS is a little harder to defend.  We moved A6 to
HISTORIC even as A6 queries continued to land at the root servers.

I am having a hard time understanding what problem you are trying to
solve.  Consistency is nice, sure, but I wonder whether we're into
hobgoblin-land here.

Best,

A

-- 
Andrew Sullivan
[email-masked]