Subject: When is an idea a good idea?

From: Spencer Dawkins <[email-masked]>
Date: Tue, 28 Jan 2014 14:14:33 -0600

Disclaimer - I'm one of the ADs who will be balloting your documents 
until at least March 2015, but I'm only one of 15 ADs, and haven't 
talked about this with the rest of the IESG.

The past week has brought me yet another example of an idea that could 
be a good idea, but not under every possible set of conditions, being 
evaluated as if we had to decide whether it was a good idea in all 
cases, or a bad idea. This might sound familiar to you, but if it does, 
let's not talk about that specific case.

The longer (and more painfully) we talk during Last Call, the more 
details we unearth that help me to understand what document 
authors/working groups are thinking, and that's good, but if it was less 
painful, that would be even better.

http://tools.ietf.org/html/rfc2026#section-3.2 describes Applicability 
Statements, as distinct from Technical Specifications, and says (among 
other things):

    An Applicability Statement specifies how, and under what
    circumstances, one or more TSs may be applied to support a particular
    Internet capability.

...

    An AS may describe particular methods of using a TS in a restricted
    "domain of applicability", such as Internet routers, terminal
    servers, Internet systems that interface to Ethernets, or datagram-
    based database servers.

Speaking only for myself, I don't expect Proposed Standards to be 
perfect, and to work perfectly in every situation, but if a 
specification doesn't describe any limits on applicability, I'm going to 
be evaluating it as if it will be used on the open Internet (that's what 
the "I" in "IETF" stands for).

If a draft says it's only intended to be used within an IP subnet, I'd 
evaluate it differently. I might ask that the authors/working group 
consider what the TTL should be set to, so that what starts out within 
an IP subnet stays within an IP subnet, but (to use one actual example) 
we wouldn't be arguing about whether it's OK to send 32K max-length 
packets over an arbitrary Internet path at line rate without getting any 
feedback about path capacity - there are probably paths where that would 
work, and there are definitely paths where it would not.

If a draft says it's only intended to be used on provisioned, managed 
internetwork links subject to SLA monitoring, I'd evaluate it differently.

If a draft says "this is a hack, intended for use on old hardware that 
can't do $X", I'd evaluate it differently.

There are other limited-use scenarios that would make me evaluate a 
draft differently.

None of this is a guaranteed pass, but it would help me a lot. It would 
likely help other reviewers, too.

So, if you don't intend for your draft to be used on the global 
Internet, please say so! As per 
http://tools.ietf.org/html/rfc2026#section-3.3, it's not necessary to 
put an Applicability Statement in a different draft; just a section that 
says (another actual example) "this has been tested using these 
parameters on a lightly-loaded LAN, and it works there", that is more 
helpful than a tug of war(*) about whether something is a good idea or a 
bad idea in all situations, in front of a live studio audience.

Thanks to Alia Atlas for nudging me to think about this more.

Spencer

p.s. If you have feedback about what I'm thinking, I'd love for you to 
share it, whether on this list, privately to me, to the 2015 Nomcom, or 
to other Nomcom-eligible IETF participants when you're asking them to to 
sign the recall petition ;-)

(*) http://en.wikipedia.org/wiki/Tug_of_war

---

From: Hector Santos <[email-masked]>
Date: Tue, 28 Jan 2014 16:52:35 -0500

I think the IETF needs a new status for document submissions. 
Something more advanced than experimental and informational but less 
than a Proposed Standard (PS) perhaps called Proposed Method (PM).  A 
PM will need towards an IM (Internet Method).

When something is written as a "standard" there tends to be a higher 
bar to do something different. It provides a sense of final closure 
and resolution and this is where many debates, conflict of interest, 
are based, where IETF Appeals are more probable.  Who cares if a 
proposal is informational or experiment?  Not really strong enough and 
too risky for adoption. A "standard" has a higher bar to reach but 
also to change as well.

I believe a PM will help with the acceptance of proposals that may fit 
in this in between category where it may not be considered a standard 
material, yet, it offers some level of technology that is beyond being 
informational or experimental, i.e. a segment of the market place is 
using it, technology leader wishes to document a proprietary method as 
an open and public domain method.

I believe a PM may also help alleviate the "good vs bad" debates, 
allow for greater adoption explorations.   It tells the industry that 
we found something useful, already used in practice, try it yourself, 
but its still not something the IETF to be the "standard" method.

Anyway, the PM idea seems to come back more and more as you see more 
of these "half-baked" proposed standards when they seems to be more 
experimental or informational in nature.   You have good ideas out 
there that do need to be documented.  But you don't need to make them 
Proposed Standards which has a tremendously high bar and higher source 
for major debates, especially when there are conflicts.

--
HLS

On 1/28/2014 3:14 PM, Spencer Dawkins wrote:
> cases, or a bad idea. This might sound familiar to you, but if it
> does, let's not talk about that specific case.
>
> authors/working groups are thinking, and that's good, but if it was
> less painful, that would be even better.
>
> specification doesn't describe any limits on applicability, I'm going
> to be evaluating it as if it will be used on the open Internet (that's
> what the "I" in "IETF" stands for).
>
> an IP subnet stays within an IP subnet, but (to use one actual
> example) we wouldn't be arguing about whether it's OK to send 32K
> max-length packets over an arbitrary Internet path at line rate
> without getting any feedback about path capacity - there are probably
> paths where that would work, and there are definitely paths where it
> would not.
>
> None of this is a guaranteed pass, but it would help me a lot. It
> would likely help other reviewers, too.
>
> put an Applicability Statement in a different draft; just a section
> that says (another actual example) "this has been tested using these
> helpful than a tug of war(*) about whether something is a good idea or
> a bad idea in all situations, in front of a live studio audience.
>
> share it, whether on this list, privately to me, to the 2015 Nomcom,
> or to other Nomcom-eligible IETF participants when you're asking them
> to to sign the recall petition ;-)
>

---

From: Spencer Dawkins <[email-masked]>
Date: Tue, 28 Jan 2014 16:34:23 -0600

Just to be clear, I'm certainly not opposed to exploring this question, 
but I do think it's orthogonal to my post ("how do we review 
specifications that might not be applicable everywhere?" as opposed to 
"how do we get some deployment experience before deciding something is a 
proposed standard?").

So definitely, thanks for changing the subject line, and perhaps you 
could knock out an Internet-Draft proposing Proposed Methods?

Spencer

---

From: Brian E Carpenter <[email-masked]>
Date: Wed, 29 Jan 2014 12:11:34 +1300

Hi Spencer,

I think there's a bit more to be said, though.

The goal is to make the Internet work better, not to make
private networks work better. Actually RFC 3935 defines
the IETF's scope as: "protocols and practices for which
secure and scalable implementations are expected to have
wide deployment and interoperation on the Internet, or to
form part of the infrastructure of the Internet."

So if there are two ways of doing something, one of which
works globally and one of which only works locally, the choice
is a no-brainer. If a local-only solution comes up to the IESG,
I think the first question is "Why isn't there a globally
applicable solution to this problem?". Then the second question
is "Why is this on the standards track, rather than being an
informational document for the record?". And the third question
is "Why is this better as an IETF document rather than an
Independent Submission?".

There may be perfectly good answers to those questions, but
IMHO they need to be asked.

Regards
   Brian

---

From: "Dearlove, Christopher (UK)" <[email-masked]>
Date: Wed, 29 Jan 2014 10:29:02 +0000

I'm not sure more categories is what we want.

I have observed all of the following (not at the same time, though some were) from people not familiar with the IETF and RFCs, although invested in or using RFCs:
- Assuming that all RFCs are standards.
- Assuming that only Standards matter, that Proposed Standards and Draft standards aren't important.
- Assuming that Draft Standard is a category below Proposed Standard.
- Recognising that there are standards and not-standards, but not distinguishing in any way between Standard, Draft Standard and Proposed Standard (calling all of them Standards).
- Just plain confused.

(And the concept of BCP is probably completely unknown to anyone thinking any of the above.)

Another category isn't going to help here.

-- 
Christopher Dearlove
Senior Principal Engineer, Communications Group
Communications, Networks and Image Analysis Capability
BAE Systems Advanced Technology Centre
West Hanningfield Road, Great Baddow, Chelmsford, CM2 8HN, UK
Tel: +44 1245 242194 |  Fax: +44 1245 242124
[email-masked] | http://www.baesystems.com

BAE Systems (Operations) Limited
Registered Office: Warwick House, PO Box 87, Farnborough Aerospace Centre, Farnborough, Hants, GU14 6YU, UK
Registered in England & Wales No: 1996687

-----Original Message-----
From: ietf [mailto:[email-masked]] On Behalf Of Hector Santos
Sent: 28 January 2014 21:53
To: Spencer Dawkins
Cc: IETF discussion list
Subject: Make it a Proposed Method rather than Proposed Standard [Re: When is an idea a good idea?]

----------------------! WARNING ! ---------------------- This message originates from outside our organisation, either from an external partner or from the internet.
Consider carefully whether you should click on any links, open any attachments or reply.
Follow the 'Report Suspicious Emails' link on IT matters for instructions on reporting suspicious email messages.
--------------------------------------------------------

I think the IETF needs a new status for document submissions. 
Something more advanced than experimental and informational but less than a Proposed Standard (PS) perhaps called Proposed Method (PM).  A PM will need towards an IM (Internet Method).

When something is written as a "standard" there tends to be a higher bar to do something different. It provides a sense of final closure and resolution and this is where many debates, conflict of interest, are based, where IETF Appeals are more probable.  Who cares if a proposal is informational or experiment?  Not really strong enough and too risky for adoption. A "standard" has a higher bar to reach but also to change as well.

I believe a PM will help with the acceptance of proposals that may fit in this in between category where it may not be considered a standard material, yet, it offers some level of technology that is beyond being informational or experimental, i.e. a segment of the market place is using it, technology leader wishes to document a proprietary method as an open and public domain method.

I believe a PM may also help alleviate the "good vs bad" debates, 
allow for greater adoption explorations.   It tells the industry that 
we found something useful, already used in practice, try it yourself, but its still not something the IETF to be the "standard" method.

Anyway, the PM idea seems to come back more and more as you see more of these "half-baked" proposed standards when they seems to be more 
experimental or informational in nature.   You have good ideas out 
there that do need to be documented.  But you don't need to make them Proposed Standards which has a tremendously high bar and higher source for major debates, especially when there are conflicts.

--
HLS

> (*) http://en.wikipedia.org/wiki/Tug_of_war

********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************

---

From: Abdussalam Baryun <[email-masked]>
Date: Wed, 29 Jan 2014 11:53:04 +0100

Hi Spencer,

Yes, that is right. I mentioned that many time but still no much
following, this process problem affects the end users of our
protocols. IESG needs to force ADs and WGs to describe very well in
the applicabilitity section, or the use case of such draft-protocol.
How does the IESG evaluate such drafts if there are not much
explainations in the use of the examined protocol? Could we evaluate
IESG (including previous IESG) in its RFC outputs based on your
questions?, Therefore, I suggest your ideas should be presented to
IESG and IETF WGs Chairs, how they are evaluating drafts, what is
their methodology, or Is there any
methodology/idea/format/basic-questions?

Even reviewers should focus on your questions, but I don't think we
can force them as long as we are short with reviewers.

AB

---

From: Abdussalam Baryun <[email-masked]>
Date: Wed, 29 Jan 2014 12:05:07 +0100

Hi Hector,

The most importance is its use, or practice. So could it be a less
than a BCP, as a proposed practice (PIP), so we can use it as an
initial-proposal-BCP or an initial-proposal-standard. As we seen the
draft-farrell-perpass-attack-04 draft that SHOULD be an informational
or SHOULD be less than a BCP but still not in the right submission
path (because as we see there is a missing path).

The IETF needs to focus more on proposed used standards, best
practices, and real use cases. That will focus ideas and proposals in
the right directions.

AB

---

From: Scott Brim <[email-masked]>
Date: Wed, 29 Jan 2014 08:28:29 -0500

Yes. When applicability is declared to have a narrow scope, a draft
deserves *more* attention, to ensure it really will stay scoped and not be
used globally, to ensure it wouldn't actually have global use with some
tweaking, and clearly to ensure there isn't a global approach that could
fit as well with tweaking.

Once upon a time our scope was easy, everyone who participated in the IETF
had common interests. These days we maintain ourselves on a slippery slope,
supporting scoped use cases (there are clear cases where special
application is much better) and struggling against tremendous forces trying
to fragment the Internet. We need to continue to put out extra effort to
help others keep from sliding farther down that slope than they need to.

Scott

---

From: Spencer Dawkins <[email-masked]>
Date: Wed, 29 Jan 2014 09:45:48 -0600

I'm replying to Brian and to Scott here ...

> is a no-brainer. If a local-only solution comes up to the IESG,

I couldn't agree with Brian and Scott more. I'm focusing on "comes up to 
the IESG".

My point is that the sooner someone asks "so, is this really intended 
for deployment on the global Internet?", the better. If working groups 
and authors thought about that before, or even during, WGLC time, I 
would like to think that we would have fewer cases where the question is 
asked for the first time in AD review, or in directorate reviews, or in 
IETF Last Call.

I don't think it's controversial at all that the IETF does better when 
documents don't encounter Late Surprises (tm). If the first time someone 
says "but this is only intended to be used when $whatever" is in an 
email exchange during IETF Last Call, that's a Late Surprise, and the 
chances that the best outcome will happen aren't going up at that point.

As I said in my note, this is my personal suggestion. I'm not asking for 
an Applicability Statement as a mandatory section in all IDs, precisely 
because of Brian's point that the default is "this proposal is suitable 
for deployment on the global Internet". I'm asking working groups and 
authors to consider whether that's true for their proposals, or if I 
should be reviewing with something else in mind.

And as always, I encourage people to Do The Right Thing.

Spencer

---

From: Dave Crocker <[email-masked]>
Date: Wed, 29 Jan 2014 07:57:42 -0800

...
> Internet, please say so!

Spencer,

What an excellent bit of constructive suggestion.

However you've cast it as being for the purpose of guiding IESG 
evaluation, and I think that it really has much broader... 
applicability.  Anyone reading the document needs to know the intended 
scope of use.  For example, obviously an implementer ought to understand 
the operational limitations meant for a specification.

I take your suggestion to reduce to a simple guideline:

      The default applicability for an IETF specification is the 'open' 
Internet.  Any specification intended for more constrained use needs to 
describe the constraints.  One means of achieving this can be an 
"Applicability" section in the specification, with a description of 
intended use.

A potentially larger benefit of this consideration is that it might get 
designers to think a bit more explicitly about the use of their work.

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Scott Brim <[email-masked]>
Date: Wed, 29 Jan 2014 11:15:30 -0500

> "Applicability" section in the specification, with a description of

+1. Where would you put it?

---

From: Spencer Dawkins <[email-masked]>
Date: Wed, 29 Jan 2014 10:16:09 -0600

On 01/29/2014 09:57 AM, Dave Crocker wrote:
> scope of use.  For example, obviously an implementer ought to 
> understand the operational limitations meant for a specification.
>
> Internet.  Any specification intended for more constrained use needs 
> to describe the constraints.  One means of achieving this can be an 

Dave, I think that sounds about right.

Part of what I think about during IESG evaluation is the potential for 
someone to use the proposal in a way that trips over unstated 
assumptions. Seeing that a proposal includes some thoughts about 
intended uses (and why) makes me worry less. Downstream readers and 
implementers are definitely part of the food chain.

>
> A potentially larger benefit of this consideration is that it might 
> get designers to think a bit more explicitly about the use of their work.

That could be lovely ...

Spencer

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 30 Jan 2014 08:46:32 +1300

> +1. Where would you put it?

IMHO it should have been in RFC 1958, and I'm embarassed that
we missed this point back in 1996, because it's so fundamental.
I have a tickle at the back of my brain, because I think there
is text somewhere about how solutions designed for local use
have a nasty habit of appearing on the open Internet and
causing trouble, but I can't find it.

   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 30 Jan 2014 13:32:13 +1300

> causing trouble, but I can't find it.

grep found this in RFC 3365 (BCP 61):

   History has shown that applications that operate using the TCP/IP
   Protocol Suite wind up being used over the Internet.  This is true
   even when the original application was not envisioned to be used in a
   "wide area" Internet environment.  If an application isn't designed
   to provide security, users of the application discover that they are
   vulnerable to attack.

       Brian

---

From: "Fred Baker (fred)" <[email-masked]>
Date: Thu, 30 Jan 2014 05:56:53 +0000

On Jan 28, 2014, at 3:14 PM, Spencer Dawkins <[email-masked]> wrote:

> Speaking only for myself, I don't expect Proposed Standards to be perfect, and to work perfectly in every situation, but if a specification doesn't describe any limits on applicability, I'm going to be evaluating it as if it will be used on the open Internet (that's what the "I" in "IETF" stands for).

Speaking for myself, I expect any solution that exists to be used on the open Internet. This is primarily because history says "they are", but also because many supposedly private networks in fact operate environments that aren't so very different from the open Internet. My reference point for that second factoid-from-my-perspectve is my employer's network, whose network dwarfs anything we had deployed in 1990 and probably what we had deployed in 1995, including managed private interconnection to (I'm told) O(200) other companies and VPN access from computers via VPN (such as I am typing on right now) and from home office networks via VPN. Every company isn't Cisco, but I think you may be surprised how many have similarities - I'd guess that Fortune 1000 enterprise networks have strong structural similarities.

If you wouldn't use a solution, or a mitigated version of a solution, on the open Internet, I'm not so very sure I would recommend it to my IT department or my customers.

---

From: t.p. <[email-masked]>
Date: Thu, 30 Jan 2014 10:03:51 +0000

----- Original Message -----
From: "Spencer Dawkins" <[email-masked]>
To: "Brian E Carpenter" <[email-masked]>
Cc: "IETF discussion list" <[email-masked]>
Sent: Wednesday, January 29, 2014 3:45 PM

Spencer

By this criterion, would MPLS ever have entered the Standards Track?

It is a protocol that works supremely well within the private network of
an ISP; extending it to work across the Internet worldwide is more of a
challenge and, from what I recall, was not contemplated at the time of
RFC3031.

Tom Petch

> >> cases, or a bad idea. This might sound familiar to you, but if it
does,
> >> authors/working groups are thinking, and that's good, but if it was
less
> >> an IP subnet stays within an IP subnet, but (to use one actual
example)
> >> None of this is a guaranteed pass, but it would help me a lot. It
would
> >> put an Applicability Statement in a different draft; just a section
that

---

From: Spencer Dawkins <[email-masked]>
Date: Thu, 30 Jan 2014 08:59:57 -0600

Hi, Tom,

Keeping in mind that my ask was based on what I, personally, would find 
helpful to see in documents and not just revealed during extended Last 
Call discussions as if they were the Dance of the Seven Veils, not 
intended to express opinions about what the IESG then was doing, and 
this is slippery-sloping towards a re-litigation of a couple of decades 
of IETF work ...

( http://en.wikipedia.org/wiki/Dance_of_the_Seven_Veils)

> By this criterion, would MPLS ever have entered the Standards Track?

Disclaimer: I'm actively chasing history on the "I stands for Internet" 
topic, so I may be smarter tomorrow, but this is what I'm looking at today.

I wasn't there (coincidentally, I was working on TSV stuff during the 
early days of tag and label switching), and folks on this list were, but 
I'm given to understand that this was discussed at the time. I've heard 
the statement "Anything that two consenting routers do over a link layer 
is their own business" attributed to Tony Li, and that makes more sense 
if it was half of a discussion, and the other half could be simplified 
to "is not".

RFC 3031 is dated 2001. The IETF mission statement, 
http://www.ietf.org/rfc/rfc3935.txt, was published after that (in 2004, 
but those ideas didn't spring from nothing in December of 2003), and it 
says things like

 >    In attempting to resolve the question of the IETF's scope, perhaps
 >    the fairest balance is struck by this formulation: "protocols and
 >    practices for which secure and scalable implementations are expected
 >    to have wide deployment and interoperation on the Internet, or to

I'm assuming the last part of this paragraph is intended to cover things 
like MPLS, DHCP, and such.

Perhaps there are people on this list who would correct me if I 
misstated this.

> challenge and, from what I recall, was not contemplated at the time of

Indeed. I offer 
http://www.ietf.org/mail-archive/web/ietf/current/msg85344.html for your 
perusal. It's a thread I'm reading with some interest :p

Spencer
Spencer

---

From: Scott Brim <[email-masked]>
Date: Thu, 30 Jan 2014 10:16:54 -0500

Tom: but MPLS could be used anywhere. It isn't designed in a way that using
it would be dangerous.

Fred: but there will always be environments that would benefit from special
optimizations, such as cellular or satellite or high-performance computing.
It's reasonable for us to standardize for the special needs of those
environments too - they need interoperability and they use IP.

Scott

---

From: Michael StJohns <[email-masked]>
Date: Thu, 30 Jan 2014 12:28:17 -0500

Comments at the end.

At 03:14 PM 1/28/2014, Spencer Dawkins wrote:

>So, if you don't intend for your draft to be used on the global Internet, please say so! As per <http://tools.ietf.org/html/rfc2026#section-3.3>http://tools.ietf.org/html/rfc2026#section-3.3, it's not necessary to put an Applicability Statement in a different draft; just a section that says (another actual example) "this has been tested using these parameters on a lightly-loaded LAN, and it works there", that is more helpful than a tug of war(*) about whether something is a good idea or a bad idea in all situations, in front of a live studio audience.
>

To add to your list of possible limiters:

Subnet Scaling:  How large of a subnet will this work with before blowing up?  (mesh routing protocols, multicast security stuff etc).

Field of Use:  What was this designed to do?  What can this do without fear?  What shouldn't this do?  (e.g. DNS being used as a swiss army knife because its a global database implementation, TLS being used as the only security protocol including link layer for mesh).

Layer Violations:  What layer violations are acceptable for this protocol without breaking the model?  Does this assume a specific LLP (e.g. TCP using the IP address in a pseudo-header as part of the header checksum calculation)?

Convention Violations:  What violations of normal IETF and network standards convention does this protocol depend on?  (e.g.  current discussion in TLS about using little-endian on-the-wire representation for Curve22519 suites instead of the normal big-endian).  

Global vs Common vs Limited:  Is this protocol designed to be deployed in all or almost all internet devices (Global e.g. IP, UDP, TCP, DNS), in many devices - generic types (Common HTTP client and server, smtp, pop/imap), or in specific limited numbers or classes of devices (Limited - smart energy protocols, sensor node protocols - things like thermostats, street lights, bridge sensors, security systems).

Static vs Dynamic:  Is this protocol designed for use with clusters of internet nodes that can self-organize (dynamic) and change over time, or for clusters that are set-up/configured and remain relatively stable?

*sigh*  It might be Taxonomy time....

Mike

---

From: Dave Crocker <[email-masked]>
Date: Thu, 30 Jan 2014 09:40:06 -0800

On 1/29/2014 9:56 PM, Fred Baker (fred) wrote:
> Speaking for myself, I expect any solution that exists to be used on the open Internet.

Including DHCP and OSPF?

The point that efforts, which start with a limited usage scope, 
typically expand to the open Internet is an important one.  Moving from 
open Internet to controlled intranet is typically pretty easy, because 
the latter is faster and maybe more reliable.  Moving in the reverse 
direction is typically disastrous.

However the counter-examples I'm suggesting demonstrate that there 
really are legitimate cases of declaring limited scope as part of he 
design effort.

d/

---

From: t.p. <[email-masked]>
Date: Fri, 31 Jan 2014 09:40:34 +0000

----- Original Message -----
From: "Scott Brim" <[email-masked]>
To: "t. p." <[email-masked]>
Cc: "IETF discussion list" <[email-masked]>; "Spencer Dawkins"
<[email-masked]>
Sent: Thursday, January 30, 2014 3:16 PM

> Tom: but MPLS could be used anywhere. It isn't designed in a way that
using

Mmmm ... let me see

Congestion control - nil
Security - nil
Management - not much until the recent ITU-T inspired effort.

Yup, let it rip on the open Internet, no dangers there. I am sure the
IESG would agree:-)

Tom Petch