Subject: RE: Consensus based on reading tea leaves (was: Re: Alternativeformatsfor IDs)

From: "Yaakov Stein" <[email-masked]>
Date: Mon, 2 Jan 2006 18:10:15 +0200

<snip>

We could certainly base declared consensus on other things. My point is
that doing so likely requires a fundamental rethink of IETF process -
simply encouraging the IESG to disregard the current IETF process BCPs
on a case-by-case basis does not point me in any direction I'm
comfortable with, and nicely justifies anyone who wants to appeal the
decision as a process violation - not likely to make change happen MORE
quickly...

[YJS] That is why we proposed that the draft be part of a process change
process.

We purposely wanted to avoid the question that William Leibzon asked
here 
on the general list, namely how many of the people who follow the
general list
are against using alternative formats. We have seen based on past
discussions
that there is a strong corrleation between following the general list
and
not admitting to be able to read any format other than ASCII.

We need some rethinking as to how to judge what the IETF community as a
whole
wants to do about some general issues. The only thing I am sure about is
that
consensus on this list is for keeping everything exactly as it is.

Y(J)S

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 03 Jan 2006 07:46:10 +0100

--On mandag, januar 02, 2006 18:10:15 +0200 Yaakov Stein <[email-masked]> 
wrote:

> The only thing I am sure about is
> that

I'm pretty sure there's no such consensus.

I do, however, see a rather strong consensus-of-the-speakers against using 
MS-Word document format for anything "official".

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 05 Jan 2006 13:35:56 +0100

Harald Tveit Alvestrand wrote:
> 
> 
> --On mandag, januar 02, 2006 18:10:15 +0200 Yaakov Stein 
> <[email-masked]> wrote:
> 
> I do, however, see a rather strong consensus-of-the-speakers against 
> using MS-Word document format for anything "official".

I think we need to tackle this whole issue, if we do decide to
tackle it, in a much more systematic way.

- what are our functional requirements?
- which of them are not met today?
- what are the possible solutions, and what is their practical
   and operational cost?
- which, if any, solutions should we adopt, on what timescale?

I believe that if we took a systematic approach like that, the issue
of how we determine consensus would be broken into enough small
steps that it really wouldn't be an issue.

     Brian

---

From: Ralph Droms <[email-masked]>
Date: Thu, 05 Jan 2006 08:04:00 -0500

Brian - you've hit on an important point here.  It strikes me that the
process for defining our own document standards has no fundamental
differences from the process for defining any other standard.  Why shouldn't
this archival document standard be developed and adopted as a Standard in
the same way?

I've explicitly refrained from contributing any observations from my 15
years of experiences working with IETF docs in vi, emacs, nroff, Word,
LaTeX, PDF, ASCII-art diagrams, XML2RFC, dot-matrix printers, etc., as well
as experience with Word in CableLabs/DOCSIS specs - because those
contributions would not be part of an engineering process like the one you
describe, and would be simply more hot air if posted outside of a process.

Well, one might say, haven't we tried the IETF process on the archival
document format problem in the past?  And the document format hasn't
changed.  Yup, and there are a couple of (not necessarily mutually
exclusive) conclusions we might draw:

* the current format is the best solution we can devise for our requirements
* the IETF engineering process is flawed

- Ralph

---

From: Stewart Bryant <[email-masked]>
Date: Thu, 05 Jan 2006 14:22:15 +0000

> tackle it, in a much more systematic way.

I am in favour of any practical method that allows us to progress towards
the best tools for the job.

My personal end-goal is simple: I want us to be able to use modern
graphical techniques in normative text to help me to describe problems
and their solutions. There are many other nice-to-have's, but at the end
of the day it is the diagrams that are the key missing feature in
our document process.

The following would be a fine set up steps on the way to
determining the way forward. Perhaps my co-authors and I should
attempt another draft with this structure.

> steps that it really wouldn't be an issue.

Maybe.

The discussion on the list illustrates the well known problem
of determining consensus in the presence of highly vocal members
of the IETF community. This, as I recall,  is a problem that was
discussed some time ago in the "Miss Manners" talk. However it is 
separate problem from the issue of document formats and
should be addressed as a different work item.

- Stewart

---

From: Sam Hartman <[email-masked]>
Date: Mon, 09 Jan 2006 10:41:59 -0500

>>>>> "Stewart" == Stewart Bryant <[email-masked]> writes:

    Stewart> I am in favour of any practical method that allows us to
    Stewart> progress towards the best tools for the job.

    Stewart> My personal end-goal is simple: I want us to be able to
    Stewart> use modern graphical techniques in normative text to help
    Stewart> me to describe problems and their solutions. There are
    Stewart> many other nice-to-have's, but at the end of the day it
    Stewart> is the diagrams that are the key missing feature in our
    Stewart> document process.

Are you looking for normative figures?  If so, can you point to an
example where you think they are necessary?  (I'd like to avoid a
discussion of packet diagrams for the moment if that's OK)

Or are you looking at normative text that is easier to understand with
illustrative figures?

---

From: Scott W Brim <[email-masked]>
Date: Mon, 09 Jan 2006 10:53:37 -0500

> discussion of packet diagrams for the moment if that's OK)

Normative figures perhaps.  Normative equations definitely.

Is there any input format for *just* equations (or figures), standing
by themselves, which we can agree is open, standardized, stable and
deterministic in output?

---

From: Sam Hartman <[email-masked]>
Date: Mon, 09 Jan 2006 11:05:40 -0500

>>>>> "Scott" == Scott W Brim <[email-masked]> writes:

    Scott> On 01/09/2006 10:41 AM, Sam Hartman allegedly wrote:
    >> Are you looking for normative figures?  If so, can you point to
    >> an example where you think they are necessary?  (I'd like to
    >> avoid a discussion of packet diagrams for the moment if that's
    >> OK)

    Scott> Normative figures perhaps.  Normative equations definitely.

    Scott> Is there any input format for *just* equations (or
    Scott> figures), standing by themselves, which we can agree is
    Scott> open, standardized, stable and deterministic in output?

Mmm.  I have some significant accessibility concerns with that if you
expect me to review the documents.  I think that's a small (although
probably impossible at the current point) matter of technology.

If you go so far as normative figures I have a lot more questions
about why and what you are trying to accomplish.

--Sam

---

From: Dave Crocker <[email-masked]>
Date: Mon, 09 Jan 2006 09:02:21 -0800

> probably impossible at the current point) matter of technology.

Sam,

Thank you for raising this basic point.  Since the IETF seeks the widest 
possible availability for its documents, it might help discussion about format 
choices to understand how your ability to review documents is affected.

For starters, what sorts of things about the *current* IETF document format make 
your reviewing easier or more difficult?

(Given the discussion that triggered this note, specific comments on figures and 
formulas would be helpful.  I will add my own curiosity about the extent to 
which punctuation choices aid or hinder review.)

d/

-- 

Dave Crocker
Brandenburg InternetWorking
<http://bbiw.net>

---

From: Sam Hartman <[email-masked]>
Date: Mon, 09 Jan 2006 12:47:28 -0500

>>>>> "Dave" == Dave Crocker <[email-masked]> writes:

    >> Mmm.  I have some significant accessibility concerns with that
    >> if you expect me to review the documents.  I think that's a
    >> small (although probably impossible at the current point)
    >> matter of technology.

    Dave> Sam,

    Dave> Thank you for raising this basic point.  Since the IETF
    Dave> seeks the widest possible availability for its documents, it
    Dave> might help discussion about format choices to understand how
    Dave> your ability to review documents is affected.

Hi.  I was actually not trying to raise the general issue of how
accessible the IETF should make its documents.

I was simply trying to raise the practical issue that while I'm on the
IESG, I will hold a discuss if I cannot understand some aspect of a
document well enough to review it.
It's not trying to make a political statement, just that I've been asked to do a job by the nomcom and I will try and do that job.

Naturally should I hold such a discuss I'd work with the authors to
try and figure out enough detail to quickly understand their document.

I think the general issue is important but should be examined in the
broader context of the document format issue.

However I'd like to wait and see if we're going to actually do
anything about the broader issue before spending cycles on this
question.  I proposed a specific set of steps that the proponents
could take to conduct an experiment; so far no one has disagreed with
that proposal.  So let's see if it looks like people are going to try
that approach.  If so, I agree this would be an important thing to at
least start thinking about.  It may well not matter for the
experiment, but it may matter long-term.

If anyone wants to chat about document accessibility in person I'd be
happy to do so.  I just would rather not get into a long email
discussion right now.

--Sam

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 09 Jan 2006 17:59:19 +0000

Sam Hartman wrote:

>Are you looking for normative figures?  
>
Yes

>If so, can you point to an
Sam

In no particular order:

State diagrams - The issue here is density and line-crossing.

Packet Diagrams

Annotated network scenarios - I have particularly seen this problem
in BGP with ASBRs, packets, ASs and filters, but it applies to
any case where we wish to say this is the state in this node,
so this packet flows, so this is the resultant state this node.
The problem is particularly difficult when you get to four or
more nodes and need to show their topological context.
The problem here is the need for higher density line and
object density than you can achieve with ASCII art as well
as the need for small fonts to include the annotation.

Network snapshots, which require a set of network states to
be presented side by side by side. This is needed in showing
the dynamics of network state change such as the dynamics
of routing convergence.

Overlay networks of the sort that get built with tunnels - I have
seen problems in PWE3, RTGWG (with IP fast re-route), and L*VPN.
I would think that there are similar problems of expression in
the mobile area. This is an area that really benefits from both
lines and colours.

Non-trivial network scenarios where you want to show asymetric
costs commonly needed in routing.

Scenarios where you wish to illustrate reachibility of different
regions of a network.

Equations in general.

Network diagrams annotated with equations (such as routing costs
or timing).

Graphs, in general, but they become normative when you
wish to apply a performance mask.

Timing diagrams

G.802 diagrams (this is a formalized notation for layered
networks). These are sort of like equations in that the symbols
are well known. The diagrams can be quite complex.

It has also been pointed out that you can't write a draft showing
the dependency relationships between other drafts using the tools
group tools output. Not that such a draft would be normative.

I am planning to write a draft to illustrate the problem, which
I will submit in both ASCII and pdf using the current dual
submission procedure. This in itself will be an interesting
exercise to see if it still works.

- Stewart

---

From: "Steven M. Bellovin" <[email-masked]>
Date: Mon, 09 Jan 2006 12:59:38 -0500

>by themselves, which we can agree is open, standardized, stable and

LaTeX is the standard in the math and theory world.  It's free, and 
runs on just about everything.  If I recall correctly what Kernighan
once said, eqn was designed so that its input language was more or 
less what one mathematician would say to another over the phone, which 
(I assume) would help with accessibility.  There are open source versions
of eqn; I think that they run on more or less anything, too.
In the pure HTML world, there's MathML, though it's *really* ugly to 
read.  I have no idea how much it's supported by today's browsers.  
(Kernighan started working on an eqn to HTML translator some years ago, 
but back then no browser really worked properly for it.)

Note that I'm *not* saying we should adopt any of these for RFCs; I'm 
simply saying that there are some well-known systems that satisfy at 
least your four criteria.

		--Steven M. Bellovin, http://www.cs.columbia.edu/~smb

---

From: Sam Hartman <[email-masked]>
Date: Mon, 09 Jan 2006 14:01:23 -0500

Hi.  With the exception of packet diagrams, I think all the examples
you bring up benefit significantly from clear textual description.  I
believe I'd think that even if I could see the diagrams and I believe
I have enough experience with visualization (although not sight) to be
confident in that belief.

As such, I don't believe these diagrams should be normative.

I actually thin that many of the tunnel overlay network documents
(PWE3, L2VPN, L3VPN) could benefit significantly from more focus on
the text of the descriptions of situations being described.

If you believe that normative documents are required, I'd like you to
explain why the diagrams cannot be described in the text, why doing so
would decrease the quality of the specification, or why doing so would
not be a useful investment in our time.

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 09 Jan 2006 19:46:42 +0000

Sam Hartman wrote:

>you bring up benefit significantly from clear textual description.
>
Sam

I am not saying that clear text is not needed to accompany a diagram.
However a diagram allows a lot less text to be written producing a
shorter clearer draft with less clutter.

For example you could say the following in text : router A connects to
router B and D, the cost from A to B is 2, and the cost from A to D is 4.
Router B connects to router C. The cost to router A is 6, and the cost to
router C is 10. Router C connects to router D. The cost to router B is 9
and the cost to router D is 8. The cost from router D to router A is 16
and the cost to router A is 99.

In fact you would probably need a table to make sure that you did not
make a mistake. In either case it is hard for most people to figure
out the  what the topology is much less imagine packet actions.

Now compare this to:

"The network and costs are as shown in figure foo."

Simple text and the visualisation is already done for you so you
can focus on the problem.

Now I realise the above could be done in ascii art, but we have problems
that need 13 or more nodes to set up.

>  I
It is probably a question of how we work and the tools that we
find most effective.

>not be a useful investment in our time.
One reason is that it takes a lot of text to describe what can be drawn
with a few lines and symbols. Many people will get lost in the text and
in any case would have to transcribe the text into a diagram for
themselves before they could understand what is happening.

- Stewart

---

From: Ned Freed <[email-masked]>
Date: Mon, 09 Jan 2006 11:35:00 -0800 (PST)

> runs on just about everything.

LaTeX or some other TeX variant. AMS-TeX is one such but there are a bunch of
others. (Always fun when you get TeX source that uses a macro package you don't
have available.)

> If I recall correctly what Kernighan

Personally, I'm unconvinced that design goal was met.

> but back then no browser really worked properly for it.)

It is probably worth noting that another very popular format in the math world
for equations is essentially ASCII art. This is what you tend to get as output
from the various symbolic manipulation packages like Reduce, Macsyma, Maple,
and last time I looked, Mathematica. Stuff like:

                            2    3   2         2    3   3
             w    w x   (w k  + w ) x    (3 w k  + w ) x
(D7)/T/      -- + --- - -------------- - ---------------- + . . .
              4    3            4                 3
             k    k          6 k               6 k 

is actually pretty readable. And any of these packages can be used to generate
this sort of thing; no need to piddle around in an editor. (The example above
is a Taylor series expansion from Maxima.) These packages often have TeX and
eqn output modes as well.

Another interesting development in this space is the new programming language
Fortress being developed by Guy Steele, which sticks with simple character
strings but uses the full UTF-8 repetoire to advantage. I personally find it
hard to get used to after so many years of looking at ASCII art equations,
especially when lots of matrix and vectoor stuff is involved, but this might be
an interesting avenue to pursue if we ever started allowing UTF-8. See:

   http://research.sun.com/projects/plrg/

for additional information.

> least your four criteria.

Indeed. I will also point out that as with anything having to do with which
notation is better or best, getting agreement even in the relatively narrow
confines of the math community is next to impossible. You know how it is: "The
arguments are nasty because the stakes are so low."

				Ned

---

From: "Dassa" <[email-masked]>
Date: Tue, 10 Jan 2006 07:31:15 +1100

|> -----Original Message-----
|> From: [email-masked] [mailto:[email-masked]] 
|> On Behalf Of Stewart Bryant
|> Sent: Tuesday, January 10, 2006 6:47 AM
|> To: Sam Hartman
|> Cc: Harald Tveit Alvestrand; [email-masked]
|> Subject: Re: Normative figures
|> 
|> Sam Hartman wrote:
|> 
|> >Hi.  With the exception of packet diagrams, I think all the 
|> examples 
|> >you bring up benefit significantly from clear textual description.
|> >
|> Sam
|> 
|> I am not saying that clear text is not needed to accompany a diagram.
|> However a diagram allows a lot less text to be written 
|> producing a shorter clearer draft with less clutter.
<SNIP>

Perhaps this is getting to the crux of the issues.  I see the IETF documents
as breaking down the problems into smaller chunks that can be dealt with one
at a time and which add up to a big picture description of the whole Internet.
I see each individual document as being simple within itself, limiting the
context to the smallest level an issue may be dealt with.  By adding more
complexity to the documents I feel it is allowing more complex issues to be
described in the documents but the documents then become larger, more
difficult to comprehend and will be more difficult to process.

Using the example you gave for routing costs, I see the description of routing
cost basics or specifics as one document and the description of how they may
be dealt with as another.

By forcing the documents to be in a simple format, there are limitations on
the complexity that may be explained in a single document, but I consider this
a good thing.  It forces everyone to break the problems and issues down to
their lowest levels and forces simple explanations that make it easier for
everyone to understand.

If more complex documents with full diagramatic process flows are required,
these could be books written linking a number of IETF documents together to
describe a more general practical picture of their implementation.

Darryl (Dassa) Lynch

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 9 Jan 2006 12:40:06 -0800

>others. (Always fun when you get TeX source that uses a macro 

Why are we talking about the input formats and not the output 
formats? Are people suggesting that we allow LaTeX and so on as 
*output* formats and appear in Internet Drafts and RFCs?

--Paul Hoffman, Director
--VPN Consortium

---

From: Marshall Eubanks <[email-masked]>
Date: Mon, 9 Jan 2006 17:33:23 -0500

Note that LaTeX is an input format - it will output to many output  
formats - even to ASCII. (And, of course, it sits on  top of and  
drives TeX itself.)

As a physicist, I have used LaTeX a lot. It has a steep learning  
curve - much more than with, say, html.
When I did a set of conference proceedings in LaTEX -

Proc. USNO Workshop on Relativistic Models for use in Space Geodesy,  
1991.

most of the  papers went  fine, a few just would not display  
correctly and took a lot of tweaking just
to get them to look halfway right. These last took a lot of time.  
TeX / LaTeX is like that.

So, most journals today use style sheets, such as the AMS-TEX ones,  
and templates, to enforce their look and feel.
I suspect they have issues from time to time even with that.

Oh, and if an email line begins with "From" and get turns into  
">From", LaTex will turn that into
"<upside down ?>From", which you can see in the strangest places,  
once you know to look for it.

There are a few WSIWYG editors for Tex/LaTex, but most people I know  
use a text editor.

I certainly would not regard moving  everything to LaTeX as a trivial  
change.

Regards
Marshall Eubanks

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 10 Jan 2006 10:29:54 +0100

On Mon, Jan 09, 2006 at 07:46:42PM +0000,
 Stewart Bryant <[email-masked]> wrote 
 a message of 73 lines which said:

> For example you could say the following in text : [long and
> complicated example deleted]

For such examples (do note that your example is an illustration of a
point and therefore does not need to be normative, like, say, a state
diagram), graph people produced several languages which are
non-ambiguous, expressed as ASCII and produce nice output. Some even
are free (as in free speech and as in free beer). (Unfortunately, none
is "standard" in any way, AFAIK.)

The one I recommend is Graphviz
(http://www.research.att.com/sw/tools/graphviz/). An example of
Graphviz code for a real network:

http://www.seekingfire.com/projects/metanetwork/maps/meta.dot

(See http://www.seekingfire.com/projects/metanetwork/info.html for the
context)

See also:

"Managing IP Networks with Free Software"
http://www.nanog.org/mtg-0210/ppt/stephen.pdf has a chapter about
Graphviz

"A Systematic Approach to BGP Configuration Checking"
http://www.nanog.org/mtg-0310/pdf/feamster.pdf uses cflow to produce
Graphviz ".dot" files.

(ACL compilers like Netspoc http://netspoc.berlios.de/ also have an
ASCII language to represent networks.)

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Tue, 10 Jan 2006 16:29:48 +0100

On Mon, Jan 09, 2006 at 07:46:42PM +0000,
 Stewart Bryant <[email-masked]> wrote 
 a message of 73 lines which said:

> For example you could say the following in text : router A connects
> to router B and D, the cost from A to B is 2, and the cost from A to
> D is 4.  Router B connects to router C. The cost to router A is 6,
> and the cost to router C is 10. Router C connects to router D. The
> cost to router B is 9 and the cost to router D is 8. The cost from
> router D to router A is 16 and the cost to router A is 99.

Here is the Graphviz code, to compare (I also attached the
automatically produced PNG but Graphviz can make PDF or SVG as well):

// See http://www1.ietf.org/mail-archive/web/ietf/current/msg39858.html

digraph bryantnetwork {
  
  label = "Stewart Bryant's network";
	 
 // Routers
 node [shape=box, fontsize=15];

 // Links
 edge [len=2.0];
    A -> B [label="2"];
    A -> D [label="4"];
    B -> A [label="6"];
    B -> C [label="10"];
    C -> B [label="9"];
    C -> D [label="8"];
    D -> A [label="16"];
    D -> C [label="99"];
    
}

---

From: Sam Hartman <[email-masked]>
Date: Tue, 10 Jan 2006 16:08:57 -0500

>>>>> "Stewart" == Stewart Bryant <[email-masked]> writes:

    Stewart> For example you could say the following in text : router
    Stewart> A connects to router B and D, the cost from A to B is 2,
    Stewart> and the cost from A to D is 4.  Router B connects to
    Stewart> router C. The cost to router A is 6, and the cost to
    Stewart> router C is 10. Router C connects to router D. The cost
    Stewart> to router B is 9 and the cost to router D is 8. The cost
    Stewart> from router D to router A is 16 and the cost to router A
    Stewart> is 99.

I think we fundamentally disagree here that the text is not needed.
There's a bit of fuzz in that I think the normative text needs to
describe the properties of the graph necessary to make your point and
to make it obvious that such a graph exists; it may not need to
describe all the labels of all edges.

---

From: Frank Ellermann <[email-masked]>
Date: Wed, 11 Jan 2006 01:02:55 +0100

Stephane Bortzmeyer wrote:

> automatically produced PNG but Graphviz can make PDF or SVG
> as well)

Nice, I've always loved graph theory.  Now let it colour the
shortest path fromn B to D, and then ask it for some decent
ASCII art output... <g>  Joke off, this "code" should work as
it is forever for everybody who wants it to work.  Bye, Frank

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Wed, 11 Jan 2006 09:24:15 +0100

On Wed, Jan 11, 2006 at 01:02:55AM +0100,
 Frank Ellermann <[email-masked]> wrote 
 a message of 11 lines which said:

> this "code" should work as it is forever for everybody who wants it
> to work.

Yes, the good point about Graphviz is that it is readable even if you
do not know / have the software and that it is represented in ASCII,
therefore requiring no change to the procedures or rules.

For an ASCII-art output of an arbitrary graph, I wait for a clever CS
student to modify Graphviz in that way :-)