Subject: Re: Source Format of RFCs

From: "Bruce Balden" <[email-masked]>
Date: Tue, 1 Dec 1998 01:00:19 -0800

I fully understood before I posted the earlier message that revising the official
format for RFC's is a divisive and difficult issue.

Although a revision of this format might be useful, my proposal is more modest, and
more clearly explained as follows.

De facto, all of the lengthier documents, if not the shorter ones as well, are
prepared by their authors orginally in some richer format, and when they are in draft
form, can frequently be found on company ftp sites (although not at the IETF) in their
original format, which is of some benefit to anybody with access to a Windows machine,
because MS makes a free MS Word viewer available. Similarly, for UNIX documents there
are groff and LaTEX under Linux.  Whatever the format, be it MS-Word or LaTEX, or
troff, this format is later projected  into ASCII, and the diagrams in particular
sometimes suffer greatly.  When the drafts become RFC's, these original forms
disappear never to be seen again.  As far as official activities go, this is perhaps
appropriate, but I wouldn't mind having the original form around.

Referring to some of the other comments, I would be sorry so see the text versions
disappear, having them searchable is extremely useful.  I frequently hope for, in
fact, a long unformatted file without the text headings and footers, so as to make it
easier to search with a text editor.

I don't greatly value the bold face headings, etc., but I found serious errors in a
version of an RFC which had been printed in a book and none-too-carefully typeset,
obviously retrofitted from the text version.  I do value proportionally spaced fonts
as far easier for the eye to read, and also taking up less paper.

The basic problem with ASCII and RFC's is that we are dealing with networking, which
sometimes requires signficant non-verbal information to be conveyed in the form of
interaction diagrams, topology diagrams, frame layouts, etc.

So as another commentator pointed out, maybe special attention should be placed on
diagrams, and a tar or zip file of relevant diagrams be made available.

MS Word is a good format because it is searchable, but IETF would need a guarantee
from Microsoft never to require a licensing fee for the Word viewing program. It
should also be noted that the Word viewing program is available for Solaris and
perhaps other UNIX-like OS's.

Before I get too many comments from MS foes, let me say that I dislike each successive
version of MS Word a little more, and I am probably going to revert to LaTEX for
technical documents, but that getting LaTEX to work can be a challenge. Even DVI
format, which is far less useful then LaTEX source from the point of view of
maintenance, can be an adventure to use.

Standards track RFC's are getting to be important documents, viewed by a much larger
audience than previous RFC's (such as the now-deleted one about a cancelled meeting),
and there far more of them than before, and their content is getting more complex.

Adopting a better source format for RFC's might be a step towards addressing this, but
it would seem than we are facing the need to use the ITU-T model of segregating RFC's
into groups by subject.

I remember twenty years ago an article in a publication noted that the programming
enterprise was one of the last tasks to benefit from software, i.e. almost all effort
in the past was expended in delivering applications to end users, with scant attention
to the programmer's own environment and comfort.

Similarly, in the IETF, the RFC's document an amazing variety of advanced networking
tools, very few of which the IETF actually employs to its own benefit.

-----Original Message-----
From: Keith Moore <[email-masked]>
To: Bruce Balden <[email-masked]>
Cc: Dunlap, Randy <[email-masked]>; [email-masked] <[email-masked]>
Date: November 30, 1998 6:12 PM
Subject: Re: Source Format of RFCs

|> A number of RFC's and drafts, such the ones regarding BGP, suffer negatively
|> from the text-only format. It would be nicer to have a richer format,
|> especially since such documents rarely originate in the text-only format,
|> but are rendered into that format by MS-Word, nroff, LaTeX or whatever.
|>
|> Would it be asking too much for all RFC's too made available in some sort
|> of rich format, and in particular their source format since that doesn't
|> require additional work?
|
|You may be asking for far more than you realize.
|
|I agree with you that the ASCII-only RFC format is a bit restrictive.
|However, ASCII is amazingly flexible.  It is one of the few formats
|that can be reasonably viewed (or searched) on any computer, and
|printed on any printer.  Neither of these is true of the document
|formats associated with common word processors.  Also, since it is
|so simple, ASCII is a good choice for formatting documents that may
|need to be readable for many decades.  For comparison, word processor
|documents from the early 1980s are difficult to read today, and
|it's not difficult to imagine that today's word processor documents
|will be difficult to read ten or twenty years from now.
|
|(The more complex the document format, the less interoperable it is.
|There have been problems with PostScript RFCs because some tools
|produce PostScript that cannot be printed on some printers.
|Similarly, version X of FooWord may produce documents that cannot
|be read by version Y of FooWord.)
|
|There are also problems with making RFCs available in multiple formats.
|One is that the RFC Editor and Internet-Drafts folks would then need
|to deal with a wider variety of tools.  Another is that multiple formats
|can make it more difficult to keep the various formats in sync.
|For instance the RFC Editor generally needs to be able to make minor
|changes to a document before publication, yet may not have the
|tools neded to modify the "source" document.  If the author is asked
|to make the changes, there is then a need to make sure that the
|author does not make other modifications to the work after it has
|been reviewed and approved by a working group and IESG.  Again,
|this generally requires special tools.
|
|Until a new format can supplant ascii text for all intents and purposes,
|there will continue to be a need to produce ascii text versions of RFCs.
|Many word processors do a very poor job of producing "RFC ASCII"
|(with margins that allow printing on both A4 and US-Letter paper,
|no overprinting, and so forth)  Documents written for a wysiwyg
|word processor are often difficult to read as ASCII, e.g. because
|the word processor will try to put too many words on a line, or too
|many characters in a column of a table. (I've seen various versions
|of this while reviewing RFCs for IESG.)
|
|Then there is a need to make sure that new RFC formats are adequately
|supported by all of the RFC and internet-drafts mirrors.
|
|For some time I have been interested in exploring the use of HTML
|for RFCs and Internet-Drafts.  This would provide several advantages,
|including multifont and multi-faced text in a variety of sizes,
|hyperlinks, images, and ease of generation with many word processors.
|HTML is openly published and widely-used, and (provided a few restrictions
|are observed) one of the few formats which can be read on every platform
|(and is easily converted to plain text).
|
|However, as far as I can tell, there are several barriers to adoption
|of HTML as an RFC format:
|
|0. Given that not all features of HTML are available on all HTML viewers,
|   We need some (perhaps loose) standards for which features can be
|   used, and which ones should be avoided.    (For example, the use of
|   tables for formatting, or embedded images for bullets, should perhaps
|   not be allowed.  Even the use of CSS might be questionable since
|   many viewers do not support it.) The standard would need to specify
|   things like which HTML tags could be used, the naming convention to
|   be used for files, the format of links to other RFCs and to embedded
|   images (presumably relative rather than absolute), and "style" -
|   how a document should be formatted if it wants to look like an RFC
|   (e.g. which tag should you use for a section header?).
|
|1. Once an "RFCs in HTML" standard were adopted, common word processors
|   might be unable to produce HTML which conformed to the standard.
|   It might turn out to be even more difficult to produce reasonable
|   HTML than it currently is to produce reasonable plain text.
|
|2. The Internet-drafts and RFC Editor people would need tools to let
|   them edit HTML, remove or alter tags that did not conform to the
|   standard, and perhaps to convert HTML to "RFC format" ASCII text.
|   (again, such tools would need to work with the garbage HTML that
|   is generated by some word processors)
|
|3. The mirror sites would have to adapt to accomodate HTML RFCs,
|   including any embedded images referenced by those RFCs.
|
|...and there are probably a few more.  None of these is insurmountable,
|but so far there has been a lack of critical mass behind the effort.
|
|A year or two ago I set up a discussion list for rfcs-in-html.
|To subscribe, send mail to [email-masked].
|
|Keith
|
|-
|This message was passed through [email-masked], which
|is a sublist of [email-masked]. Not all messages are passed.
|Decisions on what to pass are made solely by Harald Alvestrand.
|

---

From: Lloyd Wood <[email-masked]>
Date: Tue, 1 Dec 1998 12:36:21 +0000 (GMT)

On Tue, 1 Dec 1998, Bruce Balden wrote:

> MS Word is a good format because it is searchable

??? Embedded compound objects are particularly opaque. In fact, so are
the numerous file-format versions. I can search a pdf file for a text
string in acroread far better than I can search for same in a Word
document; distilling .ps to .pdf in order to search the postscript
and copy-paste text is IMO acroread's best feature.

> but IETF would need a guarantee

Single-vendor solutions don't get standardised. (Internet Explorer is 
available for Solaris and other unix OSs, but I don't know anyone
willing to use it. Well, more than once.)

> maintenance, can be an adventure to use.

There isn't a good ongoing LATeX development effort with standard
macros/functionality etc AFAIK (and if there was, it probably wouldn't
matter, if xemacs is any indication). Given how Knuth's fonts are
rendered onscreen in Acrobat's pdf viewer (generally very badly), I'm
expecting LATeX use to decline as pdf is increasingly used to tunnel
paper through networks...

LATeX interoperability is a problem IMO.

> to the programmer's own environment and comfort.

This ignores the development of programming languages, so that
programmers don't have to toggle front-panel switches, of course.
If that's not programming comfort...
 
> tools, very few of which the IETF actually employs to its own benefit.

??? If you were to list all the RFCs involved in the mere exchange of
messages on this mailing list... the IETF is effectively bootstrapping
itself into existence via use of the things it helps develop or
document.

The real problem is that volunteer efforts focus solely on the
job at hand, rather than niceties; compare e.g. the current state of
gcc/gdb with the current state of the Visual C++ environment, or the
standardised featureset of mail agents with what cc:mail et al have
offered for years.

We're doomed to reinvent the wheel until the wheel becomes a public
standard. Of all the formats mentioned for RFCs, only the
currently-used ascii/ISO8859-1 is a public standard.

The irony of arguing against ascii in an ascii email should be noted, 
too.

L.

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>

---

From: Lloyd Wood <[email-masked]>
Date: Tue, 1 Dec 1998 12:36:21 +0000 (GMT)

L.

---

From: Merton Campbell Crockett <[email-masked]>
Date: Tue, 1 Dec 1998 06:02:11 -0800 (PST)

On Tue, 1 Dec 1998, Lloyd Wood wrote:

} On Tue, 1 Dec 1998, Bruce Balden wrote:
} 
} > MS Word is a good format because it is searchable
} 
} ??? Embedded compound objects are particularly opaque. In fact, so are
} the numerous file-format versions. I can search a pdf file for a text
} string in acroread far better than I can search for same in a Word
} document; distilling .ps to .pdf in order to search the postscript
} and copy-paste text is IMO acroread's best feature.

Actually, WordPerfect would be a better choice than Microsoft Word.  It is
available for a much broader range of systems than Word.  Unfortunately, it,
also, has succumbed to the embedded object syndrome.

This leaves few choices with the needed universality.  Adobe's Postscript
and Portable Document Format are potential alternatives to ASCII text as
they can be read/viewed using gv or gsview and ghostscript on all platforms
or acroread on Windows, Macintosh, and a limited set of Unix platforms.

} > but IETF would need a guarantee
} > from Microsoft never to require a licensing fee for the Word viewing program. It
} > should also be noted that the Word viewing program is available for Solaris and
} > perhaps other UNIX-like OS's.
} 
} Single-vendor solutions don't get standardised. (Internet Explorer is 
} available for Solaris and other unix OSs, but I don't know anyone
} willing to use it. Well, more than once.)

Another problem is that Word is only available for a very limited set of
non-Windows platforms.  It has ubiquiteness but lacks universality.

Merton Campbell Crockett
Hasentaler InfoSysteme

---

From: Merton Campbell Crockett <[email-masked]>
Date: Tue, 1 Dec 1998 06:02:11 -0800 (PST)

Merton Campbell Crockett
Hasentaler InfoSysteme

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 09:21:35 -0500

> Although a revision of this format might be useful, my proposal is more 
> modest, and more clearly explained as follows.
> 
> De facto, all of the lengthier documents, if not the shorter ones as well, 
> are prepared by their authors orginally in some richer format, and when 
> they are in draft form, can frequently be found on company ftp sites 

Part of the problem is, these documents in draft form are not the
RFCs.  They should not be issued as alternatives formats for RFCs
unless they contain the same text as the final RFC, and to do that,
would essentially require that the RFC Editor make those changes.

> Whatever the format, be it MS-Word or LaTEX, or troff, this format
> is later projected into ASCII, and the diagrams in particular
> sometimes suffer greatly.  

If this happens something is terribly wrong.  If the document is
written to be an RFC, the author knows it will eventually be projected
into ASCII, and the author can concentrate on making "ASCII art"
diagrams as clear as possible.  On the other hand, I've seen a few
RFCs where the author apparently tried to make the MS-Word version as
attractive as possible, without regard for what the ASCII version
would look like.  That seems to me to be misplaced effort, and the RFC
does suffer as a result.

> The basic problem with ASCII and RFC's is that we are dealing with
> networking, which sometimes requires signficant non-verbal information
> to be conveyed in the form of interaction diagrams, topology diagrams,
> frame layouts, etc.

Indeed, and I think everyone would agree that RFCs would at least
occasionally benefit from being able to show graphical information.
(better than can be done by "ASCII art".)

> MS Word is a good format because it is searchable, but IETF would need
> a guarantee from Microsoft never to require a licensing fee for the
> Word viewing program. It should also be noted that the Word viewing
> program is available for Solaris and perhaps other UNIX-like OS's.

I would strongly object to using MS Word's native format for RFCs.
The availability of a license-free Word viewing program for a small
number of non-Microsoft platforms (none of which I happen to use)
would not address the myriad problems with using a vendor-proprietary
format.

Rumor has it that future versions of MS Word will use HTML, which
would make less of an issue.  (We'd probably still need to strip out
the Microsoft-proprietary tags in the document.)

> Standards track RFC's are getting to be important documents, viewed by
> a much larger audience than previous RFC's (such as the now-deleted
> one about a cancelled meeting), and there far more of them than
> before, and their content is getting more complex.

Again, I read a LOT of RFCs (or internet-drafts wanting to become
RFCs), and I rarely find that the draft is difficult to read because
it's written in plain ASCII.  (Occasionally there is a diagram which
is difficult to follow, but this is maybe 10% of the cases.)  I often
find that an draft is more difficult than necessary to read because it
lacks clear organization and focus, or clear wording.  (and the
percentage of drafts in this category seems to be growing)

So while I generally do support the idea of moving to HTML so that
graphics can be included, I am concerned that moving to a fancier
format will further increase the number of documents that are poorly
written - i.e. written with appearance in mind rather than content.

> I remember twenty years ago an article in a publication noted that the
> programming enterprise was one of the last tasks to benefit from
> software, i.e. almost all effort in the past was expended in
> delivering applications to end users, with scant attention to the
> programmer's own environment and comfort.
> 
> Similarly, in the IETF, the RFC's document an amazing variety of
> advanced networking tools, very few of which the IETF actually employs
> to its own benefit.

I'd argue that for both source code and RFCs, the utility of being
able to easily move the code/text to any platform, and to compile the
code or read/edit the text on any platform, far outweighs the utility
we would get from moving to a fancier format (for either code or text)
that could only be used on a limited number of platforms.

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 09:21:35 -0500

Keith

---

From: Lloyd Wood <[email-masked]>
Date: Tue, 1 Dec 1998 14:35:04 +0000 (GMT)

> or acroread on Windows, Macintosh, and a limited set of Unix platforms.

'there is a fair chance that they can be read/viewed' is probably more
accurate.

Ghostview 1.5's interpreter fails more often than not, especially on
postscript output from a number of Windows apps/printer drivers;
Acrobat Reader has exhibited oddities in the past, such as neglecting
to show any text at all if it wasn't in an available font. (Macintosh
version, 2.0 and earlier, possibly others.) And pdf has the same sort
of internal format changes that plague Word; I'm dreading an
introduction of Acrobat Reader 4.0 with an incompatible pdf format,
perhaps with different internal compression.

L.

what we really need is a standard postscript virtual machine.

---

From: Lloyd Wood <[email-masked]>
Date: Tue, 1 Dec 1998 14:35:04 +0000 (GMT)

L.

---

From: Randy Bush <[email-masked]>
Date: Tue, 1 Dec 1998 07:01:49 -0800 (PST)

> available for a much broader range of systems than Word.

I prefer TeX.  <msacras>

randy

---

From: Randy Bush <[email-masked]>
Date: Tue, 1 Dec 1998 07:01:49 -0800 (PST)

I prefer TeX.  <msacras>

randy

---

From: [email-masked]
Date: Tue, 01 Dec 1998 10:03:42 -0500

> what we really need is a standard postscript virtual machine.

Would Java be an appropriate implementation language for this PVM?

<ducking> ;)

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

---

From: [email-masked]
Date: Tue, 01 Dec 1998 10:03:42 -0500

<ducking> ;)

---

From: "Donald E. Eastlake 3rd" <[email-masked]>
Date: Tue, 01 Dec 1998 10:43:20 -0500

Hi,

From:  "Bruce Balden" <[email-masked]>
Message-ID:  <[email-masked]>
Date:  Tue, 1 Dec 1998 01:00:19 -0800
X-MSMail-Priority:  Normal
X-MimeOLE:  Produced By Microsoft MimeOLE V4.72.3110.3

>format for RFC's is a divisive and difficult issue.

No really.  Periodically someone comes out saying that all this old
musty ASCII requiements should be dumped in favor of the latest fad.
There follows a flurry of email in which the advantages of a free,
universal, permanent, trivially machine processable, archival,
standard format is pointed out, and ASCII is then retained as the
primary format.

>appropriate, but I wouldn't mind having the original form around.

All longer RFCs are not prepared in "richer" formats.  While on
occasion I have assisted in converting documents for other formats,
all of the RFCs I have originated have been maintained in ASCII nroff
files from the start, including some pretty long ones.  The fact that
you can point to a number of incompatible rich formats seem proof that
some sort of basic universal format should be used.  Note that
proprietary commercial word processors have a financial incentive to
change and add backward incompatible features so as to encourage the
purchase of upgrades.  Given that, after publication of the .txt form,
a postscript .ps version can be published, fancy formating is not
necessarily lost.

>easier to search with a text editor.

I'm not against all possible changes in how RFC information is
presented.  There is some increase in the number of RFCs which
incorporate C or XML or other code like text and it's sometimes
annoying that it gets split over pages with headers and footers.  But
improving that situtation would associate one or more additional plain
ASCII files with an RFC, not anything "richer".

>as far easier for the eye to read, and also taking up less paper.

The question is, why was it "retrofitted"?  For standards track RFCs,
the .txt version and only the .txt version is authoritative.

---

From: "Donald E. Eastlake 3rd" <[email-masked]>
Date: Tue, 01 Dec 1998 10:43:20 -0500

Hi,

---

From: Peter Parnes <[email-masked]>
Date: Tue, 01 Dec 1998 17:16:17 +0100

(. Java PostScript is an application for viewing PostScript Level-1 page 
descriptions similar to GhostScript, but written entirely in Java. 
<URL:http://table.jps.net/~cleh/java/ps/> .)

---

From: Peter Parnes <[email-masked]>
Date: Tue, 01 Dec 1998 17:16:17 +0100

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 11:27:44 -0500 (EST)

> what we really need is a standard postscript virtual machine.

To solve the page-rendering problem, yes. To solve the RFC "rich format"
'problem', no.

What we really need is a structured document format. An SGML or XML DTD
will do just fine. XML has the advantage of being Unicode, while still
supporting essentially 'ASCII' docs through UTF-8's magic ASCII-ness. Both
are grepable, with a well thought-out DTD. Both can easily be flattened
into pure, formatted ASCII through simple scripts, and intelligently so if
we can hammer out a single DTD for RFCs.

HTML, PDF, Postscript all exacerbate the problem several have pointed out,
the "style over substance" issue. A well designed SGML or XML DTD
encourages authors to focus on substance, since there is no 'style' issue. 
'Style' is handled entirely by the rendering phase, which /should/ be an
RFC Editor function. 

That is, an author should write their document in SGML/XML, using the
tools of their choice (remember that both can be hacked together in a
simple ASCII editor as long as we're staying in the lower bounds of the
character set). They should prepare graphics in one of the 'standard'
formats (more below on that). They should tar/gzip the whole mess (both
tar and gzip are freely available on every platform I can think of) and
send it off to RFC Editor.

RFC Editor then applies RFC rendering tools to generate HTML for the
website (including indices, etc.), ASCII text for FTP retrieval,
Postscript, PDF, LaTEX, etc., for people who want 'pretty' copies, and for
prepress and publication.

I understand quite well that style sheets and the like are not entirely
'finished'. That really doesn't hold us up from adopting this approach in
the short term. After all, what matters most is the /content/, and to deal
with /content/, we need only tools for dealing with /structure/. In the
short term (ie: until CSS/XSL or whatever settle down), it is enough to
write one-shot Perl/Java/whatever to render the structured source down
into nicely formatted ASCII text, or to any of the other formats mentioned
above. This is not hard work, and, since all RFCs would share a DTD, needs
only be done once per output format.

Once CSS/XSL /have/ settled down, and the tools are widely available for
making best use of them, then we are each in total control of our RFC
rendering destiny, by our ability to write our own styles for the RFC DTD.
Similarly, those publishing the RFCs in other works will be able to
directly render from the structured sources, eliminating the problem one
poster in this thread mentioned about poorly-rendered RFCs.

Why structured documents at all? What's wrong with ASCII? So many things,
where to begin? Context, or lack thereof, is a good place. ASCII and grep,
as a searching and archival mechanism, lack true support for contextual
searching. Sure, you can use the 'near' operators in some of the grep
tools. Sure, you can use the 'context' option to give you the lines
/around/ your search hit. But that's not /true/ context. /True/ context is
knowing whether the word occured in a header, a contents item, an index
item, a body text element, and so on. /True/ context is knowing that the
word was listed in the 'relevant keywords' section of the document (which,
incidentally, would get shoved into the website's database for faster
searching against keywords). 

ASCII also, as some have pointed out, makes rendering images difficult. In
a structured document format, it would be enough to refer to the image by
its function in the document, then choose a particular rendering of the
image at document rendering time. That means that we can support ASCII art
(for the terminally ASCII (heh! a pun!)), GIF (boo because of patents),
JPEG, VML, PGML, and so on. Furthermore, when the author gives RFC Editor
the images in the richest format possible, RFC Editor can 'flatten' it
down into whatever the IETF decides should be in the 'standard'
distribution for each rendering of an RFC. 

Thus, the HTML tarball for the RFC might include the body of the RFC
rendered into HTML, and the images rendered into JPEG. The 'prepress'
tarball might have the body rendered into fancy Postscript (using only the
finest, hand-wrought fonts), and the images rendered into Postscript or,
when only bitmap source is available, TIFF.

The ASCII tarball, of course, would contain a single ASCII file with cute
little ASCII graphics :-) Assuming, of course, that the author took the
time to render the diagrams into ASCII. Or that someone wrote a /really/
clever (some_vector_graphics_language) --> (ASCII_art) renderer :-)

A structured document also allows us, as the IETF sees the need, to embed
translations in the RFCs, or at least to synchronize translated RFCs with
the originals. That is to say, it is straightforward to design a DTD that
allows multiple translations of the document, or important portions
thereof, to co-exist. (This, actually, is analogous to the various
'translations' of the graphics co-existing). If it's not desirable to
embed the translations, that's fine, too. We can, instead, design the DTD
to have a set of language discriminators that allow an indexing engine to
determine that these are all the same RFC, with different translations.

Did I get to version control? A structured document system would allow us
to /properly/ version control the entire RFC space. That is, we could
properly deal with many of these issues:

	* the migration of drafts into RFCs
	* the evolution of RFCs (supersedes, extends, etc.)
	* the interrelationships of RFCs (ie: all calsch or IMAP
	  documents would be tagged as calsch or IMAP docs, and
	  therefore treatable as a set).
	* the version interrelationships of interrelated RFCs
	  (ie: versions 1.01 of foo-rfc, 2.03 of foo-rfc-extension,
	   and 0.97 of foo-rfc-other-extension all comprise the
	   'current' state of protocol 'foo')

All this, and you can /still/ monkey with it in ASCII and grep! If that
doesn't make y'all pleased as pigs in slop, what the hell will?

For further information, I refer people to http://www.w3c.org/XML/ , and
to the work of the Linux Documentation Project. The LDP, in particular,
has struggled mightily with many of these same issues (standardization,
structure, multiple graphics support, multiple translations, etc.) They
have (last time I checked) settled on an SGML DTD and some tools for
manipulating their documents. I don't track this closely enough to know
details, just concepts.

Anyway, "lemme splain. No, is too much. Lemme sum up." Choosing a
structured document format gives us big improvements but with a migration
path that's pretty non-tumultuous. Initially, we don't have to do anything
more than define a DTD, and start rendering drafts, RFCs, etc., in that
instead of something else, then flattening them down to ASCII as we've
always done. As time goes on, we can add the features that make the move
to structured docs worthwile -- the versioning, translations, multiple
image renderings, and so on. Each of those can be accomplished as an
incremental step, with little disturbance to those reading the RFCs, and
little more disturbance to those writing them.

I understand that ASCII 'has always worked', more or less. But it just
doesn't scale. From all of the traffic here about the growth, and the
influx of new RFCs, and so on, isn't a little scalability just what we
need right now? Let us not go blindly, stumbling into the future...

------------------------------------------------------------------------------
Tripp Lilley, Innovative Workflow Engineering, Inc. ([email-masked])
------------------------------------------------------------------------------
         "Given enough eyeballs, all bugs are shallow" -- Linus's Law

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 11:27:44 -0500 (EST)

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 13:00:42 -0500

> will do just fine. 

The most frequent complaint I hear from RFC authors is "I can't produce
them using my favorite word processor".  We won't solve that problem
by switching to SGML or XML.  And RFC structure is simple enough that
it doesn't really benefit from using SGML or XML.  

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 13:00:42 -0500

Keith

---

From: Ned Freed <[email-masked]>
Date: Tue, 01 Dec 1998 10:46:05 -0800 (PST)

> files from the start, including some pretty long ones.

This is also true of the RFCs I've written. Indeed, in taking over document
editing functions on a couple of occasions, I've deliberately taken documents
out of rich text formats and converted them to bare bones nroff, which I use
because it is what the RFC Editor uses. And the quality of the formatting of
the document has, without exception, improved as a result of doing this. (This
is not just my opinion, but the consensus of comments I've gotten when I've
done this.)

				Ned

---

From: Ned Freed <[email-masked]>
Date: Tue, 01 Dec 1998 10:46:05 -0800 (PST)

Ned

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 13:50:22 -0500

> I don't understand how proposing that we write RFCs in HTML is any better
> than SGML or XML, at least from the "I can't produce them using my
> favorite word processor" perspective. What I mean by that is, every word
> processor HTML export implementation I've seen is seriously buggy and/or
> generates atrocious HTML. 

This is my experience also.  My hope is that we could produce some tools
to turn buggy/atrocious HTML into reasonable HTML (by fixing some of the
brain damage and stripping out unwanted tags) and automatically filter 
new documents through such tools as part of the internet-draft 
submission process.

> As for your last point, about RFCs being "too simple" to warrant SGML or
> XML, I must wholeheartedly disagree. There is no information that is "too
> simple" to warrant being presented in a structured fashion. 

RFCs do have structure, but that structure is adequately represented by
ordinary textual conventions.  I don't see that RFCs have, or need, so
much structure that using SGML/XML would be worth the cost.

Of course, nothing prevents people from using SGML/XML to develop RFCs, 
just as people currently use MS-Word, nroff, LaTeX, and other tools.
(But while I know of several examples of RFCs produced with MS-Word,
nroff, and LaTeX, I haven't heard of an RFC being produced with SGML
or XML.)

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 13:50:22 -0500

Keith

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 14:07:49 -0500 (EST)

> new documents through such tools as part of the internet-draft 

I guess my position is, if we're writing tools anyway, why not do it
right? Certainly, it's "good enough" to represent the structure of RFCs in
plain on textual convention "right now". But, as I mentioned earlier, the
IETF is growing. Perhaps it's merely a personal belief and not truly
"engineering judgement", but I just don't think that "textual conventions"
and ASCII scale well.

I also believe, assuming the work of a few dedicated individuals (a
working group, even?) to hammer out the basics, that the incremental cost
of moving from "textual conventions" and nroff and the like to a DTD isn't
that high. Especially since, as Mr. Baker points out, "Not everyone
creates their documents on PCs."

There are also vast areas of improvement we could realize for 'users' of
the RFCs. Again, I refer you to my earlier message for the specifics.
However, to summarize, I include indexing and searching improvements,
revision control improvements, document interrelationship representation,
and internationalization/translation in my list of "vast areas of
improvement".

As I also mentioned earlier, I believe this can be an iterative,
incremental process. I believe that we can design the DTD such that people
can choose to implement their RFCs with as much or as little attention to
fine-grained structure as they like, while still providing for the basic
benefits of structuring the documents in the first place.

In other words, I think we can make going from ASCII/nroff to SGML or XML
a painless step, but one that opens doors.

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 14:07:49 -0500 (EST)

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 14:45:41 -0500

> right? 

The question is whether adding complexity and making it more difficult
to produce RFCs is the "right" thing...especially when we're getting
complaints that say it's too difficult to produce plain text.

We're already burdening authors and working groups with having to 
think about technical issues such as security, scalability, and 
internationalization.  And while such issues are important, they
already impose a considerable burden.  I don't want to burden authors
further by requiring them to use an unfamiliar text production system.
Nor do I want to burden readers further by adding more structure and
encouraging people to make RFCs longer than they are now.  

(Many of the recent protocol specs are far too long or complex.
I struggle to understand the reasons for this, but support for
security, scalability, and internationalization don't seem to be 
primary factors.)

> creates their documents on PCs."

If you want to produce such tools, make them available, and encourage
people to use them, feel free to do so.  If they really did help people
write good RFCs, and enough people become enthusiastic about them, 
then and only then should we consider adopting them wholesale.

I'll merely repeat my observation that adding complexity - to tools
or document formats or whatever - makes it more rather than less
difficult to write good technical prose.  (I have no doubt that you 
can produce something substantially better using better tools - but
it requires a lot more work to do so.)

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 14:45:41 -0500

Keith

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 15:25:31 -0500 (EST)

> complaints that say it's too difficult to produce plain text.

Could you provide more specifics of the complaints? Obviously, no one
wants to invest a lot of effort in something that no one uses because it's
too complex. Understanding the nature of the difficulties with the
existing system would help us create an alternative that (hopefully)
avoids or minimizes those difficulties.

> further by requiring them to use an unfamiliar text production system.

Understood. And I agree. I don't want to do anything to make author's
lives more complex. I would hope that another approach might even make
their lives /easier/. No details on how, just yet, since those depend on
the details requested above :-)

> encouraging people to make RFCs longer than they are now.

Readers will still be able to retrieve flat ASCII, good ol' RFCs that are
indistinguishable from those offered today. That is a basic goal of my
proposed system. 

> then and only then should we consider adopting them wholesale.

"Working code". I understand.

> it requires a lot more work to do so.)

This depends on the tools, I believe. It falls into the "managing
complexity" domain. I believe that the tools in use right now don't
necessarily make the lives of the individual authors easy (see your
assertions above about complaints that even plain text is too difficult),
and they certainly make the lives of the document 'users' difficult
(witness the threads about strategies for dealing with the volume of
RFCs).

I look forward to receiving those insights from you so we can start
working on a sound technical solution together.

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 15:25:31 -0500 (EST)

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 15:55:46 -0500

> Could you provide more specifics of the complaints? 

The typical complaint is from someone using MS-Word who can't get it 
to produce text in "RFC format".  Even when they use the recommended
template, it either generates gratuitous overprinting, white space, 
and occasional bogus characters, or it crashes.  

Another complaint I get is that it's more difficult to produce 
documents in MS-Word than in nroff, thus the RFC format favors 
the UNIX folks.  (except for MS-Word reliability concerns, the 
argument is easily refuted - both MS-Word and nroff can produce
something close to RFC format, and each needs a similar amount 
of post-processing to produce a completely valid document.)

But the point is that "use another tool" isn't an attractive option 
either.  The complaints demonstrate that people would rather use MS Word 
(or whatever their favorite word processor is) and fight it, than use 
a tool that does a better job at producing RFCs.  (e.g. Bare emacs  -
even without nroff or LaTeX - works quite well for writing RFCs, and 
versions are available for free on every platform I know of, but 
few people are willing to invest the effort to learn a new editor.)

Keith

---

From: Keith Moore <[email-masked]>
Date: Tue, 01 Dec 1998 15:55:46 -0500

Keith

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 16:03:10 -0500 (EST)

> few people are willing to invest the effort to learn a new editor.)

Understood. Fortunately, I'm an emacs user :-) However, in the interest of
making this work, I'd like a show of hands, please ;-) Those of you
following this thread who actively write/modify RFCs, could you please
send me (private email, please) the names and versions of the tools you
use, and any relevant weblinks.

Thanks!

- t.

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 16:03:10 -0500 (EST)

Thanks!

- t.

---

From: Fred Baker <[email-masked]>
Date: Tue, 01 Dec 1998 14:33:41 -0800

At 02:45 PM 12/1/98 -0500, Keith Moore wrote:
>internationalization.

We certainly wouldn't want to burden RFC authors with having to think about
technical issues...

---

From: Fred Baker <[email-masked]>
Date: Tue, 01 Dec 1998 14:33:41 -0800

---

From: Randy Bush <[email-masked]>
Date: Tue, 1 Dec 1998 15:19:53 -0800 (PST)

> We certainly wouldn't want to burden RFC authors with having to think
> about technical issues...

two curmudgeonly observations:

  o significantly more traffic on this list is generated about items of
    form, where to hold meetings, second-guessing the secretariat, how
    rfcs are formatted, ...  than issues of technical substance.

  o taking the last names of those pro and con ascii and grepping the
    rfc-index for those names produces one expected result left as an
    exercise for the student, and one unexpected result, 2170-2216 have 
    duplicate listings. [ yes, i told jkrey ]

randy

---

From: Randy Bush <[email-masked]>
Date: Tue, 1 Dec 1998 15:19:53 -0800 (PST)

randy

---

From: [email-masked] (Stuart Lynne)
Date: 2 Dec 1998 00:15:45 GMT

In article <[email-masked]>,
>prepress and publication.

So this seems a sensible approach.... who is going to write up an RFC?

And should it be in the new format and therefore self-referential?

:-;

Seriously it does sound like a reasonably good approach.

-- 
Stuart Lynne <[email-masked]>      604-461-7532      <http://www.fireplug.net>
PGP Fingerprint: 28 E2 A0 15 99 62 9A 00  88 EC A3 EE 2D 1C 15 68

---

From: [email-masked] (Stuart Lynne)
Date: 2 Dec 1998 00:15:45 GMT

:-;

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 13:44:29 -0500 (EST)

I don't understand how proposing that we write RFCs in HTML is any better
than SGML or XML, at least from the "I can't produce them using my
favorite word processor" perspective. What I mean by that is, every word
processor HTML export implementation I've seen is seriously buggy and/or
generates atrocious HTML. If we /were/ to use HTML to allow for the
inclusion of something better than ASCII graphics, I would submit that we
should /still/ want to hammer down standards about what tags to use where
in the document. And at that point, given that most word processors can't
generate clean HTML to save their lives, we might as well be writing a
full DTD for a 'clean' markup language, instead of hacking onto HTML.

Furthermore, since modern WPs seem to provide both macro languages and
template mechanisms of some sort (though admittedly geared toward
presentation moreso than structure), I believe we can quite readily help
authors out with tools to translate well-structured documents from their
WPs into an IETF DTD. All of that is merely a transitional tool, until
there is widespread support for XML (which I suspect we'll see).

As for your last point, about RFCs being "too simple" to warrant SGML or
XML, I must wholeheartedly disagree. There is no information that is "too
simple" to warrant being presented in a structured fashion. As I covered
my reasons for wanting a more structured form in my previous message,
though, I'll not restate them here.

---

From: Tripp Lilley <[email-masked]>
Date: Tue, 1 Dec 1998 13:44:29 -0500 (EST)

---

From: Randy Bush <[email-masked]>
Date: Tue, 1 Dec 1998 10:24:28 -0800 (PST)

> De facto, all of the lengthier documents, ...  are prepared by their
> authors orginally in some richer format

is this an argument for or against non-ascii?

randy

---

From: Randy Bush <[email-masked]>
Date: Tue, 1 Dec 1998 10:24:28 -0800 (PST)

randy

---

From: [email-masked] (Claus André Färber)
Date: 02 Dec 1998 15:50:00 +0100

> just as people currently use MS-Word, nroff, LaTeX, and other tools.

Exaclty. However, developing a single DTD for all RFCs and IDs and  
distributing the XML version (in addition to the tty-ASCII version) if  
the authors used XML could be a good thing.

> nroff, and LaTeX, I haven't heard of an RFC being produced with SGML

Just wait. ;-)

--
Claus Andre Faerber <http://www.faerber.muc.de>
PGP: ID=1024/527CADCD FP=12 20 49 F3 E1 04 9E 9E  25 56 69 A5 C6 A0 C9 DC

---

From: [email-masked] (Claus André Färber)
Date: 02 Dec 1998 15:50:00 +0100

Just wait. ;-)

---

From: [email-masked] (Claus André Färber)
Date: 02 Dec 1998 15:50:00 +0100

Just wait. ;-)

---

From: [email-masked] (Claus André Färber)
Date: 02 Dec 1998 15:50:00 +0100

Just wait. ;-)

---

From: [email-masked] (Claus André Färber)
Date: 02 Dec 1998 16:03:00 +0100

> complaints that say it's too difficult to produce plain text.

You don't. You do get complaints that say it's too difficult to produce  
plain text documents that are formatted in the way you want them.

The problem with producing RFCs is that there is no real set of tools  
that make it as easy as possible.

nroff is not available on every platform and misses some important  
features.
People wanting to write drafts and RFCs want a source format in which  
they only write their text, pass it through a processor and be finished.

Nroff is too complex - it bothers authors with details like indention,  
headline numbering etc. LaTeX would be much better - if we had a good  
documentclass. Word processors usually can't produce good preformatted  
plain text.

> I don't want to burden authors

You wouldn't. Actually, providing an easy-to-use, one-step production  
system would make authors' lives much easier.

---

From: [email-masked] (Claus André Färber)
Date: 02 Dec 1998 16:03:00 +0100

---

From: Henning Schulzrinne <[email-masked]>
Date: Wed, 02 Dec 1998 12:35:29 -0500

Claus André Färber wrote:

> documentclass. Word processors usually can't produce good preformatted

We (RTP, SIP and RTSP authors) have used LaTeX for a number of large
drafts and RFCs. We found the automatic section numbering and the
ability to cross-reference sections quite helpful. In a large draft,
maintaining section references manually is basically hopeless. We
haven't done this, but adding an index would also be relatively easy.
However, dvi2tty is too unpredictable to be of much use, so I ended up
writing a conversion tool that takes a subset of LaTeX, including simple
tables, and outputs nroff (ms macros) or HTML. Aside from a few
surprising nroff glitches (like having a major chunk of text re-arranged
due to some nroff-looking piece of ASCII-art), this has worked
reasonably well for drafts of well over a hundred pages. This makes it
trivial to produce ASCII, PostScript (for the I-D editor), PDF (for the
Win95 crowd that doesn't like PostScript) and HTML specs, from a common
source. A LaTeX macro automatically either includes the PostScript
figure or the ASCII art. Compared to manually editing ASCII, creating
floats also makes it less likely that figures or code snippets have to
be broken across pages. (You do end up with more flipping back and
forth, but you can't have both...)

LaTeX also has the advantage that among all the PostScript and PDF
sources in common use, LaTeX with dvips produces the most troublefree
PostScript and PDF (if you use the native PostScript fonts rather than
the cm fonts that are unreadable in Ghostview or PDF). [Microsoft Word
PostScript is the number one troublemaker, based on my experience
running several large conferences.] There are versions of LaTeX that
support hyperlinks, so if links to any cited RFC is your thing, the HTML
version (and maybe even PDF version) can do that, too.

One other major advantage is that I can use an automatically generated
bibtex database of RFCs, I-Ds and ITU specs, so that there is less
likelihood that references have mistakes (or have to be typed in by
hand).

Another advantage is that the format works well with CVS for version
management, which makes multi-author document production much less
painful.

Changebars work for PostScript, but unfortunately not for ASCII. [Does
nroff do changebars?] (This is one area where I find PostScript of some
advantage for drafts in the later stages of the process - it does make
scanning for changes a lot easier.)

There are rfc.sty and i-d.sty files that do all the right things in
terms of ISOC copyright, front matter and so on.

Clearly, LaTeX is not for everyone, but there are at least tools for all
the major platforms, including commercial GUI tools. I don't see the
current HTML tools as being suitable for large documents. They don't do
automatic section numbering or have any support for bibliographies,
among other problems. Plus, the current HTML-to-ASCII conversion is
generally poor.

If you are interested in these components (.bib files and l2x
converter), let me know.
-- 
Henning Schulzrinne   [email-masked]
Dept. of Comp. Sci.   ph  +1 212 939-7042
Columbia University   fax +1 212 666-0140
New York, NY 10027    http://www.cs.columbia.edu/~hgs

---

From: Henning Schulzrinne <[email-masked]>
Date: Wed, 02 Dec 1998 12:35:29 -0500

Claus André Färber wrote:

---

From: Keith Moore <[email-masked]>
Date: Wed, 02 Dec 1998 15:10:16 -0500

> that make it as easy as possible.

Sure.  But you could make the same statement about any other format.

And if you take any other document format, the set of tools
for producing, reading, searching that format, that are widely 
available, and that people are willing to use, are worse than those
tools available for producing, reading, and searching plain text.

Put another way, there's no real problem with producing RFCs except 
for those stubborn people who don't want to learn new tools.

Keith

---

From: Keith Moore <[email-masked]>
Date: Wed, 02 Dec 1998 15:10:16 -0500

Keith

---

From: Jon Crowcroft <[email-masked]>
Date: Thu, 03 Dec 1998 08:49:15 +0100

In message <[email-masked]>, Henning Schulzrinne typed:

 >>drafts and RFCs. 
 
metcalfe's law says that the value (power) of a network is the square
of the number of users - however, you can refine this law - a network
need not just be a shared ethernet or NBMA switch full of IP hosts, or
an intranet, or the web - it can be a virtual network of collaborators
- but then you need to define 
i) the distributed application (e.g. document authoring and reading)
ii) the interface between the distributed application

RFC and I-d's are collaborativbely authored ,and continually revised -
this latter point is implicit in most people's posting's here,
although often people confuse the application program (e.g. latex,
nroff, framemaker, word, etc) with the distributed itnernet
application  itself (distributed authoring and revising and reading)

the Internet at all levels has a major design success embedded in the
idea of "hourglasses" - for the transmission
network (datagram service, addressing, routing) this
is IP - for email its SMTP, for WWW its http and URLs, both the latter
supported by a parsimonious name service, namely DNS etc

for internet authoring to date, the exchange protocol is not clearly
specified, but as bob braden pointed out, the interface is - the RFC
and I-d formats are, and even to the extent of a set of (not well
specified) conventions for ascii art for 
packet formats
topologies
time-sequence diagrams
equations
and so on-  this means that the format for exchange of documents
between authors _using different authoring packages_ is ameneable,
even to non-english speaking/writing users to some extent

A legitimate reason other organisations do not take this approach can be seen
in the actions ofthe ACM and IEEE in their electornic publications and
digital library efforts - these typically provide index and search on
author/title/abstracts, but provide papers in PDF (and postscript)
only, because these are stamped as referreed, reviewed, and ARCHIVAL
documents - i.e. they are immutable - for lots of good reasons.....including
(but not just) : if you want to modify it you have to leave available for 
later scholars, in the archive, the set of earier papers.....you want to
able to prevent trivial plagiarism - 
none of these goals is true of an open standards organisation - note
that the ITU and ISO have confused the roles of standards and
publication, and explicitly attempted to earn revenue at various
points thru being a publisher - this is at odds with the standards
developement role....

now someday, an exchange format such as XML, which is open, AND widely
available as a backend for most common tools, will replace ascii - but
right now it isn't and I believe we are left with the worst of all
possible exchange formats, except for all the rest......

 cheers

   jon

p.s. prof g. colouris at QMW wrote a note on this  - a version with
some additonal economic sctibbles by me is at
http://www.cs.ucl.ac.uk/staff/jon/doc-std.html
comments (but not revisions:-) welcome...

---

From: Jon Crowcroft <[email-masked]>
Date: Thu, 03 Dec 1998 08:49:15 +0100

cheers

   jon

---

From: "Frank O'Dwyer" <[email-masked]>
Date: Fri, 04 Dec 1998 22:53:21 +0000

Tripp Lilley wrote:
> supporting essentially 'ASCII' docs through UTF-8's magic ASCII-ness.

Yes please. Given even a simple DTD, anyone could write a backend to
produce any rendering (HTML, RTF, etc.) they liked, and people could
choose their favorite. An XML hacker could probably knock out a backend
to cut the current ASCII format in half an hour.

(And that's just the rendering. As you rightly point out, there are many
other benefits.)

Cheers,
Frank O'Dwyer.

---

From: "Frank O'Dwyer" <[email-masked]>
Date: Fri, 04 Dec 1998 22:53:21 +0000

Cheers,
Frank O'Dwyer.