Subject: Re: User Petition on Standards to Netscape and Microsoft

From: "William Allen Simpson" <[email-masked]>
Date: Sun, 28 Dec 97 22:18:30 GMT

This may be a real improvement to the process.  Another message
indicates that the newer documents will follow IETF practice; on-line
and freely available.

But I have a very basic question: Since the replacement is not backward
compatible, why was it not given a new label, such as ASN.2?

As to S/MIME, I respectfully request that all references to ASN.1 be
removed.  Most of the definitions can easily be re-cast in our own RFC
BNF-style syntax.  There would be no changes to bits on the wire.

Many of the PKCS uses of ASN.1 can be treated as simple octet-string
"magic numbers".  For example, the PKCS-1 (and PKCS-7) representation of
a message hash prior to signing is simply:

    DigestInfo ::= SEQUENCE {
      digestAlgorithm DigestAlgorithmIdentifier,
      digest Digest }

    Digest ::= OCTET STRING

    ...

    md2 OBJECT IDENTIFIER ::=
      { iso(1) member-body(2) US(840) rsadsi(113549)
          digestAlgorithm(2) 2 }
    md4 OBJECT IDENTIFIER ::=
      { iso(1) member-body(2) US(840) rsadsi(113549)
          digestAlgorithm(2) 4 }
    md5 OBJECT IDENTIFIER ::=
      { iso(1) member-body(2) US(840) rsadsi(113549)
          digestAlgorithm(2) 5 }

These can be reduced to octet strings in the specification.  There is no
technical security reason that the ASN definition space was used.  It is
just a unique cookie.  Any form of unique cookie will work!

In fact, defining the resulting octet string will be a great boon to
developers, as we will not have to purchase a compiler to generate
such simple byte constants.  It has been historically problematic to get
them correct just using our heads....

> From: Bancroft Scott <[email-masked]>
> Due to how widely ASN.1 is used outside of ISO and ITU-T (e.g., electronic
> commerce, radio paging, aviation, transportation industry, etc.) it was
> placed in a new category within ISO a few months ago.  In this new
> category you no longer have to be affiliated with a "National Body"
> standards organization to participate in controlling ASN.1.  Anyone off
> the street can now participate in ASN.1 standards meetings, whether face
> to face or by email, without having to a formal representative of a
> national body standards organization and without having to pay yearly dues
> such as are required by ITU-T and typically by ISO member bodies.  This is
> a new process and I will be able to say more about it in about two and a
> half weeks.
>

[email-masked]
    Key fingerprint =  17 40 5E 67 15 6F 31 26  DD 0D B9 9B 6A 15 2C 32

---

From: Karl Auerbach <[email-masked]>
Date: Sun, 28 Dec 1997 15:40:05 -0800 (PST)

> them correct just using our heads....

You know the old joke: A camel is a race horse designed by a committee. 

It is amazing how many people can't get ASN.1/BER encoding of the tag (or
tags) and length fields correct. 

Oops, sorry, it isn't really that amazing when one examines the rather
weird ways in which BER has rules for encoding that shift the encoding
algorithm depending on type of thing [tag, length, or data] or the
particular value range being encoded.  (For example, the encoding
*algorithm* for encoding the length fields shifts depending on the value
of the length field.  The same holds true for the encoding of tags.) 

The compilers that invoke pre-written encoding/decoding methods are
usually safer to use but they don't necessarily do well for the memory
footprint of the resulting code (which is still very important for folks
building embedded systems, and still important for everyone else.)

And, of course, because ASN.1 (at least the older forms - I don't know
about neo-ASN.1) doesn't have the concept of unsigned integers (which are
kinda popular items in protocol fields).  Thus anyone who is sloppy about
signed/unsigned issues is eventually going to get badly burned even if
they use a compiler.

(Actually, the implementor rarely gets burned; the problems show up after
the implementation is put into production and a value gets into the range
where the would-be sign bit gets turned on.  In other words, the customer
is more often the one who gets burned.) 

Of course, if we restrict ourselves to octet strings, then why bother with
ASN.x at all?

By-the-way we should not forget that there are other encoding methods
under ASN.1 than BER.  (But that doesn't fix things like the lack of
unsigned integers.)

		--karl--

---

From: [email-masked] (Christian Huitema)
Date: Mon, 29 Dec 1997 13:05:44 -0500 (EST)

I must say that I fully agree with Bill Simpson  on the advantage of
completely separating S-Mime from ASN.1.   Yes, ASN.1 has changed over
time.
It went from a limited and somewhat broken form in 1984/1988 to 
a very ambitious object-oriented form in 1994.  Doing so, it did not
only correct a few bugs, but also added a whole set of new
"functionalities." basically, it promots the ASN.1 compiler to a central
role in the implementation, where it should not only provides
encoding and dcoding stubes, but also a complete database of options,
tabless and controls.  The chances of getting a correct implementation
without such a compiler are about nil.
This is precisely the proble.m.  Compilers introduce all kinds of
weirdnesses in protocol development.  They encourage gratuitous
complexity.  They tend to generate very large code.  They also tend
to  make code hard to prot; specifically, they make it bvery difficult
to mix modules developed with different compilers,because each one
generates its own API and  uses its own set of libraries.
When the task at hand is simple -- and the specification of S-MIME 
certificates ought to be simple --  the solutions that Bill proposes
makes a whole lot of sense.  Just specify the bits on
the wire using some good old notation.  It may well be that the
notation will closely match ASN.1.  But it will definitely
improve portability, reliability and interoperability.

-- 
Christian Huitema
----------
See you at INET'98, Geneva 21-24,July 98 http://www.isoc.org/inet98/

---

From: "Phillip H. Griffin" <[email-masked]>
Date: Mon, 29 Dec 1997 15:03:20 -0500

Christian,

With all due respect, I must say that I completely disagree.
I decided not to respond to the post from Bill Simpson, because
some of the assertions seemed so off base from my perspective, 
and I was not really sure if he was serious after reading statements 
like "It has been historically problematic to get them correct just 
using our heads." Your note leads me to conclude otherwise.
 
Some comments inline below...

> without such a compiler are about nil.

Tell that to all the folks using toolkits that do not depend on 
a formal ASN.1 compiler. I recall that Verisign and Terisa Systems,
were major contributors to the development of Visa and MasterCard's 
SET protocol. SET is defined in ASN.1:1994, and makes extensive use 
in its definitions of table constraints, information object sets, 
parameterized types, and supports national languages through the use 
of BMPString. The reference implementation, is written in C, and does 
not require an ASN.1 compiler. 

While it is true that an ASN.1 compiler may couple the base coding
functions with higher level functionality (user defined constraints,
permitted alphabet constraints, component relation constraints, etc.)
there is no law that states that such coupling is mandatory. ASN.1:1994
merely makes such coupling possible, since the corrected 1994 notation
can be machine processed. This was not always possible using ASN.1:88,
particularly when MACROs were involved.

There is no one best approach that will suit all implementations. In 
SET, for example, it is possible to greatly simplify the ASN.1 and
eliminate all of the E*{} parameterized types, replacing them with
PKCS #7 EnvelopedData. All of the component relation constraints and
the user defined constraints can also be removed from the SET ASN.1,
so long as the implementor takes on the burden of writing the necessary
code to replace the fuctionality lost in a modified version of the 
ASN.1. You can find precise examples of such simplifications under
the http://www.oss.com/setasn1.htm URL.

The approach varies with the implementor. Some folks insist on
taking a standard "as is" and pumping it through a compiler, and 
have great fear in introducing interoperability problems in making 
any modification whatsoever. Others hire someone with strong ASN.1
skills to carefully rewrite standards specified in ASN.1 in ways
that best suit their needs. 

> complexity.  

I've done it three ways, hand coding, with coding toolkits, and with 
a compiler. There is absolutely nothing more complex or prone to
error than trying to perform an automatable task by hand - a hand 
job. Where compilers can really shine is in insulating an application
from frequent changes to an ASN.1 specification. 

As far as gratuitous complexity goes, I disagree that this can be 
blamed solely on ASN.1, and often arises from the complexity of 
the problems to be solved (or in the perception of the problems to 
be solved). Look at Ford's lengthy book explaining Certificate 
policy extensions, for example. The ASN.1 there is relatively 
simple, clean and neat, but understanding the ASN.1 definitions 
gets you nearly nowhere close to understanding Certificate policy 
implementation issues. Without Ford's book, I think I'd give up.

> They tend to generate very large code.  They also tend

If true, how do you explain the extensive use of ASN.1 for
cell phone messages, air to ground systems, and the new 
ITU-T wireless standards?

> improve portability, reliability and interoperability.

Phil
-- 
Phillip H. Griffin         Griffin Consulting
[email-masked]        ASN.1-SET-Java-Security
919.828.7114               1625 Glenwood Avenue
919.832.7008 [mail]        Raleigh, North Carolina 27608 USA
------------------------------------------------------------
          Visit  http://www.fivepointsfestival.com
------------------------------------------------------------

---

From: Bancroft Scott <[email-masked]>
Date: Tue, 30 Dec 1997 04:34:13 -0500 (EST)

> indicates that the newer documents will follow IETF practice; on-line

I don't know this to be the case.  I'll be able to say better towards
the end of next week.
 
> compatible, why was it not given a new label, such as ASN.2?

Same reason that C was not renamed D after the bugs were cleaned up,
yielding a better but non-backward compatible version of C.  And the same
reason that C++ was not renamed to C+++ when it was cleaned up, yielding a
better but non-backward compatible version of C++.

The essense of ASN.1 was not changed between 1988 and 1994, and ASN.1
modules can be written that conform to both 1988 and 1994 ASN.1 so long as
they don't use any of the buggy notation corrected by the 1994 version of
ASN.1 (e.g., macros), or use any features unique to 1994 ASN.1 (e.g.,
BMPString).  This is not unlike what happened when C and C++ got their
last major facelift. 

> BNF-style syntax.  There would be no changes to bits on the wire.

This is peculiar.  When X.208 came out in 1988 one of the the big
complaints was that it was possible to write valid ASN.1 modules that was
not compilable - meaning that encoders/decoders could not be automatically
be generated by it.  Now you propose going back to a notation that cannot
be compiled to produce encoders/decoders?  That's not a step forward.  It
is best to use a notation that allows use of compilers, runtime toolkits
or manual encoding/decoding.

--------------------------------------------------------------------------
Bancroft Scott                                Toll Free    :1-888-OSS-ASN1
Open Systems Solutions, Inc.                  International:1-609-987-9073
[email-masked]                                  Tech Support :1-732-249-5107
http://www.oss.com                            Fax          :1-732-249-4636

---

From: Bancroft Scott <[email-masked]>
Date: Tue, 30 Dec 1997 05:01:15 -0500 (EST)

> BMPString).  This is not unlike what happened when C and C++ got their

I forgot to mention that another reason that ASN.1 was not changed to
ASN.2 is that in addition to the changes being mostly bug fixes, none of
them affected the encoding of messages.  Ensuring that the bits on the
wire were not changed was a key objective and it was achieved.

---

From: [email-masked] (Christian Huitema)
Date: Tue, 30 Dec 1997 10:14:22 -0500 (EST)

> wire were not changed was a key objective and it was achieved.

Uh?  I certainly agree that the ASN.1 MACRO was not exactly easy to
compile, but ASN.2 contains much more than bug fixes -- it does not
only add a colon here or there in the syntax, but it goes on defining
a whole new set of concepts including tables, objects, constraints,
etc.  While ASN.1, if you remove the macro notation, is a relatively
benign interface description language, ASN.2 is a monstruous behemoth,
very much the last brain child of the OSI effort.

I would advocate that IETF standards should never use ASN.1.  In the
rare cases where notations have to be imported from ITU documents,
then we should take the approach that Bill suggest, i.e. to explictely
recommend one specific encoding rule, removing all gratuitous variations,
and presenting the binary encoding format.

---

From: Stephen Kent <[email-masked]>
Date: Tue, 30 Dec 1997 10:55:16 -0500

Christian,

	Enough people successfully make use of ASN.1 compilres in products
that I think it unnecessary to argue that we avoid using it at this point.
I agree that the newest version of ASN.1 is not yet well supported by as
many compilers, including freely available ones, as I would like.  But that
will always be trie of any new language, syntax, etc.  In PKIX we addressed
this problem by providing a version of the specs in the older (1988)
version, which has fairly widespread support.  However, there is no
difference in the ultimate encoding, echoing a sentiment that Bancroft
cited earlier.  We merely choose to use an older syntax to make it easier
for more people to use freely available compilers.

Steve

---

From: Bancroft Scott <[email-masked]>
Date: Tue, 30 Dec 1997 11:30:08 -0500 (EST)

On Tue, 30 Dec 1997, Christian Huitema wrote:

> only add a colon here or there in the syntax,

Bug fixes, Christian, bug fixes.  The colons were added in CHOICE value
and open type value strictly to correct ambiguities in the grammar.  They
are present for no other reason.

> but it goes on defining
> etc. 

No, Christain, these are not new concepts.  They are are precisely what
macros and ANY DEFINED BY sought to define but failed.  Take for example
the OPERATION macro. It was used to define macro instances (this same
concept is "information objects" in ASN.1:1994) that were meant to relate
"identifier" values with types carried within ANY DEFINED BY's.  However,
the notation relied upon your intuition to figure out that the macro
instances were to be grouped together to form a table (this same concept
is "information object sets"  or "tables" in ASN.1:1994), or if you were
lucky you could find text that suggested that this is what you should do
with the macro instances.  The 1988 notation then went on to to define ANY
DEFINED BY's with an "identifier" (this same concept is a "constraint" in
ASN.1:1994) which in our example would be of type OPERATION; however, this
constraint failed miserably, for it did not provide a syntactic link
between the macro instances and the ANY DEFINED BY.  Worse, it had the
very serious flaw that revealed itself only at implementation time when
you tried to use the ANY DEFINED BY whose identifier is an INTEGER type,
should you try and combine several modules that define identifiers of type
INTEGER.

Contrast this with the fulfillment of these concepts in ASN.1:1994, which
requires no intuition and guess work on your part, where the relationship
between identifiers and open types (the replacement for ANY DEFINED BY)
are clearly spelled out via the syntax.

> While ASN.1, if you remove the macro notation, is a relatively

Pagh!  More of the "not invented here" nonsense.  Take a look at how
rapidly Microsoft was able to develop NetMeeting, or Intel its VideoPhone
products, or how quickly the many others in the audio/video over Internet
arena were able to get the products to market, then explain to me how they
could have arrived at interoperating products in such short time if
ASN.1:1994 was anything short of beautiful?  Audio/video is just one
example, I can go on for pages describing the other successful uses of
ASN.1:1994 if you wish.  The wide adoption of ASN.1 outside of the OSI
arena since 1994 is testament to the major improvement that ASN.1:1994 is
over the 1988 version, and how good a solution ASN.1 is overall.

---

From: Karl Auerbach <[email-masked]>
Date: Tue, 30 Dec 1997 10:33:35 -0800 (PST)

> example

Ummm, those of who build audio/video products based on RTP/RTCP and which
carry higher speed streams (near broadcast quality) don't use ASN.1 in any
flavor, shape, or form. 

Indeed, when doing high performance audio/video, one wants the CPU doing
productive things.  We don't want the CPU wasting cycles bit twiddling
through the complicated encodings of ASN.1's BER. 

(I am aware of the arguments that say that modern processors are memory
starved and that because fully compacted ASN.1 BER uses few bytes that it
can actually be faster than something like XDR.  But I have not seen that
argument borne out, or even given signs that it might have validity, in
practice.)

By-the-way, we should be careful not to slip into the mistake of not
distinguishing clearly between:

  - ASN.1 the language for expressing data structures

  - ASN.1 the macro language for use (and abuse) as a means of expressing
    higher level structures.

  - The underlying Encoding Rules, such as BER, which put ASN.1 data
    structures into a wire format (and vice versa.)

		--karl--

---

From: Chris Newman <[email-masked]>
Date: Tue, 30 Dec 1997 12:43:08 -0800 (PST)

> and presenting the binary encoding format.

I concur.  We have ABNF (RFC 2234) for representing syntax, and it
encourages textual protocols.  Textual protocols can be debugged with
off-the-shelf or free network snoopers and telnet clients.  They can be
written with libraries built into most programming languages (e.g.,
fprintf) and they make up 80% of the most successful IETF application
protocols.  We have a notation and application protocol design technology
which is not only proven in the field, but is far more successful than
competing ASN.1 based protocols.  Given that, I think ASN.1 based
application protocols should be rejected without a strong justification.

In those rare cases where an application protocol or object contains
mostly binary data, it is far more productive to use a simple
tag-length-value format (with a single canonical representation) than an
ASN.1 encoding.  PNG (RFC 2083) and SSH (draft-ietf-secsh-*) are good
examples of this design.  Finally, when space is at a premium, one can
simply use a ZLIB layer (RFC 1950) to achieve better compression than a
complex packed encoding.  This can all be done with freely available tools
anyone can get today, and is documented in clearer and shorter
specifications than ASN.1 and its encodings.

		- Chris

---

From: Marshall Rose <[email-masked]>
Date: Tue, 30 Dec 1997 13:55:47 -0800

> > I would advocate that IETF standards should never use ASN.1.
> off-the-shelf or free network snoopers and telnet clients.
> ...

thank you for a very concise statement of the solution.  while ASN.x has
its uses, i think it is fair to say that its pain has, regrettably,
outweighed its gain in the Internet space.  this may not be true in
other spaces, but there is a clear preponderence of the facts from the
ietf perspective.

/mtr

---

From: Stephen Kent <[email-masked]>
Date: Tue, 30 Dec 1997 17:50:47 -0500

Marshall & Chris,

	I would not disagree with the characterization of textual protocols
as  easier to debug without the need for more sophisticated tools. However,
debugging ease is not the only consideration when designing protocols.  IP,
TCP, UDP, PPP, and other widely used Internet protocols are not textual,
yet we have managed to debug them and deploy interoperable implementations
for many years.  We respectually disagree about the relative merits of
using non-textual syntax for protocols, whether it's ASN.1 or alternatives.

Steve

---

From: Stephen Kent <[email-masked]>
Date: Tue, 30 Dec 1997 17:50:47 -0500

Marshall & Chris,

Steve

---

From: Marshall Rose <[email-masked]>
Date: Tue, 30 Dec 1997 15:54:16 -0800

> We respectually disagree about the relative merits of

all too true, i'm afraid.  for myself, i whole-heartedly reject
complexity.  it serves no good purpose.

> However,
> for many years.

an interesting selection.  could you perhaps cite a successful example
of this approach at the application layer?  (if you wish to claim that
the dns is an application-layer entity, then i'll give you one...)

the problem with your examples is that they focus on one rather narrow
slice of the architecture.  the lower layers are well served by tightly
optimized, highly constrained packet formats.  while i wouldn't want to
use text for them, i certainly wouldn't want to use asn.1 either.

and herein lie the rub: history shows us that:
    - the higher layers work best with abnf; and,
    - the lower layers are best served by 32-bit packet formats rendered
      in ascii art
where does that leave poor asn.1, i wonder?

/mtr

---

From: [email-masked] (Christian Huitema)
Date: Tue, 30 Dec 1997 19:00:31 -0500 (EST)

Steve,

IP and TCP are optimized for speed of processing, with the goal of
handling billions of packets per second some day.  If you take their
example, you end up specifying fixed position + fixed length fields.
I have done extensive measurement of marshalling protocols. Basically:

1) Text is slightly slower than variable length binary,

2) Shorter is almost always better,

3) Complexity kills you, because it increases the size of the code
   and thus jeopardize your usage of cache memory,

4) The difference between text and binary is dwarved by the cost
   of network transmission, specially through socket interfaces,

5) The cost of encryption, be it DES, RSA or DSS is orders of magintude
   higher than the small variations in the cost of marshalling.

In short, other factors, not performance, decide which marshalling
technique one will use.

---

From: Stephen Kent <[email-masked]>
Date: Tue, 30 Dec 1997 19:21:09 -0500

Marshall,

Complexity is not an inherently desirable feature, unless you're an author
or consultant getting paid to explain complex protocols and systems to you
readers and clients.  So, how many books have been written trying to
explain the complexity of RFC 822 e-mail, sendmail, the DNS, ...?   Textual
syntax does not make complex protocols simple and one might even argue that
it gives the impression of simplicity because its textual.  We have some
amount of experience with complex protocols in both types of formats, and
I'm not sure that the choice of format has been a distinguishing factor.

You're right that the protocols I cited are lower layer ones where tighter
encoding and fixed formats are important for better performance.  Also,
they are admittedly less complex (syntactically) than the application
protocol formats, and thus have less need for the flexibility (and
attendant complexity) that we see in applications. So, on those points we
agree.  The real question is whether the greater complexity intrinsic in
application protocols is best served by textual representations and ABNF or
by any form of binary encoding and an associated syntax.  I just don't
accept the premise that our experience shows the intrinsic superiority of
textual formats and abnf vs. binary formats and ASN.1.

Steve

---

From: Chris Newman <[email-masked]>
Date: Tue, 30 Dec 1997 16:42:51 -0800 (PST)

> using non-textual syntax for protocols, whether it's ASN.1 or alternatives.

If you read my message carefully, you will note that I said "application
protocols."  The four protocols you've listed are not application
protocols and therefore have different design criteria (which tend to
discourage the use of ASN.1 for different reasons).

The most successful IETF binary-encoded application protocol is Telnet,
which is generally considered a good example of how not to design a
protocol (due to the complexity of option negotiation and debugging
thereof).  It took me significantly longer to write and debug a telnet
client than it took me to write and debug clients/servers for textual
protocols.  I had to build a complex debugging service into the client
which I've never needed in textual protocols.  I will note that telnet has
orders of magnitude more deployment than its ASN.1 based counterpart which
is yet another strike against ASN.1, even when a binary encoding is used.

As an application protocol developer, I trust the judgement of lower-level
protocol developers to make the right choice in the binary vs. text
encoding tradeoff.  But at the applications level, both IETF history and
my personal experience weigh heavily in favor of textual protocols.  Let's
not ignore our successful history.

		- Chris

---

From: Marshall Rose <[email-masked]>
Date: Tue, 30 Dec 1997 19:58:48 -0800

> readers and clients.

speaking as, among other things, an author and a consultant, i'd have to
say that complexity isn't a particularly desirable feature.  there are
always things to write about, issues to advise on.  better to spend
one's time working on describing the silk purse of simplicity rather than
the sow's ear of complexity.

complexity, is however, a desirable feature to some, e.g., people who
would rather endlessly debate rather that simply getting work done.  for
myself, i'd rather solve 90% of a problem really well in finite (and
small) time, than spend a seemingly endless amount of time trying to
come up with the 100% solution.  take, for example, internet security.
but, i digress.

> So, how many books have been written trying to
> explain the complexity of RFC 822 e-mail, sendmail, the DNS, ...?

not very many.

a quick search of amazon.com yields:
    - eighty-one books with MIME in the title, but only one is on e-mail
      the rest are on performance art
    - three books with sendmail as subject or title
    - one book with smtp in the title ("X.400 and SMTP: Battle of the
      E-Mail protocols", published in august of this year... "Earth to
      Author, Earth to Author: The War's Over.")

> Textual
> it gives the impression of simplicity because its textual.

the point is not to make complex protocols simple.  the point is to
design simple protocols.  people who design with this in mind, at the
application layer, will favor textual syntax.  they have no need of the
complexities of ASN.1, because they will never go there in their design.

> We have some

maybe i've been sleeping, but could you cite a complex protocol that
uses a textual syntax?

> agree.

well, we agree on something.

> The real question is whether the greater complexity intrinsic in
> by any form of binary encoding and an associated syntax.

well, we don't agree here.  where is it mandated that

	application protocol = complexity

821, one of the great application protocols of our age, is the essence
of simplicity.  whenever someone asks me for an example of a well-designed
protocol, after a couple of disclaimers, i just toss them a copy of 821.

> I just don't

okay, fine.  however, it still remains that simplicity is better than
complexity, and that simplicty works with application protocols.

/mtr

---

From: Marc Horowitz <[email-masked]>
Date: 30 Dec 1997 23:30:06 -0500

>> the dns is an application-layer entity, then i'll give you one...)

NFS and NTP, to name a few.

I suspect a key issue is not upper vs lower layer, but how the
protocols think about their data.  To IP, UDP, TCP, etc, the addresses
and payloads are binary objects.  To SMTP, FTP (on the control
channel), and POP, the addresses and payloads are generally textual.
While I can't think of any lower layer protocols whose components are
textual, I can think of upper layer protocols whose components are
binary: NFS, NTP, anything multicast.  Unsurprisingly, the protocols
themselves are binary in nature.  I can't think of any very successful
protocols offhand which tend to mix structured binary objects (i.e.,
not octet strings) in textual protocols, or vice versa.

		Marc

---

From: Stephen Kent <[email-masked]>
Date: Wed, 31 Dec 1997 11:02:22 -0500

Marshall,

It's always a treat to see how you can turn a technical argument into a
personal one, with such eloquence, though it's less enjoyable when one is
the target of your wrath.  I didn't suggest that you, as an author and
consultant, were guility of exploiting the complexity inherent in many
Internet protocols.  'tis a pity that your response was so defensive.

I'll avoid responding to your comments re Internet security, and promise
not to bring up SNMP v3 in return.

I didn't search amazom.com, but my recollection is that there are a number
of books that try to explain Internet e-mail, MIME, sendmail, etc.  A Yahoo
search on "books+sendmail" found two examples immediately, though "MIME"
does not make for a good query.

But, back to the meat of the argument. We agree that the point is to design
protocols that are no more complex that necessary to meet the percieved
requirements.  The question is whether the use of textual syntax or binary
syntax is preferable in application protocols to achieve the agreed upon
goal or not creating extra complexity.  Perhaps my perspective is skewed
because I look at the sorts of things I have to deal with in security
protocols and they are rarely textual data. The typical building blocks for
security protocols include ciphertext (with is pure binary), one-way hash
values, sequence numbers, version numbers, timestamps, nonces, etc.  All of
these are binary and to represent them otherwise seems like added
complexity.  The real issue ought to be what sort of data are we moving
around and what added complexity is introduced by the chosen syntax.  I
believe that reaosnable folks can come to different conclusions about what
are the right answers here, and that the IETF should accommodate both camps.

Steve

---

From: Stephen Kent <[email-masked]>
Date: Wed, 31 Dec 1997 11:02:22 -0500

Marshall,

Steve

---

From: Stephen Kent <[email-masked]>
Date: Wed, 31 Dec 1997 11:23:07 -0500

Christian,

	I agree with your analysis of efficiency concerns at different
layers.  I would not argue too strongly that binary syntax, because it may
be more compact, is instrinsically preferable to text (although everytime I
download a text-encoded file over a low speed connection this thought does
come to mind).  However, if an application requires complex synatx to allow
for variable length fields, optionally present fields, etc., then it's not
clear to me that a textual representation is preferable to a binary one.

Steve

---

From: Craig Partridge <[email-masked]>
Date: Wed, 31 Dec 1997 08:57:45 -0800

Hi folks:

Some time ago I spent time looking carefully at data formats (for
net management and other purposes).  Basically my conclusion was
that from a performance angle, binary and ASCII are pretty much equal.
These results are all from memory -- flame me if I've misremembered
something.

For text, you're encoding the text about as efficiently as possible
(7-bit characters in 7-bit ASCII).  Certainly you are
more efficient than ASN.1 or XDR.

For digits, if you use all or most of the ASCII values for encoding numbers
(rather than just 0-9 or 0-F), you lose about 20-30% efficiency in your coding
(the cost of taking, say, a 32-bit value and encoding it in 7-bit ascii, which
requires 5 bytes) and efficiency loss goes down as the size of the value
being coded goes up.  XDR does better for big values and about as well for
small values -- ASN.1 does worse for all sizes.

Craig

---

From: Marshall Rose <[email-masked]>
Date: Wed, 31 Dec 1997 11:00:17 -0800

> Internet protocols.  'tis a pity that your response was so defensive.

hi.  was there a personal argument in something you wrote or i wrote?
must have missed it.  sorry, i'll pay more attention next time.

> not to bring up SNMP v3 in return.

well, okay, but v3 isn't in my repertoire.  you might recall that i
bailed from the land of snmp after taking responsibility for the v2
debacle.  v3 is the work product of others, and while i'm not paying
attention to it, some friends have recently expressed some satisfaction
with respect to the direction it is taking.

> does not make for a good query.

well, okay, but if i can find only three or four books at amazon.com, and
you can find only two at yahoo, i don't see how this supports the
assertion that there are a lot of books on the subject.

> requirements.

actually, i don't think we agree here, because i don't know if we can
agree on what "perceived requirements" are.  for example, many
requirements are, sadly, simply unrealistic.  it doesn't mean that
users are bad for wanting solutions to those requirements, but the
engineering community has to exert a level of intellectual honesty in
simply saying "no" when well-intentioned requirements outreach its
ability to deliver cost-effective, interoperable solutions in a timely
fashion.

> The question is whether the use of textual syntax or binary
> goal or not creating extra complexity.

i take a more holistic view of things: protocols don't have to be
complex, whether they are at the IP layer, the dns layer, or the
applications layer.  one should always do the simplest thing, and by
extension, use the simplest representations to get the job done.  where
we often err as engineers, is in using frameworks that encourage
extensibility in the absence of well-understood evolutionary paths.  in
the short-term, we pay a terrible cost in terms of getting a correct,
interoperable, and well-tested product out the door in a timely fashion.
in the long-term, we never truly see the benefits that the framework was
supposed to realize because none of us are any good at predicting the
future, and that our well-laid plans for evolving requirements very
rarely come out as we hope.

> The real issue ought to be what sort of data are we moving
> around and what added complexity is introduced by the chosen syntax.

perhaps.  i think this is still secondary.  the real issue i would
prefer the ietf to address is the philosophy of protocol architcture.
the sorts of data, the formats, and the like, i think are simple
byproducts, once the engineering work has begun guided by architectural
philosophy.

> I

i don't think we agree here.  i agree that reasonable people can come to
different conclusions.  i don't agree that the ietf should, or even can,
accomodate both camps.  our job is not to be all inclusive, despite the
satisfying warm-and-fuzzy it gives us; our job is to deliver
cost-effective, interoperable solutions in a timely fashion.  as
politically incorrect as it may seem, this often requires situation in
which compromise or accomodation is simply contrary to first
principles.  we should discuss, we should deliberate, and then we should
move forward.

as such, let us recall what started this exchange between us:
i asked for specific examples of successful protocols at the application
layers that use binary data.  chris mentioned the painful example (poor
ol' telnet), and marc mentioned nfs and ntp.  although i agree with marc
that these two protocols are successful, i question whether they are
truly "application" protocols or not.  even if i conceed this point,
which, pending a concensus view on the definition of an "application
layer" protocol in the Internet, i might observe that neither uses
asn.1, but rather rely on our 20-year old 32-bit ascii art syntax.

/mtr

---

From: Karl Auerbach <[email-masked]>
Date: Wed, 31 Dec 1997 12:07:58 -0800 (PST)

One more item to add to your list:

 - textual representation has potential problems due to lack of consistent
end-of-line indicators (especially on the last line, which often lacks an
end-of-line indicator.)  In addition, mapping of end-of-line often causes
difficulties when trying to validate message digests and other similar
authenticators.

		--karl--

---

From: Chris Newman <[email-masked]>
Date: Wed, 31 Dec 1997 15:35:57 -0800 (PST)

> around and what added complexity is introduced by the chosen syntax.

I agree, with the caveat that the decision should be backed up by 
successful history whenever possible.

> I believe that reaosnable folks can come to different conclusions about what

I largely disagree.  The IETF should only accommodate camps which can
technically justify their position as superior to proposed alternatives.
When we have a choice between "we know this method works well" and "this
method might work", I see no reason the IETF should accommodate the latter
option.

For the record, I do think an application protocol with a mostly binary
payload should have a simple binary encoding and consider NTP a reasonable
example of this.  But if an application protocol has a majority textual
payload then it SHOULD have a textual encoding. 

		- Chris

---

From: Simon Spero <[email-masked]>
Date: Wed, 31 Dec 1997 19:14:53 -0500 (EST)

This discussion has covered a lot of ground-much of it old, some of it
several times, butdefinitely a lot of it.  This makes it easy to me to
slip in some trailers for something I'm working on for the W3 meeting in
two weeks :-) . ASN.1 stuff will follow after this message. 

Does Text Mean BNF - Does Structure mean ASN.1?
================================================

The current discussion seems to be taking as a given the assumption
that in order to text protocols, you need to use BNF, and that any
other form of specification inevitably leads to binary encodings,
genital warts, and other assorted nastiness.

If ABNF is the best and simplest way to specify textual protocols,
then we're in trouble.  The following example comes from a recent
message to an IETF mailing list.  The author is an ABNF guru, and the
semantics that needed to be expressed are quite simple.

-----
There was a bug in the ACL BNF in draft-7 which I attempted to fix.

Draft-7 had:
   acl                = "(" *acl-identrights ")"
The RFC has:
   acl                = "(" [acl-identrights *(SP acl-identrights)] ")"
                              *(SPACE acl-identrights)] ")"
The correct version is:
   acl                = "(" [acl-identrights *(SP acl-identrights)] ")"
----

BNF is the machine code of textual protocols. I don't know about you,
but I haven't written much machine code lately.  Read, yes. Write -
no.

BNF is the QWERTY keyboard of textual protocols.  It's most commonly
stressed virtue is that it slows people down, and by getting
exponentially more complicated, puts a limit on how complex and ugly
protocols can get. This didn't stop IMAP.

If BNF is too low level a language, why don't we use a higher level
interface definition language, with encoding rules that correspond to
the kind of textual encodings that might have come from EBNF generated
by hand encoded.  What if we had a system that could take a
distributed object system like CORBA and turn it into something as
easy to implement by hand as old-fashioned http. What if we had IHOP?

IHOP - The Internet HTTP-like Object Protocol.

      Interface (IDL)				        Protocol (IHOP)
--------------------------------------------+---------------------------------
module sendmail {                           |>  send /default/SMTP IHOP/1.0
  typedef sequence<octet> body;             |>  from: [email-masked]
  exception badAddr {                       |>  to: [email-masked]
     string address;                        |>  Content-Boundary: .
  };                                        |>  
                                            |>  Subject: Dude!
  interface SMTP {                          |>
    void send(in string from, in string to, |>  Hi Bill - can Buddy come play
              in body  data) raises badAddr;|>  fetch tommorow? Hugz, Simon
  };                                        |>  .
};                                          |<  IHOP/1.0 200 OK
--------------------------------------------+---------------------------------

Remember when 'S' stood for Simple, and 'L' stood for Lightweight? IHOP.  

-----------
We now return to the ASN.1 flame war already in progress.

ASN.1
-----

A lot of the problems that people here have with ASN 1 are due to one
trivial but fatal flaw in the language itself: its case sensitivity.
Upper case reserved words, and capitalization rules for identifiers
and types automatically bring to mind the ridiculous bondage and
discipline languages(*) like Pascal and Modula-2 used for so long in
universities to teach students to be glad they're not Swiss. Of course
this brings up bad memories...

Programs in these languages are not written to do anything useful;
there written to be marked. This leads naturally to two of the most
annoying features of most ASN 1 specifications: ridiculously long
identifier names, and deeply nested structures.  It only takes a few
levels of AttributeTypeAndValues before you start running out of
screen real estate.

I've occasionally been tempted to write and ASN runtime that would
select fields within object using a search process, creating
intermediate levels where necessary, using the first match found in
cases of ambiguity.  This can be extended to allow partial matches
through wild cards, in much the same way as in Knuth's literate
programming languages.  It's amazing how much redundancy there is, and
how few characters you need to locate the appropriate field.

Encoding Rules:

1) BER is short for Broken Encoding Rules.  Nobody is challenging
that.  However, they aren't really that complicated-after I've been
working on BER for a few days, I can sight read hex encoded dumps
pretty well.  I once even generated a BER encoded request using a
Telnet client.  It's not something I'm proud of, and I hope never ever
to be able to do it again :-).

2) A lot of people have made rather definitive statements about PER
(the Packed Encoding Rules), despite not actually knowing what they
are.  To most people's surprise, the packed encoding rules are pretty
slick.

Numbers are encoded in the minimum number of bits, and multiple values
can be packed into a single byte.  For example, if a value is
constrained to be between 100 and 107, the encoded form will only use
three bits.

If the size of a field is known, then there's no need to send a
length.  If the size is constrained, then it's encoded in the minimum
number of bits as discussed.  The lengths for SEQUENCE OF types is
given in terms of the number of items in the list, rather than the
length of the encoded form.  Anyone who's written a DER encoder will
appreciate that.

Unlike the broken encoding rules, PER doesn't use tags; the selected
type in a choice is indicated by encoding a number of the appropriate
length before encoding the type itself.  The presence or absence of
optional elements in a SEQUENCE is indicated by a bit field at the
start of the sequence.  This allows high-performance implementations
to create specialized decoders for various combinations of present
options, increasing the number of fields whose location is known.

Encodings in PER can be 50-80% smaller than those in BER, with
corresponding improvements in bus usage.  For many architectures,
especially those with single cycle shift and mask instructions
(e.g. PowerPC), or provide free shifts with every operation (ARM),
this will most always win over cubbier encodings like XDR.

(*) My voice recognition software insists on rendering ASN 1 as SM 1.
Coincidence?

----
Now available - The Freddy Hayek Kayak            | "Pass me another elf
Paddle Your Own Canoe! Be Rowed To Surfdom!       | Sergeant- this one's
>From The Taco Institute for Dyslexic Libertarians | split"
  Moments ago I had everything. Now, there's a cow in my nose - La Salla

---

From: Rob Earhart <[email-masked]>
Date: Wed, 31 Dec 1997 21:02:50 -0500 (EST)

>    acl                = "(" [acl-identrights *(SP acl-identrights)] ")"

  I think it's fairly clear that the author simply forgot to erase the
second line from the RFC; this could happen to anyone, using any
system.

> but I haven't written much machine code lately.  Read, yes. Write -

  BNF is fairly low-level; nevertheless, I'd hardly call it the
machine code of textual protocols.  It *is* fairly easy to read and to
write; you need to think about what's going on, but it's quite clear.

  ABNF's always been fairly easy to turn into a yacc-style grammar;
one of the nice things about rfc2234 is that ABNF is now standard and
regular enough that one could write a tool to take an ABNF
specification along with actions associated with each rule, and turn
them directly into code for a parser.

> protocols can get. This didn't stop IMAP.

  I would venture to say that IMAP's problems are not related to its
grammar as much as they are related to the semantics of the various
messages being passed back and forth.

  (In addition, we (the applications community) have learned quite a
bit about writing protocols, partially through experience with IMAP; I
don't see the same mistakes being made now.)

> easy to implement by hand as old-fashioned http. What if we had IHOP?

  HTTP has its attractions as a protocol infrastructure. I honestly
prefer IMAP/ACAP's scheme style syntax to HTTP's rfc822 style syntax;
I think there're a couple of small technical points in favor of
IMAP/ACAP's scheme style commands, but it's mostly a personal
preference.

  There're quite a few things which ought to exist in a general
application protocol infrastructure.  I tried to identify a few early
last summer in draft-earhart-ap-spec-00.txt; I've since submitted an
updated 01 draft based upon subsequent issues which came up in the
discussion of ACAP over the course of the summer.

  A higher-level IDL which could be translated into IHOP or AP's
framework and automatically turned into generators and parsers
wouldn't be a bad thing at all; I'd suggest translating to ABNF, and
from there to parsers, though.  I've had nothing but problems with
integrating systems that tried to do everything for me with whatever
non-standard system I'm trying to get work done in.

> constrained to be between 100 and 107, the encoded form will only use

  I'll admit to not knowing much about PER, but I'm curious: what
happens when, in the future, that value becomes a number from 100 to
108?

  )Rob

---

From: [email-masked] (Christian Huitema)
Date: Fri, 2 Jan 1998 09:42:12 -0500 (EST)

> happens when, in the future, that value becomes a number from 100 to

The field size increases from 3 to 4 bits.  This is the Achille's heel
of PER.  It has all the good points that Simon mention,which makes it
comparable to XDR in terms of complexity, in a more compact way.
However, it can only be decompiled if the receiver has a full 
knowledge of the syntax used by the sender.

One could argue that the full knowledge is also required in XDR, but
PER pushes this one step further.  One shall not only specify that
this type is coded as an integer -- one must also specify the
range of values, and changing the range changes the encoding.  
Similarly, there is an extension facility to add components to Sequences
in a protocol revision, but the only thing PER can do is detect that the
extension was used.  In cannot do the common trick of just ignoring
the added elements -- something that the IETF protocols such as RFC-822,
or HTTP use and abuse.

---

From: Joerg Ott <[email-masked]>
Date: Fri, 2 Jan 1998 17:01:50 +0100 (MET)

> the added elements -- something that the IETF protocols such as RFC-822,
[this is not because of the original discussion or religion but because
 of accuracy]

This is actually what the extension markers in ASN.1 have been designed
for: to keep e.g. integer or other value ranges extensible and not to 
be required to have full knowledge of all the fields following a base
set of fields in a data structures.  This allows version 1 endpoints of
some standards to talk to version n endpoints without having to
understand all the additions or range extensions of versions 2 ... n.
So it is perfectly possible to skip unknown extensions or figure out that
a value is not in a known range and skip it.  However, this is achieved
at the cost of a more inefficient encoding if extensions are used.  And
all extensions have to be collected in a single place and typically
require a well-defined encoding order -- which is defintely different
with the RFC 822 and HTTP mechanisms -- unless an application level
extension is defined (such as {name/id, value} pairs).

Joerg

---

From: [email-masked] (Christian Huitema)
Date: Fri, 2 Jan 1998 11:14:48 -0500 (EST)

Joerg,
The extension mechanism that you describe does not work well with PER.
In the case of PER, the decoder will encounter a bit saying that
the element was extended.  This bit will be followed by a bit field
with one bit for each optional element in the extended set.  If the
decoder does not know the number of such elements, it will loose
synchronization.  Thus, a PER decoder that encounters an extension
will most likely just give up.

This points to another drawback of PER.  The encoding is extremely
compact because it does away with any form of redondancy. This also
means that wild errors, such as messages decrypted with the wrong
key, are likely to generate semantically valid components.  You can
experience that by feeding random bits into, for example, a PER
decoder for H.225.  About 1/4th of the time, you will obtain a valid
decoding...

---

From: Stephen Kent <[email-masked]>
Date: Fri, 2 Jan 1998 14:59:46 -0500

Chris,

>successful history whenever possible.

I think we have a situation where each camp can cite good and bad examples
of using each syntax in some application context.

>payload then it SHOULD have a textual encoding.

I don't think we're disagreeing here.  My earlier examples were ones in
which most of the data was binary and thus a binary control structure seems
most appropriate to me.  In general I think the IETF should allow a WG to
use which ever encoding and specification appropach seems most appropriate
to the WG.  One could imagine scenarios where this gets out of hand, but
there are suitable review processes in place in the standards tracks to
suggest that this approach will continue to work.

Steve

---

From: Stephen Kent <[email-masked]>
Date: Fri, 2 Jan 1998 15:47:16 -0500

Marshall,

>must have missed it.  sorry, i'll pay more attention next time.

We're both old enough to suffer from vision problems.  Maybe it's getting
harder to read between the lines.

>with respect to the direction it is taking.

Got me there!  I meant v2, not v3.

>assertion that there are a lot of books on the subject.

I was just pointing out that amazon.com is not the definitive reference for
books of this sort, and backing up my claim with a trivial search in
another venue.  I'd suggest that neither search is sufficient to support or
disprove my earlier claim.
requirements.

>ability to deliver cost-effective, interoperable solutions in a timely

I used the phrase "percieved requirements" precisely because I agree that
users sometimes get carried away.  Nancy Regan was right when she
emphasized  "Just Say No."  Maybe she really was referring to
"requirements."

>rarely come out as we hope.
Again, I agree that we should not make life more complex than necessary,
but different set of people have different views of what is necessary.  One
could argue that many web pages are way too complex, using flashy graphics
in a fashion that does not support the basic information function of the
site, and that it makes accessing the pages tedious on a dialup medium.
The counter argument is that flashy graphics are needed to attract visitors
(with high speed access?).  Also, the design/approval by concensus approach
that arises in standards bodies, including the IETF, creates more
opportunities for complexity due to the inclusion of options as a means of
compromise.

>byproducts, once the engineering work has begun guided by architectural

The IETF does not seem to have an architectural framework.  We used to,
when life was simplier, but we've increasingly adopted a free market
approach that is not consistent with having a set of strong architectural
principles that are used to rule out potential standards at an early stage.

>principles.  we should discuss, we should deliberate, and then we should

The IETF already does accommodate multiple camps, in that each WG has been
free to adopt syntax and encodings that are deemed appropriate for the
protocol in question.  One can argue that WGs that make bad choices will
fail in the marketplace due to unnecessary complexity, greater time to
market, etc.  That seems to be the approach that we have adopted for
several years.

>asn.1, but rather rely on our 20-year old 32-bit ascii art syntax.

Sorry, I lost track of the original query.  But, I'm not sure of the
relevance of the answer, anyway.  The nature of the data we move around in
applications is evolving rapidly.  Even if most (almost all?) of our
successful experience in developing and deploying application protocols is
based on ASCII syntax, what does that necessarily say about future
application protocols?

I agree with Chris in so far as he suggested that the syntax ought to be
appropriate for the application data.  Yes, I agree that there are more
issues here, as you noted above.  However, an argument of the form "we've
always used ASCII and we've done well, so let's not change" sounds like a
programmer who's always been satisfied with assembly and thus is resistant
to using a hig level language.  After all, good compilers cost money,
require fancy (not free) debuggers that can deal with the optimization
introduced by a fancy compiler, etc.

Steve

---

From: Stephen Kent <[email-masked]>
Date: Fri, 2 Jan 1998 15:47:16 -0500

Marshall,

Steve

---

From: Bancroft Scott <[email-masked]>
Date: Fri, 2 Jan 1998 17:54:33 -0500 (EST)

>>> Joerg, The extension mechanism that you describe does not work well with
>>> PER. In the case of PER, the decoder will encounter a bit saying that
>>> the element was extended.  This bit will be followed by a bit field with
>>> one bit for each optional element in the extended set. 
>> 
>> No, Christian, this is not how the extension mechanism in PER works. You
>> are confusing how SETs and SEQUENCEs with components marked with OPTIONAL
>      ^^^^^^^^^
> or DEFAULT are encoded versus how extension additions are encoded. 
>
>However, this and the deleted following discussion exactly illustrates 
>the problem with asn.1 -- it is, to use your word, confusing. 

It does not follow that if Christian misunderstands something about PER
that PER is confusing.  Even Christian can make oversights if he has
never used PER.  I have used PER, and every now and then I too have
oversights with it, just as I do with Java.

> My last use of asn.1 was several years ago, implementing a protocol
> library for the NASA Portable Batch System (PBS) using ISODE.  That
> was a very simple use of asn.1, for what is clearly an application
> level protocol, and I grant that once we had the protocol defined it
> was relatively simple for me to make changes.  However, no one else on
> the project could make those changes without going through an ugly
> learning curve, and after I was pulled off the project I believe that
> later versions of PBS threw away the asn.1 version, and changed to a
> simple ascii protocol. 

I'm not surprised.  This reflects the findings of the European
Telecommunications Standards Institute (the European equivalent of ANSI) 
which in a 1993 survey of ASN.1 compilers disqualified the ISODE compiler
in its first round review.  ISODE was one of the first attempts to
implement an ASN.1 compiler, and not surprisingly, (at least the first
version) fell short of the mark in ease of use and in other ways. However,
without the likes of ISODE blazing the trail, later attempts at
implementation would have likely made the same mistakes.  While I have not
looked at the ISODE compiler in a few years, I would imagine that it has
improved since responsibility for it was assumed by the ISODE Consortium.

> Maintaining expertise in something as complex as asn.1 is a
> significant overhead for most software engineers -- unless you want to
> make a career of using it. 

ASN.1 can be complex for complex specifications, and for simple ones too
if the person doing the writing does a poor job.  The same is true for
many other things, such as C or Java.

---

From: Karl Auerbach <[email-masked]>
Date: Fri, 2 Jan 1998 15:54:57 -0800 (PST)

> many other things, such as C or Java. 

Sometimes we do need to move fairly complex data structures around within
a protocol -- RSVP with the Integrated Services payloads is such a case.

In those instances, ASN.1 provides a useful means of expressing the packet
form and, by selection of a particular encoding method, provides a useful
means of consistently encoding things for transmission.

My rule-of-thumbs for ASN.1 is:

  - Don't use ASN.1 macros.  Period.  Don't even think of doing so.

  - When there are mildely nested data structures to be transmitted,
    especially when those structures tend to contain binary information,
    consider using both ASN.1 to express the structures and as a method of
    encoding.

  - When moving very complex objects (such as are typically found within
    programs -- things like interlinked tables with internal pointers or
    references, the best approach is to flatten the data structure using
    object-specfic methods of persistance, and then to transmit the
    flattened object as an opaque chunk without trying to represent it in 
    ASN.1.

  - Avoid using things like "optional" and "default" as those, nice as
    they may be, tend to trigger interoperability problems.

  - Avoid double (explicit & implicit) tags.  One should be able to design
    with one level of tags on a datum.

  - Watch out for the use of any unsigned number or signed number where
    the 32nd bit may be on -- this triggers a very common bug in
    implementations and leads to apparent interoperability but incorrect
    data transfers.

My personal experience with textual protocols is that they look easy, but
that I spend a great deal of effort dealing with minutae, like
end-of-line/end-of-buffer conditions, case-folding, long-line handling (to
avoid buffer overflow attacks), UTF-x/Unicode/ASCII encoding transitions,
tabs versus blanks, etc.

So, I do like ASN.1 -- in concept.  But since it is loaded with bells on
the bells and whistles on the whistles, I find that it is too painful to
look at, much less to use.

So, I'd love to find an ASN.0 and Simple-ER:

  -- something that doesn't have macros,

  -- something that drops many of the "features" that are essentially
     useless (except to their proponent)

  -- something that understands unsigned numbers

  -- something that sanely understands floating point

  -- something that says "let's only represent" numbers on sane computers,
     i.e. leave bignum packages and public keys to use opaque encodings
     within octet strings.

  -- something that doesn't have "default"

  -- something that has encoding methods that can be understood without
     years of indoctrination in the monastery of ASN.1/BER/PER/xxx.[*]

[*] My favorite arcane fact about BER regards what tags are used on the
inner elements of constructed forms.

		--karl--

---

From: [email-masked] (Christian Huitema)
Date: Sat, 3 Jan 1998 12:12:18 -0500 (EST)

Karl,

I agree that a subset of ASN.1 could make a useful binary encoding
format, when binary encoding is necessary.  So could XDR, by the way.
I would make an even stronger requirement than yours in the subsetting.
First, we should limit the scope of the language to simple data structures.
This means:

1) never use the MACRO extension. I think everyone agree on that.

2) never use the VALUE notation. This is in fact the source of most
   bugs in the 1988 version.

3) because you don't want to use the VALUE notation, never use the
   DEFAULT feature.  OPTIONAL and CHOICE can do the job.

4) don't even think of using the object oriented notations of 1994.

Then, I would learn from experience and try to disallow the features
of ASN.1 that did pose interoperability problems in the past. This
means, in particular:

5) Never use the SET construct. SEQUENCE dos the job, is just fine.

6) Always use simple (IMPLICIT) tags, and always use the CONTEXT
   notation (or the 1994 default numbering feature, which is 
   equivalent).

7) Settle for exactly one notation of dates.

8) Do the right thing with strings.  Do we really need more than
   an octet string notation?  Should the character set used by an
   application be checked by an interface stub?

Finally, the last recommendation would be to settle for exactly one
binary representation.  DER comes to mind as the representation of
choice --it is actually mandated for security applications.  If
a comapct encoding is required, just run DER through a compression
algorithm, e.g. gzip. This will compress the tags and extra zeroes,
probably just like PER.  It will also compress the string values,
something that PER does not.

Someone willing to write an ASN.1 profile for IETF standards ?

---

From: [email-masked]
Date: Mon, 5 Jan 1998 16:21:23 -0500

Date: Sat, 3 Jan 1998 12:12:18 -0500 (EST)
   From: [email-masked] (Christian Huitema)

Use an integer number of seconds from some fixed date/time.  It's much
easier to convert into an internal time format, and it's not subject to
Y2K problems, bugs in leap year calculations, leap second issues, time
zone problems, etc.  It's also much more compact that an ASCII
representation of a date, which the ASN.1 date formats likes to use for
some stupid reason.

Kerberos V5 made this mistake by using ASN.1 date type in its protocol
definition, and it's caused us no end of problems that should have
otherwise been avoidable.

						- Ted

---

From: "Philip J. Nesser II" <[email-masked]>
Date: Mon, 5 Jan 1998 17:27:00 -0500 (EST)

> representation of a date, which the ASN.1 date formats likes to use for

And make it larger than 32 bits.  The year 2036 is going to cause a lot of
problems for protocols that use seconds since 1/1/1970.

--->  Phil

---

From: Frank Solensky <[email-masked]>
Date: Mon, 05 Jan 1998 18:05:13 -0500

> problems for protocols that use seconds since 1/1/1970.

Picking nits, I know, but we're notorious that way.  Should that be the
year 2106 (Feb 5 06:28:16, excluding leap seconds) instead?
							-- Frank

---

From: Frank Solensky <[email-masked]>
Date: Mon, 05 Jan 1998 18:25:02 -0500

> year 2106 (Feb 5 06:28:16, excluding leap seconds) instead?

Never mind, I know: signed.   2038 Jan 19 03:14:08 instead..
(*grumble* mondays..)

---

From: Phil Karn <[email-masked]>
Date: Mon, 5 Jan 1998 18:23:21 -0800 (PST)

>Use an integer number of seconds from some fixed date/time.  It's much

I heartily second this. I propose the GPS timescale, which has its
epoch at 00:00:00 UTC Sunday, January 6, 1980. Note that GPS does
*not* follow UTC leap seconds, so it is currently 12 seconds ahead of
UTC. This difference will increase as more leap seconds are inserted
into the UTC time scale.

In my opinion, GPS's lack of leap seconds makes it vastly preferable
to UTC for use as a basic, monotonically increasing internal time
scale in computer protocols and operating systems. Leap seconds
introduce any number of problems; just consider the transient effects
on NTP every time a leap second occurs. Dave Mills may consider them
really fun to watch, but I consider it a basic design flaw of NTP that
such transients occur in the first place.

Any complaints that GPS time is not directly "user friendly" don't
wash with me.  Conversion between GPS time (for protocol/internal
program use) and UTC/local time (for human consumption) is something
that computers do very well.

The GPS itself usually expresses time as some integer number of whole
weeks from the epoch (the week of January 6, 1980 being week #0) plus
seconds within the week (there being 604800 seconds in a week). But
nothing says we can't reduce things further to a simple integer count
of seconds from the same epoch. It simplifies calculations, and uses
bits more efficiently too.  Make the count 64 bits long, and we won't
have to worry about rollover for some time.

And it goes without saying that it's getting cheaper and easier all
the time to get continuous, high accuracy access to GPS
time. Stationary GPS timing receivers such as the TAPR TAC-2 (based on
the Motorola ONCORE VP GPS engine) can give you ~50 ns accuracy for
less than $500.

Phil

---

From: [email-masked] (Richard Prout)
Date: Tue, 06 Jan 1998 08:21:33 +0000

Sorry couldn't resist but I think the problem will occur at 06:28:17.

---

From: "Louis A. Mamakos" <[email-masked]>
Date: Tue, 06 Jan 1998 10:17:44 -0500

> such transients occur in the first place.

This isn't so much a problem with NTP, but with the implementation of the
local clock model.  If ctime(), et. al. presumed a monotonically increasing
timescale and consulted a database of leap seconds, you wouldn't introduce
a discontinuity.  That the time scale that the UNIX kernel uses "includes"
leap seconds is the botch.

Of course, you've now made the conversion between this time scale and 
a particular instance of wall-clock time much more complicated than adding 
a fixed offset.  Grief is conserved.

louie

---

From: Garrett Wollman <[email-masked]>
Date: Tue, 6 Jan 1998 10:47:22 -0500 (EST)

> a fixed offset.  Grief is conserved.

Except, of course, that Arthur Olson has already solved that problem
for us (and, for recent-historical times, this was already the case).

There was some flamage on comp.protocols.time.ntp not too long ago
saying that NTP should have been designed to tick TAI, and leave it up
to the end users to figure out whether they want to observe TAI, TAI
plus a fixed offset, or UTC.  (Of course, the POSIX botch strikes
again.)

-GAWollman

--
Garrett A. Wollman   | O Siem / We are all family / O Siem / We're all the same
[email-masked]  | O Siem / The fires of freedom 
Opinions not those of| Dance in the burning flame
MIT, LCS, CRS, or NSA|                     - Susan Aglukark and Chad Irschick

---

From: Karl Auerbach <[email-masked]>
Date: Tue, 6 Jan 1998 09:49:31 -0800 (PST)

> have to worry about rollover for some time.

I'd like to suggest that there be sub-second resolution.  That would make
the same time representation useful for those who need to synchronize
media sources (like when lip-synching voice and video).

This could add another 16 to 24 bits to the length.

		--karl--

---

From: "sanjay kulkarni" <[email-masked]>
Date: Tue, 06 Jan 1998 11:16:39 -0800

Ted,

This approach though very simple is not without its problems. Remember
time_t
gets into trouble in 2036. (Assuming 32 bits for an integer).

sanjay.

---

From: "Theodore Y. Ts'o" <[email-masked]>
Date: Tue, 6 Jan 1998 17:22:10 -0500

Date: Tue, 06 Jan 1998 11:16:39 -0800
   From: "sanjay kulkarni" <[email-masked]>

If we're still using 32 bit OS's by the year 2036.....

						- Ted

---

From: Phil Karn <[email-masked]>
Date: Wed, 7 Jan 1998 17:04:35 -0800 (PST)

>plus a fixed offset, or UTC.  (Of course, the POSIX botch strikes

Right. TAI = international atomic time. There's a fixed 19-second
difference between TAI and GPS time; this was the difference between
TAI (no leap seconds) and UTC (with leap seconds) on the GPS epoch
date of January 6, 1980. So it really doesn't matter whether you use
TAI or GPS time, as both are monotonically increasing scales without
leap seconds making it trivial to convert between the two.

Actually, if you want submicrosecond accuracy things get slightly more
complicated. Due to clock drift, the GPS-UTC (and GPS-TAI) difference
is not *exactly* an integral number of seconds. But the residual is
always less than a microsecond, and the GPS navigation messages give
you the current residual that will get you within 50 ns or so.

And yes, time fields in protocols should have better than 1 sec
resolution. I sometimes use 64-bit doubles to represent GPS time, but
I estimate this has a current LSB resolution of about 63 ns -- plenty
adequate for file timestamping, but too coarse for some GPS work. A
64-bit integer count of nanoseconds has a period of 585 years, which
isn't *too* short.

Phil