Subject: RE: I-D ACTION:draft-jaffer-metric-interchange-format-02.txt

From: "Manoj Dhooria" <[email-masked]>
Date: Wed, 24 Jan 2001 17:33:50 +0530

Good overview, Jaffer. Am copying to general list too to draw attention to a
related issue of which I don't know of an adequate internet standard & for
which I had to invent my own representation.

I am into distributed engineering software development, & keeping precision
of floating point values is extremely important for me - even when they will
be transmitted over the net & across CPU architectures. Using printf()
format is guaranteed to lose precision, even though I can limit the loss by
giving lots of significant digits. Also, using a large number of significant
digits increases number of octets to be included in the message.

What I essentially do is transform the source CPU's native representation to
IEEE floating point representation; then print it out as a sequence of hex
bytes; then run an optimizer on it to drop some of the "unnecessary" octets
(guaranteeing that the original hex sequence can be reconstructed at the
receiver end; no dropping of bytes for numbers where I can not guarantee
this).

If there are others on the mailing list facing this issue, I would like to
interact. May be we can have a standard way. I could publish my technique as
an internet draft, if required. If there is already a public spec for it,
may be someone would be kind enough to share. Thank you.

Manoj Dhooria
Geometric Software [www.geometricsoftware.com]
Bombay

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On
Behalf Of [email-masked]
Sent: Tuesday, January 23, 2001 6:04 PM
To: IETF-Announce:
Subject: I-D ACTION:draft-jaffer-metric-interchange-format-02.txt

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

	Title		: Representation of numerical values and SI units in
                          character strings for information interchanges
	Author(s)	: A. Jaffer
	Filename	: draft-jaffer-metric-interchange-format-02.txt
	Pages		:
	Date		: 22-Jan-01

This document describes a character string encoding for numerical
values and units which:
* is unambiguous in all locales;
* uses only

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-jaffer-metric-interchange-format-0
2.txt

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-jaffer-metric-interchange-format-02.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-jaffer-metric-interchange-format-02.txt".

NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

---

From: John C Klensin <[email-masked]>
Date: Wed, 24 Jan 2001 08:53:14 -0500

--On Wednesday, 24 January, 2001 17:33 +0530 Manoj Dhooria
<[email-masked]> wrote:

> Good overview, Jaffer. Am copying to general list too to draw
> attention to a related issue of which I don't know of an
> adequate internet standard & for which I had to invent my own
> representation.
>
> I am into distributed engineering software development, &
> keeping precision of floating point values is extremely
> important for me - even when they will be transmitted over the
> net & across CPU architectures. Using printf() format is
>...
> What I essentially do is transform the source CPU's native
> representation to IEEE floating point representation; then
> print it out as a sequence of hex bytes; then run an optimizer
> on it to drop some of the "unnecessary" octets (guaranteeing
> that the original hex sequence can be reconstructed at the
> receiver end; no dropping of bytes for numbers where I can not
> guarantee this).
>... 

Hi.

I am long out of the field (and on a different continent from my
own library), but, if it is of interest to either of you, there
was an extensive literature on optimal representation of
precision of numbers in the statistical computing and statistical
and scientific database literature in the decade or so starting
in the mid-1970s.  That period overlapped the completion and
introduction of IEEE floating point, and the literature deals
with machines with both fixed precision and the IEEE floating
point gradual degradation. 

One of the interesting problems is that one really wants to
understand data (measurement) precision and precision degradation
under computation (and different algorithms).  Basing the display
precision on the apparent (machine-representation) precision of
the floating point values themselves is ultmately as misleading
as use of a constant display precision independent of the numeric
properties of the machine representations.

    john

p.s. Contemporary versions of Fortran and PL/I also accept ISO
6093 numbers.  The latter is probably no longer of importance,
but, if you are making reference to the numerical and scientific
programming communities, Fortran still is.

---

From: Aubrey Jaffer <[email-masked]>
Date: Wed, 24 Jan 2001 13:42:34 -0500 (EST)

| From: "Manoj Dhooria" <[email-masked]>
 | Date: Wed, 24 Jan 2001 17:33:50 +0530
 |
 | Good overview, Jaffer.  Am copying to general list too to draw
 | attention to a related issue of which I don't know of an adequate
 | internet standard & for which I had to invent my own representation.
 |
 | I am into distributed engineering software development, & keeping
 | precision of floating point values is extremely important for me -
 | even when they will be transmitted over the net & across CPU
 | architectures.  Using printf() format is guaranteed to lose
 | precision, even though I can limit the loss by giving lots of
 | significant digits.  Also, using a large number of significant digits
 | increases number of octets to be included in the message.
 |
 | What I essentially do is transform the source CPU's native
 | representation to IEEE floating point representation; then print it
 | out as a sequence of hex bytes; then run an optimizer on it to drop
 | some of the "unnecessary" octets (guaranteeing that the original hex
 | sequence can be reconstructed at the receiver end; no dropping of
 | bytes for numbers where I can not guarantee this).
 |
 | If there are others on the mailing list facing this issue, I would
 | like to interact.  May be we can have a standard way.  I could
 | publish my technique as an internet draft, if required.  If there is
 | already a public spec for it, may be someone would be kind enough to
 | share.  Thank you.

I think this problem has been solved for IEEE 754-1985 numbers:

 [R4RS] William Clinger and Jonathan Rees, editors.  The revised^4
 report on the algorithmic language Scheme.  In "ACM Lisp Pointers"
 4(3), pages 1-55, 1991.

which is also

 [IEEEScheme] "IEEE Standard 1178-1990.  IEEE Standard for the Scheme
 Programming Language." IEEE, New York, 1991.

(http://swissnet.ai.mit.edu/~jaffer/r5rs_8.html#SEC56)
specifies that:

 The procedure `number->string' takes a number ... and returns as a
 string an external representation of the given number ... such that
 `string->number' of the string returned by `number->string' will be
 equal to the original number.

 ... the result ... is expressed using the minimum number of digits
 (exclusive of exponent and trailing zeroes) needed to make the above
 ... true [howtoprint], [howtoread]...

Not only is fidelity guaranteed, but using the minimal decimal
representation.  I thought there was a C library implementation of these
algorithms; but I can't find it today.

 [howtoprint] Robert G. Burger and R. Kent Dybvig.  Printing
 floating-point numbers quickly and accurately.  In "Proceedings of the
 ACM SIGPLAN '96 Conference on Programming Language Design and
 Implementation", pages 108-116.
 http://citeseer.nj.nec.com/cs?profile=976907%2C28233%2C1%2C0.25%2CDownload&rd=http%3A//www.cs.indiana.edu/%7Eburger/FP-Printing-PLDI96.ps.gz

   ABSTRACT

   This paper presents a fast and accurate algorithm for printing
   floating-point numbers in both free- and fixed-format modes.  In
   free-format mode, the algorithm generates the shortest, correctly
   rounded output string that converts to the same number when read back
   in, accommodating whatever rounding mode the reader uses.  In
   fixed-format mode, the algorithm generates a correctly rounded output
   string using special # marks to denote insignificant trailing digits.
   For both modes, the algorithm employs a fast estimator to scale
   floating-point numbers efficiently.

 [howtoread] William Clinger.  How to read floating point numbers
 accurately.  In "Proceedings of the ACM SIGPLAN '90 Conference on
 Programming Language Design and Implementation", pages 92-101.
 Proceedings published as "SIGPLAN Notices" 25(6), June 1990.
 http://citeseer.nj.nec.com/cs?profile=0%2C224562%2C1%2C0.25%2CDownload&rd=ftp%3A//ftp.ccs.neu.edu/pub/people/will/howtoread.ps

   ABSTRACT 

   Consider the problem of converting decimal scientific notation for a
   number into the best binary floating point approximation to that
   number, for some fixed precision.  This problem cannot be solved
   using arithmetic of any fixed precision.  Hence the IEEE Standard for
   Binary Floating-Point Arithmetic does not require the result of such
   a conversion to be the best approximation.

   This paper presents an efficient algorithm that always finds the best
   approximation.  The algorithm uses a few extra bits of precision to
   compute an IEEE-conforming approximation while testing an
   intermediate n result to determine whether the approximation could be
   other than the best.  If the approximation might not be the best,
   then the best approximation is determined by a few simple operations
   on multiple-precision integers, where the precision is determined by
   the input.  When using 64 bits of precision to compute IEEE double
   precision results, the algorithm avoids higher-precision arithmetic
   over 99% of the time.

   The input problem considered by this paper is the inverse of an
   output problem considered by Steele and White: Given a binary
   floating point number, print a correctly rounded decimal
   representation of it using the smallest number of digits that will
   allow the number to be read without loss of accuracy.  The Steele and
   White algorithm assumes that the input problem is solved; an
   imperfect solution to the input problem, as allowed by the IEEE
   standard and ubiquitous in current practice, defeats the purpose of
   their algorithm.

 |
 | Manoj Dhooria
 | Geometric Software [www.geometricsoftware.com]
 | Bombay
 |
 | -----Original Message-----
 | From: [email-masked] [mailto:[email-masked]]On
 | Behalf Of [email-masked]
 | Sent: Tuesday, January 23, 2001 6:04 PM
 | To: IETF-Announce:
 | Subject: I-D ACTION:draft-jaffer-metric-interchange-format-02.txt
 |
 |
 | A New Internet-Draft is available from the on-line Internet-Drafts
 | directories.
 |
 |
 |	Title		: Representation of numerical values and SI units in
 |                           character strings for information interchanges
 |	Author(s)	: A. Jaffer
 |	Filename	: draft-jaffer-metric-interchange-format-02.txt
 |	Pages		:
 |	Date		: 22-Jan-01
 |
 | This document describes a character string encoding for numerical
 | values and units which:
 | * is unambiguous in all locales;
 | * uses only
 |
 | A URL for this Internet-Draft is:
 | http://www.ietf.org/internet-drafts/draft-jaffer-metric-interchange-format-0
 | 2.txt
 |
 | Internet-Drafts are also available by anonymous FTP.  Login with the username
 | "anonymous" and a password of your e-mail address.  After logging in,
 | type "cd internet-drafts" and then
 |	"get draft-jaffer-metric-interchange-format-02.txt".
 |
 | A list of Internet-Drafts directories can be found in
 | http://www.ietf.org/shadow.html
 | or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
 |
 |
 | Internet-Drafts can also be obtained by e-mail.
 |
 | Send a message to:
 |	[email-masked].
 | In the body type:
 |	"FILE /internet-drafts/draft-jaffer-metric-interchange-format-02.txt".
 |
 | NOTE:	The mail server at ietf.org can return the document in
 |	MIME-encoded form by using the "mpack" utility.  To use this
 |	feature, insert the command "ENCODING mime" before the "FILE"
 |	command.  To decode the response(s), you will need "munpack" or
 |	a MIME-compliant mail reader.  Different MIME-compliant mail readers
 |	exhibit different behavior, especially when dealing with
 |	"multipart" MIME messages (i.e. documents which have been split
 |	up into multiple messages), so check your local documentation on
 |	how to manipulate these messages.
 |
 |
 | Below is the data which will enable a MIME compliant mail reader
 | implementation to automatically retrieve the ASCII version of the
 | Internet-Draft.
 |