Subject: [Cbor] Survey of floating-point

From: Sean Leonard <[email-masked]>
Date: Sun, 22 Jul 2018 18:23:23 -0700

Just wanted to throw this out there:

What do current CBOR implementations do when comparing two floats (major 
type 7, values 25, 26, 27) for equality, specifically in the case of 
comparing keys for maps?

Are there any implementations that will compare any float with a float 
of a different size (e.g., binary32 and binary64) and return equal?

Are there any implementations that will compare any float with an int 
(major types 0 or 1) and return equal?

Are there any implementations that will compare any float containing 
some NaN, with any other float containing some NaN, and return equal?

I really would like to know what implementations do (or, at least, what 
they are programmed to do), not discuss how it should be.

Regards,

Sean

---

From: Thiago Macieira <[email-masked]>
Date: Sun, 22 Jul 2018 21:19:56 -0700

> type 7, values 25, 26, 27) for equality, specifically in the case of

Using floating point as keys for maps is a bad practice in almost every 
programming language. I think the CBOR specification should be very clear that 
CBOR maps should prefer to use integers and strings for keys, with tagged 
(extended) types as a distant second. Other keys are permitted, of course, but 
discouraged.

> of a different size (e.g., binary32 and binary64) and return equal?

Yes. My implementation in QCborValue decodes any and all floating point to a 
an internal storage of double. So the following map ends up "invalid" due to 
duplicated keys:

	{ 0.0: "one", 0.0f: "two" }

> some NaN, with any other float containing some NaN, and return equal?

Yes, QCborValue compares the bit pattern for the floating point, which means 
two NaN compare equal, instead of comparing different as IEEE dictates. It 
will also perform sorting according to the big-endian bit pattern, not 
according to the number stored, so 1.0 < -1.0.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

---

From: Joe Hildebrand <[email-masked]>
Date: Mon, 23 Jul 2018 09:24:43 -0600

> On Jul 22, 2018, at 10:19 PM, Thiago Macieira <[email-masked]> wrote:
> 

Agree.

> 	{ 0.0: "one", 0.0f: "two” }

Mine will compare the generated bytes for each key, same as for ordering.

> according to the number stored, so 1.0 < -1.0.

Same here.

— 
Joe Hildebrand

---

From: Sean Leonard <[email-masked]>
Date: Mon, 23 Jul 2018 09:06:10 -0700

> (extended) types as a distant second. Other keys are permitted, of course, but

Not taking a moral stance on it at this time. :-) I can think of 
perfectly valid applications (scientific, sensor measurements), but do 
not think they need to be delved into yet. Better to get the data first.

How about the question of: can a float ever compare to an int (positive 
or negative), and return equal, in your implementation?

I read 
<http://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/serialization/qcborvalue.cpp?h=dev>, 
and my guess is "no", since the two CBOR items have to have the same 
major type for further comparisons to proceed.

Sean

---

From: Jim Schaad <[email-masked]>
Date: Mon, 23 Jul 2018 10:09:42 -0700

> -----Original Message-----
> From: CBOR <[email-masked]> On Behalf Of Joe Hildebrand
> Sent: Monday, July 23, 2018 8:25 AM
> To: Thiago Macieira <[email-masked]>
> Cc: [email-masked]; Sean Leonard <[email-masked]>
> Subject: Re: [Cbor] Survey of floating-point
> 
> > Using floating point as keys for maps is a bad practice in almost
> > every programming language. I think the CBOR specification should be
> > very clear that CBOR maps should prefer to use integers and strings
> > for keys, with tagged
> > (extended) types as a distant second. Other keys are permitted, of
> > course, but discouraged.
> 
> Agree.
> 
> >> Are there any implementations that will compare any float with a
> >> float of a different size (e.g., binary32 and binary64) and return equal?
> >
> > Yes. My implementation in QCborValue decodes any and all floating
> > point to a an internal storage of double. So the following map ends up
> > "invalid" due to duplicated keys:
> >

The version that I use will compare the CBOR Type before comparing values - so I get the same answer as Joe.

Jim

> > Yes, QCborValue compares the bit pattern for the floating point, which
> > means two NaN compare equal, instead of comparing different as IEEE
> > dictates. It will also perform sorting according to the big-endian bit
> > pattern, not according to the number stored, so 1.0 < -1.0.
> 
> Same here.
> 
> —
> Joe Hildebrand
> 
> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor

---

From: Thiago Macieira <[email-masked]>
Date: Mon, 23 Jul 2018 10:10:44 -0700

On Monday, 23 July 2018 09:06:10 PDT Sean Leonard wrote:
> <http://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/serialization/qcborva
> lue.cpp?h=dev>, and my guess is "no", since the two CBOR items have to have
> the same major type for further comparisons to proceed.

Correct, that's how it's implemented.

However, the front-end class does allow toInteger() on a double and toDouble() 
on an integer.

    qint64 toInteger(qint64 defaultValue = 0) const
    { return isInteger() ? value_helper() : isDouble() ? qint64(fp_helper()) : 
defaultValue; }
    double toDouble(double defaultValue = 0) const
    { return isDouble() ? fp_helper() : isInteger() ? double(value_helper()) : 
defaultValue; }

So I can still change this (either or both).

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 23 Jul 2018 20:14:37 +0200

> On Jul 23, 2018, at 03:23, Sean Leonard <[email-masked]> wrote:
> 
> What do current CBOR implementations do when comparing two floats (major type 7, values 25, 26, 27) for equality, specifically in the case of comparing keys for maps?

CBOR-Ruby does not compare any data items.  (CBOR-canonical for Ruby does order them, according to the recommendations in Section 3.9 of RFC 7049, i.e. in the canonically encoded form.)

When Ruby-cbor builds maps, it uses the infrastructure of Ruby for that (class Hash, which is based on “.hash” hashing and “.eql?” comparison).  Map building occurs at the data model level, so representation invariants are not visible any more.

The peculiarities of Ruby floating point support mean that 0 and 0.0 are valid as two different map keys, while 0.0 and -0.0 are not (same “.hash”).
One could consider that to be a bug (in Ruby, that is).

> 
> Are there any implementations that will compare any float with a float of a different size (e.g., binary32 and binary64) and return equal?

All reasonable ones will (representation variants do not enter at the data model level).

> Are there any implementations that will compare any float with an int (major types 0 or 1) and return equal?

Implementations for languages with a limited numeric type system may; this depends on whether a map type is available that allows for keys at the data model level.

> Are there any implementations that will compare any float containing some NaN, with any other float containing some NaN, and return equal?

For the purpose of map keys, it is not normal comparison that counts, so you get the following anomaly:

>> {0 => "0", 0.0 => "0.0", -0.0 => "-0.0", Float::NAN => "NaN", -Float::NAN => "NaN", Float::NAN => "NaN"}.to_cbor.hexs
=> “a4 00 61 30 f9 00 00 64 2d 30 2e 30 f9 fe 00 63 4e 61 4e f9 fe 00 63 4e 61 4e”
    {  0: “0”,  0.0:     “-0.0”,        NaN:     “NaN”,      NaN:     “NaN”        }

The reason is that 0.0 and -0.0 are considered equal for hashing purposes, as are NaN and NaN, but not NaN and -NaN.  Ouch.  I didn’t consider this language wart something that CBOR-Ruby should try to fix.  Both NaN variants are normalized to 0xf9fe00 by CBOR-Ruby, while they canonically should be 0xf97e00 according to Section 3.9.  It seems that 0xf9fe00 is a bug (line 174 of ext/cbor/packer.h).

Grüße, Carsten