Subject: [Cbor] New pull request proposal for shortest float encoding

From: Laurence Lundblade <[email-masked]>
Date: Sat, 25 Aug 2018 13:38:31 -0700

https://github.com/cbor-wg/CBORbis/pull/32

This is a proposal to reduce CBOR message size without losing floating point precision by encoding the larger floating point values as smaller ones when it can be done without dropping any significant bits.

I personally don't think this should be part of canonicalization, or part of preferred encoding. It's just a very useful way of encoding floating points that seems worth including in the RFC.

LL

---

From: Sean Solberg <[email-masked]>
Date: Sun, 26 Aug 2018 22:25:32 -0500

Question on this:  Even though the actual value transmitted has not lost
any precision for that particular value, how does the receiver then know
that the intended data type actually does have higher precision than what
was transmitted?  Are there some flags somewhere that notify the receiver
that it's actually a double instead of a single or its actually a single
instead of a half?

..Sean

On Sat, Aug 25, 2018 at 3:38 PM, Laurence Lundblade <[email-masked]>
wrote:

> This is a proposal to reduce CBOR message size without losing floating
> point precision by encoding the larger floating point values as smaller
> ones when it can be done without dropping any significant bits.
> I personally don't think this should be part of canonicalization, or part
> of preferred encoding. It's just a very useful way of encoding floating
> points that seems worth including in the RFC.
>
> LL
>
>
> _______________________________________________
> CBOR mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cbor
>
>

---

From: Sean Solberg <[email-masked]>
Date: Sun, 26 Aug 2018 22:34:56 -0500

Follow up question:  the proposal states:   Generally the decoder should
promote half- and single-precision values to double-precision when they are
received as it is implied that all half- and single-precision values
transmitted are actually double precision. Why is it implied that a
transmitted single precision value is actually a double precision value?
 That makes no sense.  A single precision value is implied to be a single
precision value in my mind.   If there was a "please treat as a double"
flag sent along with a single precision value, then the transmitted data
explicitly defines that the transmitted single precision value should be
treated as a double precision value by the receiver.  without the flag, the
single precision value transmitted should be treated by the receiver as a
single precision value.  Explicit behavior is great.  Implicitly expecting
a receiver to treat all floats (half, single & double) as doubles when
received is implicitly a wrong choice.

In general, any time things are done "implicitly", we end up with
compatibility problems because people make different implicit
choices/assumptions.  I believe keeping things explicit is a much better
way to go.
..Sean

On Sun, Aug 26, 2018 at 10:25 PM, Sean Solberg <[email-masked]> wrote:

> On Sat, Aug 25, 2018 at 3:38 PM, Laurence Lundblade <[email-masked]
> > wrote:
>

---

From: Thiago Macieira <[email-masked]>
Date: Sun, 26 Aug 2018 22:32:20 -0700

On Sunday, 26 August 2018 20:25:32 PDT Sean Solberg wrote:
> Question on this:  Even though the actual value transmitted has not lost any
> precision for that particular value, how does the receiver then know that
> the intended data type actually does have higher precision than what was
> transmitted?  

That's not part of CBOR's purpose. That's why integrals are also sent in the 
smallest form depending on the value, regardless of the original data type.

> Are there some flags somewhere that notify the receiver that
> it's actually a double instead of a single or its actually a single instead
> of a half?

No.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

---

From: Thiago Macieira <[email-masked]>
Date: Sun, 26 Aug 2018 22:44:44 -0700

> > it's actually a double instead of a single or its actually a single

That said, we could have a set of tags that could be applied to integrals as 
well as floating point to denote the original type. For example:

	Original8Bit
	Original16Bit
	Original32Bit
	Original64Bit
	OriginalUnsigned

The 16, 32 and 64-bit ones can be used with both floating point and integrals; 
the integral ones can be combined with the Unsigned one to indicate the type 
was originally unsigned. It may be even possible to combine them with the 
Decimal tags to indicate decimal32 an decimal64.

Possibly add Original80Bit, Original128Bit too, allowing for long double, 
__float128, decimal128.

Or expand out the combinations, thus gaining us Float64x different from 
Float64.

---

From: Laurence Lundblade <[email-masked]>
Date: Mon, 27 Aug 2018 13:58:51 -0700

Sean, really glad you brought all this up and wrote it down. I was thinking some of it myself and it needs to be addressed.

Seems like we have a few options for indicating precision for shortest float encoding
- Each CBOR protocol specification explicitly states that shortest float encoding is used for particular values. There is nothing explicit in the protocol messages
- We use a tag like Thiago suggested
- Both of the above are allowed

Since the tag takes up space, maybe the tag could be applied to maps and arrays to indicate all float data items in them are of a particular type. This would complicate the decoder as it has to keep track of some context at each nesting level.

I don’t think the preference for shortest form that we apply to integers should apply to floats. Floats are fundamentally different from integers.  Floats are only approximations of the actual value. Precision tells you how good the approximation is which matters in some use cases.  If you encode 3 as 0x00000003 instead of 0x03 you don’t really get any more information, but there is a difference between 3.000 and 3.000000. That is why I propose that float and double are not subject to any shortest-form rule in the preferred serialization pull request <https://github.com/cbor-wg/CBORbis/pull/28>.

I thought about leaving shortest float encoding entirely out of the RFC for most simplicity. If we leave it out, any CBOR protocol could still define it and use it. 

However, shortest float encoding does seems truly useful in reducing message sizes a lot for a lot of use cases. It also isn’t easy to figure out (at least it took me a while…), so I’m inclined to put it in.  Note that I believe the only purpose of shortest float encoding is to reduce message size. I see no value in it for canonicalization or for interop.  In fact it is extra complexity so it will hurt interop.

LL

> On Aug 26, 2018, at 8:34 PM, Sean Solberg <[email-masked]> wrote:
> 
> Follow up question:  the proposal states:    Generally the decoder should promote half- and single-precision values to double-precision when they are received as it is implied that all half- and single-precision values transmitted are actually double precision. 
> 
> Why is it implied that a transmitted single precision value is actually a double precision value?   That makes no sense.  A single precision value is implied to be a single precision value in my mind.   If there was a "please treat as a double" flag sent along with a single precision value, then the transmitted data explicitly defines that the transmitted single precision value should be treated as a double precision value by the receiver.  without the flag, the single precision value transmitted should be treated by the receiver as a single precision value.  Explicit behavior is great.  Implicitly expecting a receiver to treat all floats (half, single & double) as doubles when received is implicitly a wrong choice.    
> 
> In general, any time things are done "implicitly", we end up with compatibility problems because people make different implicit choices/assumptions.  I believe keeping things explicit is a much better way to go.  
> ..Sean
> 
> On Sun, Aug 26, 2018 at 10:25 PM, Sean Solberg <[email-masked] <mailto:[email-masked]>> wrote:
> Question on this:  Even though the actual value transmitted has not lost any precision for that particular value, how does the receiver then know that the intended data type actually does have higher precision than what was transmitted?  Are there some flags somewhere that notify the receiver that it's actually a double instead of a single or its actually a single instead of a half?
> 
> ..Sean
> 
> On Sat, Aug 25, 2018 at 3:38 PM, Laurence Lundblade <[email-masked] <mailto:[email-masked]>> wrote:
> https://github.com/cbor-wg/CBORbis/pull/32 <https://github.com/cbor-wg/CBORbis/pull/32>
> 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/cbor <https://www.ietf.org/mailman/listinfo/cbor>
> 
> 
>