Subject: Re: [apps-discuss] Draft on representing values of sensors in JSON (draft-jennings-senml)

From: Cullen Jennings <[email-masked]>
Date: Sun, 24 Oct 2010 16:49:01 -0600

Thanks for the great comments ... more inline. Please have a look at the one thing below on time representation - really need your input on that. 

Diff of new draft is at:
http://svn.resiprocate.org/rep/ietf-drafts/fluffy/draft-jennings-senml.diff.html

On Sep 21, 2010, at 5:34 PM, Thomson, Martin wrote:

>> I'd love to get some feedback on
>> 
>> http://www.ietf.org/id/draft-jennings-senml-03.txt
> 
> Hi Cullen,
> 
> The Unified Code for Units of Measure might be of interest to you.  This is based on ISO 2599 and ANSA X3.50.  It might save you the need for a new registry.
> 
>  http://unitsofmeasure.org/
> 
> I have my own reservations about the grammar (too many special cases), but it seems a shame to waste all the good effort.

Interesting I did not know about that one but I have looked a bunch of these. They mostly seem around covering the way humans not machine would want to use units and I  wonder how one would get interoperability if using them. I also had  problems with the process to add new things to some of these existing units lists. I finally gave up and went with the create a new registry approach. That allows us to add things that are not really units in the traditional sense but turn out to be really useful in telling the system reading the JSON object know what to do. 

> 
> 
> You also have a TODO on accuracy/confidence.  Additional optional attributes seem appropriate: in metrology, it's commonplace to describe a confidence interval.

I somehow suspected you might have a well informed opinion on this topic. When I wrote TODO, I was hoping it was something someone else would Do. Thank you for the text. It's been inserted. 

> 
> Measurement quality attributes can be added:
> 
>  unc: the uncertainty in the measurement, that uses the same units as the base; if absent, the value is unknown (i.e., don't assume that this is zero)
>  c: the confidence of the measurement, as a percentage; if absent, you can pick a default value
> 
> An alternative is to report RMS error, which is a common measure of quality where there are multiple samples taken, but only a mean value reported:
> 
>  rms: the RMS error in the measurements; if absent: unknown
>  num: the number of samples taken; if absent: assume either a very large number such that RMS =~ standard deviation, or assume one (pick one)

I asked around on what would be more useful to receive the unc/c or the rms/num and seemed most people preferred the unc approach for this stuff.

> 
> In a lot of sensors, uncertainty and confidence are known, but discrete observations do not necessarily carry specific values for these.  Instead this is a known characteristic of the device.  Thus, it could make sense to carry a base value (a base uncertainty).  Within the format you describe, you will need to associate a base value with each sensor.
> 
>  { q: { [ n: "foo", unc: 0.43, c: 90 ] } }
> 

will think about this one ...

> 
> I don't see any value in providing a valid time interval.  This is going to either be based on predictions that use measured values, or it's going to be virtually impossible to predict.  Any case where a measurand is time-varying is going to be known by the recipient for that recipient to do anything reasonable with the data.

The problem is more around detecting a dead sensor. Take a power meter of some sort that is not polled but pushes data - some report every second, some every minute, some every day, some every week, some whenever it changes by some threshold not at least once every 10 minutes. The sensor knows this update schedule but the system receive the data does not. The receiver would like to be able to indicate when the data is so stale it should no longer be used. I've been having a hard time figuring out how to describe this in the draft but I think some idea of maximum update interval is needed. I agree the sensor trying to do something like predict how long the temperature would stay the same for or how long before a location sensor moves sounds pretty unlikely.  

> 
> 
> Is the intent for the sensors to be labelled with a globally unique identifier?

Yes 

>  If so, why?  I would have assumed that the identified source of data would provide some amount of disambiguation.

Clearly on could make it work either way. Its pretty easy to make a unique name for most sensors - if nothing else one can use the MAC from where the sensor meets the IP network or a UUID. Adding it later is much harder and one just gets into all the confusion about if something is a new measurement or the same measurement reported by multiple paths.  Trying to do broad sensor fusion projects really convinced me of the value of having a unique identifier. 

> 
> 
> You have a lot of questions.  Is it really that much more of a burden for a constrained device to produce an RFC 3339 date+time?  Certainly, the size isn't going to be a significant burden.  Maybe you could have 'bt' include this value, and 't' can be left as a delta.  

So theres no way to do timezone on a small device but yes 3339 format is probably not too big a deal. Leap seconds might turn out to be a bit of a problem.  But is it any easier or better? It seems like this work out better as long as you assume the end system converts to something human readable. Arguments either way? The 3339 format looked good for human readability but just looked like for more of a hassle for any system producing the or consuming it. 

> Certainly, I'd like to see a resolution of better than seconds.  Nanosecond resolution would be handy for something like location ;)

So I moved this to a floating point double. But that only gives microseconds precision over the next 100 years or so. How badly do we need nanoseconds? Aren't nanoseconds times often the result of a sensor reading and relative? So we could easily say that a device had measured the reflection delay up a cat 5 cable and it was 72.3 nano seconds but saying that measurement was made at exactly 3 ns after midnight today is something we could not do. Thoughts on what is needed here? If we really do need this, then what I would propose is we have two fields one is the t (bt) which just gives integer number of seconds and a second sub second field (perhaps called "st") that has a double float representation of time that is added to the integer time. This would give us pico second time precision. 

> 
> 
> (Nit: Your examples have trailing commas.  The JSON syntax doesn't allow this.)''

oops - all example pass JSONLint now

> 
> --Martin
> 
> p.s., I like this:
> 
>   ... The number of significant digits in any
>   measurement is not relevant, so a reading of 1.1 has exactly the same
>   semantic meaning as 1.10.  ...
>

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 25 Oct 2010 07:56:12 +0800

Hi Cullen,

Good to see that you're making progress :)

A few follow-up comments below.

On 2010-10-25 at 09:49:01, Cullen Jennings wrote:
> ...
> I  wonder how one would get interoperability if using them. 

That's the challenge.  The grammar doesn't seem conducive to a simple implementation.  That's never good.  The process for new registrations isn't well enough understood for me either.

On the other hand, it might be worthwhile doing the unit thing properly.  A formal grammar is the next step (and not a bad idea in my opinion), then there is the question of standard prefixes (k, M, G, T, P, ... m, μ, p, f, ...), and kg fouls that up a little.

> > I don't see any value in providing a valid time interval.
> 
> The sensor knows this update schedule but the system receive the data does not. 

The question is how much a receiver can be assumed to know about a sensor.  I imagine that in many circumstances, the unique id will be sufficient to convey a wealth of information that you wouldn't even bother finding codepoints for: manufacturer, uncertainty characteristics, reporting interval, and so forth.  You have to allow for a range of configurations.

I'm always leery of attempts to overload semantics.  The direct approach that you use is good: next report in <ut> seconds.

> > I'd like to see a resolution of better than seconds.
>
> So I moved this to a floating point double. But that only gives
> microseconds precision over the next 100 years or so. How badly do we
> need nanoseconds?

It's a bit crude, but is should be possible to make a pseudo-sensor that reports a time delta where better precision is required.

> Aren't nanoseconds times often the result of a sensor
> reading and relative? 

That is more likely.  For measurements of that size, a double has plenty of free bits.

--Martin

---

From: John C Klensin <[email-masked]>
Date: Mon, 25 Oct 2010 13:54:59 -0400

--On Sunday, October 24, 2010 16:49 -0600 Cullen Jennings
<[email-masked]> wrote:

>...
>> You also have a TODO on accuracy/confidence.  Additional
>> optional attributes seem appropriate: in metrology, it's
>> commonplace to describe a confidence interval.
> 
> I somehow suspected you might have a well informed opinion on
> this topic. When I wrote TODO, I was hoping it was something
> someone else would Do. Thank you for the text. It's been
> inserted. 
>...

>>  unc: the uncertainty in the measurement, that uses the same
>>  units as the base; if absent, the value is unknown (i.e.,
>>  don't assume that this is zero) c: the confidence of the
>>  measurement, as a percentage; if absent, you can pick a
>>  default value

Cullen,

Confidence expressed as a single probability and uncertainty
expressed as a single value inherently assume that the
distribution of the measure is central, symmetric and, for most
purposes, unimodal.  While it does not assume normality, one
might end up making that assumption to interpret the numbers.
The difficulty is that, while most measurements of physical
phenomena obey that constraint, many other phenomena (especially
many biological and socio-economic ones) are characterized by
asymmetrical distributions with one very long tail or even
multimodal distributions... and the biological ones, at least,
are often measured using sensor-based technologies.

Independently, if you are going to express uncertainty as a
scalar, you get involved with the question of whether you are
providing absolute bound/limits or values at some confidence/
probability function.  The latter would imply that "c" and "unc"
would need to be used together or not at all.  I think the text
above (and now in the draft) assumed that pairing, but it isn't
clear in context (and doesn't eliminate the
central-symmetric-unimodal assumption).

If you are going to describe uncertainty or confidence in
situations where either the sensor or the underlying phenomenon
exhibit asymmetric behavior, you should make provision for
either a description of the distribution, multiple (identified)
values, or both (but see the last comment below).

>> An alternative is to report RMS error, which is a common
>> measure of quality where there are multiple samples taken,
>> but only a mean value reported:

A different set of distributional assumptions, noting that just
about anything based on mean values is of dubious value with
asymmetric distributions.  While you excluded this on the basis
of the preferences of the people you polled, it is the standard
measure for some situations and fields.

In addition, if you are going to deal with "integrated sums" and
their definition, I suspect that you should be prepared to deal
with moving averages and autocorrelation functions: even if few
sensors produce those today, they can be extremely valuable
statistically for measurements of some types of phenomena,
especially when deviations from some norm are more significant
than the norm itself or when measurements are to be used
predictively.

>> In a lot of sensors, uncertainty and confidence are known,
>> but discrete observations do not necessarily carry specific
>> values for these.  Instead this is a known characteristic of
>> the device.  Thus, it could make sense to carry a base value
>> (a base uncertainty).  Within the format you describe, you
>> will need to associate a base value with each sensor.
>> 

I don't think the fairly subtle statistical considerations I've
pointed to above ought to need treatment in the present
document.  Indeed, although I could provide some references to
systems that tried, to do so would, IMO, take you down a rat
hole.  But I think the document should be very clear about
expansion paths so that adding additional/alternate
distributional information or the like does not end up requiring
an entirely new Media Type.  Or, if you think a new Media Type
would be the right solution for such expansion, I think the
document should explain that.

best,
    john

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 26 Oct 2010 07:26:43 +0800

Hi John, Cullen,

> might end up making that assumption to interpret the numbers.

A receiver makes such assumptions either because they know they can (out of band information), because they are ignorant of the possibility, or because they know the consequences and are prepared to accept them.

This format could convey a wealth of information on the observed or theoretical probability distribution of the measurand.  That's probably overkill in these sorts of use cases.  The unc and c parameters are there when cheap is good enough.

In cases where it's important, a sensor can report discrete observations.

> The latter would imply that "c" and "unc" would need to be used together or not at all.  

Yes, there is a pairing.  Usually it goes like this:  c has a default value (0.95); unc doesn't, and is assumed to be unknown in its absence.

[I just noticed that the updated draft says that confidence should be assumed to be "approximately 1".  Better to say "exactly 0.95".  1 is impossible; 0.95 is plenty high, and plausible.]

> than the norm itself or when measurements are to be used

Moving averages and the like were certainly considered.  A sensor that runs a Kalman or particle filter is not outside the realms of possibility.
 
> expansion paths ...

Sounds reasonable to me.  Rat holes swallow a lot of good work.

--Martin