Subject: [Geopriv] Location measurement error

From: "Rosen, Brian" <[email-masked]>
Date: Mon, 14 Jul 2008 09:56:06 -0400

I'd like to start a discussion on how we deal with, in the big picture,
the notion of measurement error in general and "confidence" and
"uncertainty" in specific.

I am aware that there are a set of views that go roughly like: "there
really is no such thing as confidence and uncertainty, some vendors are
pushing this idea because they can't do what 'real' GPS devices do and
give you a 3 dimensional error in meters".

Nevertheless, in one of the most commonly encountered location
determination mechanisms (A-GPS and similar systems), confidence and
uncertainty are reported, and are the only error indicator available.  

In other systems, notably commercial GPS systems, you get a different
kind of error specification.  Sometimes it's simply error in meters,
often with a different value for Z than for x and y.  Other times you
get a more complex representation of error.

It does seem to me that when we convey measured location, that it would
be appropriate to convey the error of the measurement, and that error
indication should have the ability to represent confidence and
uncertainty.  I think we need to standardize the representation of
error.  It does occur to me to ask if such standardization would better
be done in OGC, and Carl may wish to comment.  It's also possible for
the geopriv crowd to come up with something and then OGC to incorporate
it in a future version of GML.

If we had a standardized representation of error, then we could also use
that in protocols like HELD to request a location within a specified
error bound, and we could also have a filter that requested location
within such a bound.  It would of course, be best to specify the XML for
such a bound once.

I'm specifically proposing that we allow error to be specified one of
two ways initially: with an actual error measurement, in meters for X, Y
and Z or as a shape (as in our other shape definitions) with an
"uncertainty" metric in percentage terms.  I would add it to GML if we
could do that, or additional elements in PIDF-LO.  I would then describe
an XML data structure to specify a bound, again, either in meters on X,
Y, and Z or confidence as a shape and uncertainty as a percentage.  I'd
allow the representation of both to be extended for other ways to
represent error.

Brian
_______________________________________________
Geopriv mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/geopriv

---

From: "Carl Reed" <[email-masked]>
Date: Mon, 14 Jul 2008 10:47:16 -0600

Brian -

There is some relevant work done (and being done) in the OGC related to this 
topic. Let me check on the current status and provide some feedback.  Also, 
the OGC is totally open to accepting requirements/change requests from the 
community and using these requirements to help drive enhancements to the OGC 
standards baseline.

FYI, there is an open call for change requests for GML. This open call 
closes in September. Any individual and/or organization can submit such 
change requests.

So, if the IETF community has any requirements that need to be considered 
for the next version of GML, please visit 
http://www.opengeospatial.org/pressroom/pressreleases/879

Thanks and regards

Carl

---

From: "Carl Reed" <[email-masked]>
Date: Mon, 14 Jul 2008 11:07:14 -0600

Brian -

A bit more background. I think that it would be appropriate to consider 
first starting with a well formed definition of what is meant by 
"measurement" and what is meant by "observation". The OGC has a standard 
called Observations and Measurements. This standard is well grounded in 
various ISO and related standards. From this document:
4.10

measure (noun)

value described using a numeric amount with a scale or using a scalar 
reference system

[ISO/TS 19103]

NOTE: When used as a noun, measure is a synonym for physical quantity.

4.11

measurement (noun) an observation whose result is a measure

4.14

observation (noun) an act of observing a property or phenomenon, with the 
goal of producing an estimate of the value of the property. A specialized 
event whose result is a data value.

4.15

phenomenon concept that is a characteristic of one or more feature types, 
the value for which may be estimated by application of some procedure in an 
observation. Synonym for property-type in this specification.

4.16

procedure method, algorithm or instrument, or system of these

So, within the context of your email, the phenomenon being measured is 
"location", the procedure might be A-GPS, the measurement is the actual 
measure (value) of the observation. In general terms:

An observation is an act associated with a discrete time instant or period 
through which a number, term or other symbol is assigned to a phenomenon 
[FOW1998]. The phenomenon is a property of an identifiable object, which is 
the feature of interest of the observation. The observation uses a 
procedure, which is often an instrument or sensor [NRC1995] but may be a 
process chain, human observer, an algorithm, a computation or simulator. The 
key idea is that the observation result is an estimate of the value of some 
property of the feature of interest, and the other observation properties 
provide context or metadata to support evaluation, interpretation and use of 
the result.

Why are having such definition important? Because any discussion of 
uncertaintly, accuracy, measurement error etc will use these terms so we 
need to have a common semantic.

The observation pattern and feature is primarily useful for capturing 
metadata associated with the estimation of feature properties, which is 
important particularly when error in this estimate is of interest.

In particular, observations concern properties (e.g. shape, color) whose 
values are determined using an identifiable procedure, in which there is a 
finite uncertainty in the result. This may be contrasted with properties 
whose values are specified by assertion (e.g. name, owner) and are therefore 
exact. The observation instance provides metadata for the property 
value-estimation process.

Unfortunately, the OGC O&M is "quiet" on uncertainty/location measurement 
error. This is because such description is outside the scope of that 
standard and is instead germane to metadata and data quality discussions.

This is why I have asked the OGC Data Quality gurus for guidance.

Apologies for the lengthy email, but this topic is non-trivial - as we all 
know!

Regards

Carl

---

From: Hannes Tschofenig <[email-masked]>
Date: Mon, 14 Jul 2008 20:25:54 +0300

I would feel much better if we push these type of things to the OGC (as 
much as we can).

We have so many other things todo in our group. Additionally, starting 
work on these topics within the IETF smells like a recipe for disaster.
Finally, we don't have the experts on this subject in the group in order 
to produce a high-quality document.

I agree with Brian that we should have something in our documents. The 
easiest way for us to get such a work done is to reference something 
that was already discussed and finished in OGC.

There is only a slight problem with that approach: there are a few 
pieces that reach into our work as well. For example, with the following 
documents there are aspect where uncertainty might reach into our 
protocol work:
http://www.ietf.org/internet-drafts/draft-ietf-geopriv-policy-17.txt
http://tools.ietf.org/wg/geopriv/draft-ietf-geopriv-loc-filters/

I strongly hope that the former document is fine now. I haven't checked 
the loc-filters since it was re-submitted today.

I would also suggest Martin and Jams to forward their uncertainty 
document to the OGC for review:
http://tools.ietf.org/id/draft-thomson-geopriv-uncertainty-01.txt
Maybe Carl has an idea on how to tackle that document.

Ciao
Hannes

Carl Reed wrote:
> Brian -
>
> There is some relevant work done (and being done) in the OGC related 
> to this topic. Let me check on the current status and provide some 
> feedback.  Also, the OGC is totally open to accepting 
> requirements/change requests from the community and using these 
> requirements to help drive enhancements to the OGC standards baseline.
>
> closes in September. Any individual and/or organization can submit 
> such change requests.
>
> So, if the IETF community has any requirements that need to be 
> considered for the next version of GML, please visit 
> ----- Original Message ----- From: "Rosen, Brian" 
> <[email-masked]>
> To: <[email-masked]>

---

From: "Brian Rosen" <[email-masked]>
Date: Mon, 14 Jul 2008 14:04:03 -0400

Although I am happy to have OGC do the actual work, I think we need to drive
the effort towards a (relatively) swift conclusion.

Towards that end, perhaps we could develop requirements and liaison them to
OGC?

Loc-filters does not touch this issue yet.  My hope is that there is a
result from whatever comes of this discussion which makes an "obvious"
addition to loc-filters to request a notification when a location with a
specified error ("quality") was available.  I certainly want to copy
something else.

Brian

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Hannes Tschofenig
> Sent: Monday, July 14, 2008 1:26 PM
> To: Carl Reed
> Cc: [email-masked]; Rosen, Brian
> Subject: Re: [Geopriv] Location measurement error
>

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 14 Jul 2008 19:58:56 -0500

The problem with location quality (draft plug: draft-thomson-geopriv-location-quality) is that the criteria therein are not designed for consumption by an LS.  The information is intended to be consumed by an LG.  The distinction is important.

Based on this reasoning, the situation is complicated.  Filters as they stand apply well to filtering of information (i.e. don't let me know unless...), location quality issues are related to the generation of location information.  Adding location quality parameters to a filter only result in the filter being invalidated when the LS is not also an LG.  

Dealing with the issue separately is the best approach and I am against adding more complexity to loc-filters.  Quality parameters are enough different to warrant having another document.

I can remember getting some (positive) OGC feedback on draft-thomson-geopriv-uncertainty, but I'm open to receiving more. 

Cheers,
M

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Brian Rosen
> Sent: Tuesday, 15 July 2008 4:04 AM
> To: 'Hannes Tschofenig'; 'Carl Reed'
> > >> If we had a standardized representation of error, then we could
> also
> > use
> > >> within such a bound.  It would of course, be best to specify the
> XML
> > for
> > >> such a bound once.
> > >>
> > >> two ways initially: with an actual error measurement, in meters
> for X,
> > Y

------------------------------------------------------------------------------------------------
This message is for the designated recipient only and may
contain privileged, proprietary, or otherwise private information.  
If you have received it in error, please notify the sender
immediately and delete the original.  Any unauthorized use of
this email is prohibited.
------------------------------------------------------------------------------------------------
[mf2]
_______________________________________________
Geopriv mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/geopriv

---

From: "Rosen, Brian" <[email-masked]>
Date: Mon, 14 Jul 2008 21:16:32 -0400

I don't agree with most of this.

Error is of interest to anyone who uses a measurement.  If you don't
know the error, you don't know what you have.  Error can be expressed in
many ways, one of which is confidence and uncertainty.  Error can be
passed through an LS without the LS looking at it.  On the other hand,
if you ask an LS to give you location within some error band, it does
have to understand the error it gets from the LG in order to meet the
request.

The notion that you ask for a location update when you are within some
error bound is a pretty reasonable request.  The usual emergency case
for wireless would use this; routing would be completed with cell or
coarse or quick fix.  Rather than poll for an accurate result, the PSAP
would request a notification of update when good (i.e. low error)
information was available.  That is about as useful an operation as you
could imagine, and much better than repeated bids or spin locks on
completion when it takes many seconds.

As with the time issue, I'm willing to have distinguished values that
essentially mean "best you can do" for the error spec (corresponding to
"EmergencyDispatch"), but the general case is filtering on a real error
bound.

Brian
> From: Thomson, Martin [mailto:[email-masked]]
> Sent: Monday, July 14, 2008 8:59 PM
> To: Brian Rosen; Hannes Tschofenig; Carl Reed
> Subject: RE: [Geopriv] Location measurement error
> 
> The problem with location quality (draft plug: draft-thomson-geopriv-
> location-quality) is that the criteria therein are not designed for
> consumption by an LS.  The information is intended to be consumed by
an
> LG.  The distinction is important.
> 
> Based on this reasoning, the situation is complicated.  Filters as
they
> stand apply well to filtering of information (i.e. don't let me know
> unless...), location quality issues are related to the generation of
> location information.  Adding location quality parameters to a filter
only
> result in the filter being invalidated when the LS is not also an LG.
> 
> Dealing with the issue separately is the best approach and I am
against
> adding more complexity to loc-filters.  Quality parameters are enough
> different to warrant having another document.
> 
> 
> I can remember getting some (positive) OGC feedback on draft-thomson-
> geopriv-uncertainty, but I'm open to receiving more.
> 
> 
> Cheers,
> M
> 
> > Although I am happy to have OGC do the actual work, I think we need
to
> > drive
> > addition to loc-filters to request a notification when a location
with
> > a
> > Behalf
> > > I would feel much better if we push these type of things to the
OGC
> > (as
> > > > FYI, there is an open call for change requests for GML. This
open
> > call
> > > >> pushing this idea because they can't do what 'real' GPS devices
do
> > and
> > > >> determination mechanisms (A-GPS and similar systems),
confidence
> > and
> > > >> often with a different value for Z than for x and y.  Other
times
> > you
> > > >> It does seem to me that when we convey measured location, that
it
> > would
> > > >> error.  It does occur to me to ask if such standardization
would
> > better
> > > >> be done in OGC, and Carl may wish to comment.  It's also
possible
> > > >> I'm specifically proposing that we allow error to be specified
one
> > of
> > > >> "uncertainty" metric in percentage terms.  I would add it to
GML
> > if we
> > > >> an XML data structure to specify a bound, again, either in
meters
> > on X,
> > > >> Y, and Z or confidence as a shape and uncertainty as a
percentage.
> > I'd
------------------------------------------------------------------------
--
> ----------------------
------------------------------------------------------------------------
--
> ----------------------

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 14 Jul 2008 23:14:57 -0500

Hi Brian,

Once again, I'm pretty sure that we disagree because we were talking about subtly different things.  And giving me a lesson on uncertainty and confidence isn't really necessary :)

I really needed to understand the use case for this, and I think that I am seeing where you are going.  What you are talking about is just the trigger that trips at a certain uncertainty.  I don't think you realise how little help this would be.  Setting an absolute goal for uncertainty would be detrimental in the PSAP case, since certain values might not be achievable.  The PSAP could wait indefinitely for a 10m uncertainty when a 15m result (which is much better than the 500m result it has) sits there waiting.  It would be better to have a rule that simply states "let me know when you get a more accurate estimate".  How you codify such a rule is tricky, but it's probably a worthwhile goal.

On the other hand, a target uncertainty sounds reasonable as a general application.  However, I struggle with finding a real-world application that couldn't be solved using more concrete methods.  The problem with specifying target uncertainty is that you are never really sure when, or even _if_, that uncertainty can be achieved.  Not being sure means that you need other criteria to base your application decision on, lest you be left waiting indefinitely.  Application developers will ultimately find their means: triggers based on <method>, polling, time constraints, or something entirely else.  I can't see how specifying uncertainty helps.

That doesn't mean that an application wont have a preference, but the difference between *preferring* 10m uncertainty and *requiring* it is a great deal.  Applying a filter is a strictly Boolean process.  That is why the location-quality draft focuses on the location generation side.  Having the consumer of the location directly affect the location generation process is of great advantage to them in terms of the resulting estimate.  Particularly where there are multiple means of location determination.

The other aspect to this is how a trigger or preference is applied.  I would expect that a loc-filter would be applied by any LS, in the sense that the LS could be detached entirely from the LG (my rhetorical GPS phone publishing a position to a presence server is a good example).  Without having a direct impact on location generation, there is no sense in expressing a preference.  A strictly Boolean filter is the only option when you don't have any means of affecting the source data.

So, I've probably given you a similarly unnecessary lesson, but the upshot is that I don't see the use case for target uncertainty in loc-filters.
	
What loc-filters needs most is to be implementable.  It is currently overly difficult to implement.  If I get some time, I'll give you a more complete critique, but most of what needs to be said has already been said.  Sad to say, but the draft has been neglected until now.

Cheers,
Martin

> From: Rosen, Brian [mailto:[email-masked]]
> Sent: Tuesday, 15 July 2008 11:17 AM
> To: Thomson, Martin; Brian Rosen; Hannes Tschofenig; Carl Reed
> Cc: [email-masked]
> > > Loc-filters does not touch this issue yet.  My hope is that there
> is
> a
> > > > Finally, we don't have the experts on this subject in the group
> in
> > > order
> > > > I agree with Brian that we should have something in our
> documents.
> > > The
> > > > >> be appropriate to convey the error of the measurement, and
> that
> > > error
> > > > >> uncertainty.  I think we need to standardize the
> representation
> of
> > > > >> could do that, or additional elements in PIDF-LO.  I would
> then
> > > > describe

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 14 Jul 2008 23:22:33 -0500

I have to pick on this statement, because I don't think that Hannes was blunt enough:

> I think we need to standardize the representation of
> error.  

*We* already did:

  <http://portal.opengeospatial.org/files/?artifact_id=21630>
  <http://tools.ietf.org/html/draft-ietf-geopriv-pdif-lo-profile>
  <http://tools.ietf.org/html/draft-thomson-geopriv-uncertainty>

As Hannes pointed out, there is work that relies on this.  Geopriv-policy is good now.  The solution is simple, which mercifully avoids a dependency on that last draft.  Loc-filters needs much more work because the issue is more complex.

MT

> Behalf Of Rosen, Brian
> Sent: Monday, 14 July 2008 11:56 PM
> To: [email-masked]

---

From: "Brian Rosen" <[email-masked]>
Date: Tue, 15 Jul 2008 09:05:05 -0400

Ah Martin

Allow me to go over some old ground
1. No watcher knows what to do with confidence and uncertainty.  I've asked
literally dozens of watchers from all kinds of emergency and
"pizza-delivery" equivalent services.  They all understood their measuring
device had error.  They ignore it.
2. When pressed to deal with it, all watchers want confidence set to at
least 95%.  It would be okay with them, at least today, if it was fixed at
95%.  Watchers HATE two measurements with different confidence.  Allowing
specification of uncertainty is completely unnecessary, fix it at 95% and
all watchers will be at least content (they won't be happy until it's 99%).
3. Watchers specify uncertainty in meters.  That means, to you, whatever the
shape that has the same error value for x and y and a different one for z.
They don't want any shape that doesn't look like that.  There is no need,
from a watcher point of view, to specify uncertainty any other way.  
4. Existing systems don't allow the watcher to specify confidence or
uncertainty.  They simply report what the owner of the measuring system
decides to report.  The notion that the error spec goes in either a request
or a filter is a novel idea, and I'm not sure it will be implemented.
5. All PSAPs and all responders in the U.S. have a very well defined value
for confidence and uncertainty for a filter: within 100 meters (x and y),
95% of the time. 
6. With a minRate, you cover the case of "best you can do in x seconds".
Without a way to specify error in some way, you can only say that.  If you
want "best you can do", there is no way to say that.  If you want to say
"within 10 meters is good enough" there is no way to say that.  
7. Event notification is a much more acceptable way to get a measurement
that takes seconds to compute.  The error spec in the filter is much more
useful that the error spec in the request.  If you want an error return that
is effectively "you'll never get one with this filter" I would be okay with
that, but see below.
8. The biggest problem with all of the mechanisms that have so far been
proposed is that they don't include any way for a watcher to determine what
an achievable error could be from a particular LG (or LS, since that is what
it deals with).  Until you come up with a way for the watcher to find what
the measurement mechanism's achievable error spec is, you don't have a
useful set of mechanisms.

It occurs to me to ask if "within a 100 meters, 95% of the time" is actually
the same as a circular error shape with a 100 meter radius and a 95%
confidence.  It sounds like the same thing.  Is it exactly the same thing?

Brian 

> Of Thomson, Martin
> Sent: Tuesday, July 15, 2008 12:15 AM
> To: Rosen, Brian
> Cc: [email-masked]
> Once again, I'm pretty sure that we disagree because we were talking about
> subtly different things.  And giving me a lesson on uncertainty and
> confidence isn't really necessary :)
> 
> I really needed to understand the use case for this, and I think that I am
> seeing where you are going.  What you are talking about is just the
> trigger that trips at a certain uncertainty.  I don't think you realise
> how little help this would be.  Setting an absolute goal for uncertainty
> would be detrimental in the PSAP case, since certain values might not be
> achievable.  The PSAP could wait indefinitely for a 10m uncertainty when a
> 15m result (which is much better than the 500m result it has) sits there
> waiting.  It would be better to have a rule that simply states "let me
> know when you get a more accurate estimate".  How you codify such a rule
> is tricky, but it's probably a worthwhile goal.
> 
> On the other hand, a target uncertainty sounds reasonable as a general
> application.  However, I struggle with finding a real-world application
> that couldn't be solved using more concrete methods.  The problem with
> specifying target uncertainty is that you are never really sure when, or
> even _if_, that uncertainty can be achieved.  Not being sure means that
> you need other criteria to base your application decision on, lest you be
> left waiting indefinitely.  Application developers will ultimately find
> their means: triggers based on <method>, polling, time constraints, or
> something entirely else.  I can't see how specifying uncertainty helps.
> 
> That doesn't mean that an application wont have a preference, but the
> difference between *preferring* 10m uncertainty and *requiring* it is a
> great deal.  Applying a filter is a strictly Boolean process.  That is why
> the location-quality draft focuses on the location generation side.
> Having the consumer of the location directly affect the location
> generation process is of great advantage to them in terms of the resulting
> estimate.  Particularly where there are multiple means of location
> determination.
> 
> The other aspect to this is how a trigger or preference is applied.  I
> would expect that a loc-filter would be applied by any LS, in the sense
> that the LS could be detached entirely from the LG (my rhetorical GPS
> phone publishing a position to a presence server is a good example).
> Without having a direct impact on location generation, there is no sense
> in expressing a preference.  A strictly Boolean filter is the only option
> when you don't have any means of affecting the source data.
> 
> So, I've probably given you a similarly unnecessary lesson, but the upshot
> is that I don't see the use case for target uncertainty in loc-filters.
> 
> What loc-filters needs most is to be implementable.  It is currently
> overly difficult to implement.  If I get some time, I'll give you a more
> complete critique, but most of what needs to be said has already been
> said.  Sad to say, but the draft has been neglected until now.
> 
> Cheers,
> Martin
> 
>

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 15 Jul 2008 22:58:24 -0500

Ah Brian

Familiar old ground, made interesting by your particular rhetorical flair.

> From: Brian Rosen [mailto:[email-masked]]
> Sent: Tuesday, 15 July 2008 11:05 PM
> To: Thomson, Martin
> Cc: [email-masked]

Debatable, but hardly relevant, so I wont belabour the point.

> all watchers will be at least content (they won't be happy until it's

I agree on the different confidence - given the non-linear relationship, managing two variables is a pain.

Everyone wants 100%, but fail to acknowledge the tradeoffs.  I think that I can see where you are going - you are making points about what people _want_.  I can appreciate that.

> from a watcher point of view, to specify uncertainty any other way.

The point that people understand the expression of uncertainty as "within x metres" is relevant.  But what they want is no uncertainty, failing that, they want simple.  I've describe how to accommodate these wishes, while not ignoring the real constraints: <http://tools.ietf.org/html/draft-thomson-geopriv-uncertainty>

> or a filter is a novel idea, and I'm not sure it will be implemented.

Now you've lost me.  Specifying target uncertainty is not especially novel, we've been handling it for almost 10 years now.  Quality of position parameters are a fixture of most existing LCS protocols.  They get pushed down the whole chain.  For example: Application -> GMLC -> MSC -> RNC -> SAS - all the protocols have QoP parameters of various sorts.

I've explained why I think that adding an "error spec" to filters is not useful.

> for confidence and uncertainty for a filter: within 100 meters (x and

If you are talking FCC mandates, you are still getting them wrong.  In any case, these numbers cannot be applied to an individual call - they are statistical figures that are applied across a large set of calls.  Again, this falls into the "want" basket.

> "within 10 meters is good enough" there is no way to say that.

Actually, with a minRate, you specify that the LS update you every x seconds.  Unless there is a way for the LS to push the LG, you only get a better estimate if the LG happens to get one.  That is my point - and one that you seem to be reinforcing.  I'm officially confused now.

> is effectively "you'll never get one with this filter" I would be okay

OK, say that I subscribe for location information.  You are saying that specifying a filter is better than cramming some extra parameters in the subscription request.  I can't contest that.  It misses the point, but in and of itself, the argument is sound.

The problem with a filter (or indeed anything related to the subscription and only the subscription) is that whatever you do there, you have no influence over the information.  You get whatever the LG generates.  Without some influence over the location generation, anything you do is essentially pointless.  Unless the LG is given instructions on what to do, you get whatever you get.  Assuming that the LG is independently motivated to generate an extremely accurate result isn't going to work.  The model can't assume that the LS has some secret means to influence the LG.  Unless you specify a means for the LS to do so, which I'd be very interested in.

> useful set of mechanisms.

That's because noone knows what the achievable error is.  It's not related to the "mechanism", but to the specific situation: time of day, wind speed and direction, battery levels, indoors/outdoors, access technology, what the owner had for lunch, solar winds, etc.
 
> confidence.  It sounds like the same thing.  Is it exactly the same

Yes.  I'm surprised that this realization took so long, given that the exact statement has been made many times in the past.

> > > know the error, you don't know what you have.  Error can be
> expressed
> > > in
> > > > > Towards that end, perhaps we could develop requirements and
> liaison
> > > > > them to
> > > > > OGC?
> > > > >
> > > > > > From: [email-masked] [mailto:geopriv-
> [email-masked]]
> > > On
> > > > > Behalf
> > > > > > Of Hannes Tschofenig
> > > > > > There is only a slight problem with that approach: there are
> a
> > > few
> > > > > > pieces that reach into our work as well. For example, with
> the
> > > > > following
> > > > > > I strongly hope that the former document is fine now. I
> haven't
> > > > > checked
> > > > > > > There is some relevant work done (and being done) in the
> OGC
> > > > > related
> > > > > > > to this topic. Let me check on the current status and
> provide
> > > some
> > > > > > > requirements to help drive enhancements to the OGC
> standards
> > > > > baseline.
> > > > > > >
> > > > > > >> I'd like to start a discussion on how we deal with, in the
> big
> > > > > picture,
> > > > > > >> the notion of measurement error in general and
> "confidence"
> > > and
> > > > > > >> I am aware that there are a set of views that go roughly
> like:
> > > > > "there
> > > > > > >> really is no such thing as confidence and uncertainty,
> some
> > > > > vendors are
> > > > > > >> In other systems, notably commercial GPS systems, you get
> a
> > > > > different
> > > > > > >> kind of error specification.  Sometimes it's simply error
> in
> > > > > meters,
> > > > > > >> the geopriv crowd to come up with something and then OGC
> to
> > > > > incorporate
> > > > > > >> error bound, and we could also have a filter that
> requested
> > > > > location
> > > > > > >> within such a bound.  It would of course, be best to
> specify
> > > the
> > > > > XML
> > > > > > for
> > > > > > >> such a bound once.
> > > > > > >>
> > > > > > >> and Z or as a shape (as in our other shape definitions)
> with
> > > an
> > > > > > >> allow the representation of both to be extended for other
> ways
> > > to
> > > > > > >> represent error.
> > > > > > >>
> > > > > > >> Brian
> >
> > ---------------------------------------------------------------------
> -----
> > ----------------------
> > ---------------------------------------------------------------------
> -----
> > ----------------------
> > [mf2]

---

From: "Carl Reed" <[email-masked]>
Date: Wed, 16 Jul 2008 09:54:10 -0600

All -

A bit more on the topic from the OGC community:

- In geospatial data there are a number of spatial accuracy definitions, 
notably absolute and relative accuracy which are completely different--this 
would need to come out of any standardised way to express measurement 
accuracies. z-accuracy are often a different beast altogether because of 
height reference systems which are often completely independent of the 
planimetric ones.

- The proposal talks about "an actual error measurement". Error's can't be 
measured, but are statistical expressions of likelihood between the 
measurement result and a perceived or defined true value.

- Different ways of expressing the accuracy, particularly in GPS relate to 
the use different methods (with different complexities of calculating these 
error values. The actual measurements within a GPS receiver are actually not 
the coordinates or distances but time of phase-differences of the received 
signals. The coordinates are then derived and calculated from the 
measurements. It is important to note that some there is usually a certain 
amount of correlation between individual measurements (i.e. measurements 
taken by the same instrument which distorts the measurement in the same 
way). Often these correlations are not taking into account when accuracy 
statements are given.

- For what is worth I think this is worthy of further discussion in our 
group though I agree with Carsten regarding the nature of GPS accuracy.  Our 
survey on Geospatial Data Quality showed that accuracy in general is one of 
the most important issues for both users and vendors of data. Not surprising 
I suppose.  But as Carsten said these issues of accuracy can be different 
for different groups:  Geodesists are thinking about accuracy and data 
quality in respect to changes made to the data through things like 
coordinate transformation or such things.   Yet, any data generation or 
manipulation needs/should have some sort of quality or accuracy measure.

I believe that we have an excellent opportunity for collaboration between 
communities. AS one can quickly see, this is a non-trivial issue and one 
that has not been solved in any "standardized" way.

Regards

Carl

---

From: "Brian Rosen" <[email-masked]>
Date: Thu, 17 Jul 2008 18:00:56 -0400

Sorry for the delay, but I am still wallowing

> Everyone wants 100%, but fail to acknowledge the tradeoffs.  I think that
> I can see where you are going - you are making points about what people
> _want_.  I can appreciate that.
Well, it's more of a requirement.  It's also very related to "they ignore
it".  Since the data they get has variable confidence, the uncertainty
doesn't mean the same thing.

I'm specifically suggesting, somewhat naively, that there not be a variable
for confidence, but it simply be set to 95%.  

The reality might be that the text in all the documents say:
Confidence MUST be set to 95% except for certain existing devices which will
not currently provide measurements with such a value.
> The point that people understand the expression of uncertainty as "within
> x metres" is relevant.  But what they want is no uncertainty, failing
> that, they want simple.  I've describe how to accommodate these wishes,
> while not ignoring the real constraints:
Yeah, pretty nice piece of work.  I do need to send you some comments, but I
still like it a lot.

You could reasonably ask, if the users only want simple, is it better to
push the calculations described back to the LS?

> Now you've lost me.  Specifying target uncertainty is not especially
> novel, we've been handling it for almost 10 years now.  Quality of
> position parameters are a fixture of most existing LCS protocols.  They
> get pushed down the whole chain.  For example: Application -> GMLC -> MSC
> -> RNC -> SAS - all the protocols have QoP parameters of various sorts.
In the protocols with which I am familiar, you cannot specify uncertainty,
you can only receive it when you get a measurement.  OTOH, I have limited
experience outside the emergency call system.  In that system, both the E2
and ALI protocols have a way to return uncertainty, but no way to request a
specific uncertainty (or confidence).

> I've explained why I think that adding an "error spec" to filters is not
Yeah, but we are disagreeing on that point still

> If you are talking FCC mandates, you are still getting them wrong.  In any
> case, these numbers cannot be applied to an individual call - they are
> statistical figures that are applied across a large set of calls.  Again,
> this falls into the "want" basket.
It's the only thing we have.  This is the source of endless frustration to
me and the PSAP folks.  We want to put in "6 cm uncertainty, 100%
confidence".  It's the only spec we actually have.   We have a regulation
that says 100 meters, 95% of the time.  What do you want a PSAP to put in
the request?

This is the nut of the problem I have.  The user goal is (nearly always)
infinite precision, 100% confidence.  If the system didn't deliver them
location that met some purchase, legal or other spec, they wouldn't be using
it in the first place.  There IS no other spec other than "best you can do".

Clearly, as we have discussed several times, you sometimes have an
expressable time constraint (best you can do in 10 seconds).  I can't think
of any application where there is a meaningful value that could be put into
a request other than "best you can do", but you insist we come up with some
numbers.

I'd really like to get off this merry go round.  Your end says "tell me your
spec".  My end says "best you can do" or the spec is zero uncertainty and
100% confidence is all we got.

> Actually, with a minRate, you specify that the LS update you every x
> seconds.  Unless there is a way for the LS to push the LG, you only get a
> better estimate if the LG happens to get one.  That is my point - and one
> that you seem to be reinforcing.  I'm officially confused now.
I refuse to get drawn into the difference between an LS to watcher and LG to
watcher and/or LS to LG to watcher.  We may need another protocol for LS to
LG.  So far, we don't have one.  When we are in that circumstance, we ignore
the difference between LG and LS; assume a proprietary protocol, or a single
box.  The watcher tells the LS, the LS tells the LG.  

We talked about a similar problem in a NENA call today: if you use the
enter/exit boundary filter, at what rate should the LG take measurements to
give you the desired result (how long after you entered/left the boundary
are you willing to wait to be told you entered or left)?  I suggest that you
can use minRate as a hint, and perhaps we should say that.  We could also
have a specific parameter for the purpose.

> OK, say that I subscribe for location information.  You are saying that
> specifying a filter is better than cramming some extra parameters in the
> subscription request.  I can't contest that.  It misses the point, but in
> and of itself, the argument is sound.
> 
> The problem with a filter (or indeed anything related to the subscription
> and only the subscription) is that whatever you do there, you have no
> influence over the information.  You get whatever the LG generates.
> Without some influence over the location generation, anything you do is
> essentially pointless.  Unless the LG is given instructions on what to do,
> you get whatever you get.  Assuming that the LG is independently motivated
> to generate an extremely accurate result isn't going to work.  The model
> can't assume that the LS has some secret means to influence the LG.
> Unless you specify a means for the LS to do so, which I'd be very
> interested in.
See above

> That's because noone knows what the achievable error is.  It's not related
> to the "mechanism", but to the specific situation: time of day, wind speed
> and direction, battery levels, indoors/outdoors, access technology, what
> the owner had for lunch, solar winds, etc.

Okay, so what you are saying is this:
We haven't a clue what we can do.  
The app has no clue what you can do.
The app wants zero error and 100% confidence
The app wants the best you can do (with perhaps a time constraint)

What the heck do you want it to say?

Oh, let's see, let's try 6cm and 99% confidence.  Gee, no answer.  Darn.

Okay, let's try 6cm and 95% confidence.  Gee, no answer.  Darn again.

Okay, let's try 1m and 95% confidence

You wouldn't go the other way:

Okay, let's give it 2km uncertainty and 10% confidence.  Oh, great, an
answer.  

I wonder if it will do better.  Let's try 1km and 30%...

That's the real, honest situation.

It's the reality for pizza hut
It's the reality for 1-1-2
It's the reality for turn by turn directions
It's the reality for "where is the nearest Starbucks"
It's the reality for "tell me when my kid goes outside the mall"

What the heck do you want the app to do?

> Yes.  I'm surprised that this realization took so long, given that the
> exact statement has been made many times in the past.

Actually, you just said it wasn't the same thing above.  You said that 100 m
95% of the time (the FCC mandate) was not the same as 95 meter circular
radius uncertainty with 95% confidence.  You said it was a statistical
calculation over a large number of measurements (implying it's not the spec
you use for a single measurement).

Okay, so if you are a U.S. PSAP, what DO you ask for?

Brian

---

From: "Thomson, Martin" <[email-masked]>
Date: Thu, 17 Jul 2008 18:17:10 -0500

> Sent: Friday, 18 July 2008 8:01 AM
> To: Thomson, Martin
> Cc: [email-masked]
> > > least 95%.  It would be okay with them, at least today, if it was
> fixed
> > > at
> > > specification of uncertainty is completely unnecessary, fix it at
> 95%
> > > and
> > > all watchers will be at least content (they won't be happy until
> it's
> > > 99%).
> >
> > I agree on the different confidence - given the non-linear
> relationship,
> > managing two variables is a pain.
> >

That's reasonable, and a good idea.  Attempting to change confidence would be a bad idea, but setting a target for all "new" implementations means that we don't get into any difficulties.

> > > shape that has the same error value for x and y and a different one
> for
> > > z.

Maybe we could do something with a request - provide a way to indicate that only lat,lng,r is desired.  That way, any loss of information is explicitly accepted.  I'm not sure that this is appropriate for a filter, but maybe we can attach something else to the request.

> specific uncertainty (or confidence).

E2 is quite simplistic.  The protocols for value-added services are far more sophisticated.

> > I've explained why I think that adding an "error spec" to filters is
> not
> > useful.

Not entirely - I'm not so violently opposed.  I just think that it isn't going to solve many real problems, if any.

> > > 5. All PSAPs and all responders in the U.S. have a very well
> defined
> > > value
> > > for confidence and uncertainty for a filter: within 100 meters (x
> and
> > > y),
> > > 95% of the time.
> >

If you allow the user to specify the numbers, then what you do is give them power over the time-accuracy-cost tradeoff.  Like I said before, this is common for value-added services, where cost actually means cost and application developers know something about the way that location is generated.  By specifying a target accuracy that is good enough for their application, they are able to save money (and time) when cheap methods work, but still meet the goals of their application.  Being able to deal with uncertainty is a necessary virtue for these applications, specifying targets makes it a little more palatable.

If you go for the best in every case, you end up spending a lot of money.  Take a look sometime at the performance that uplink time difference of arrival (UTDOA or U-TDOA) provides and the cost of determining a good position.  It's not cheap.  For emergency services, the FCC mandates and all the other emotive arguments mean that this cost is justified.  People value security quite highly.

So for emergency services, specifying targets like zero uncertainty might be reasonable.  We don't ever get that, but we can try (and spend all the money we have trying).  That doesn't work so well for commercial services, who will baulk at the bill.

> > > Without a way to specify error in some way, you can only say that.
> If
> > > you
> > > want "best you can do", there is no way to say that.  If you want
> to
> > > say

Actually, I think that Richard's requirements document might help us here.  Section 5, Figure 3.  If the LG were assumed to be an LS, then we have a means to do this.  Each LS is able to pass their preferences (as an LR) to the LS that served them, all the way down to the LG/LS (which might be a LIS, but could be the Target).

http://tools.ietf.org/html/draft-barnes-geopriv-lo-sec#section-5

However, this doesn't help, because this is location-specific stuff and the LS might be a presence server, using a vanilla implementation of the presence event package.  The model for filters in that package doesn't assume that the LS needs to prod its source in any particular way - there are no alternative ways to determine that someone is "Away" or "Do not disturb".

This is why I believe that the model of filters (and the presence event package), while addressing the basic goals, fails when dealing with information that can be incomplete.

> > > that takes seconds to compute.  The error spec in the filter is
> much
> > > more
> > > useful that the error spec in the request.  If you want an error
> return
> > > that
> > > is effectively "you'll never get one with this filter" I would be
> okay
> > > with
> > > that, but see below.
> >
> > > proposed is that they don't include any way for a watcher to
> determine
> > > what
> > > an achievable error could be from a particular LG (or LS, since
> that is
> > > what
> > > it deals with).  Until you come up with a way for the watcher to
> find
> > > what

I want the app to think about what their needs are.  

For pizza hut, civic might be best for the delivery driver.  Geodetic might need to be more accurate than is realistically possible.  

1-1-2 will always want the best.  No question.

Turn by turn directions will rely on their GPS and be able to make corrections based on the maps  (i.e. pick the closest road), which means that 50m might be good enough.

Nearest Starbucks only requires that the uncertainty be smaller than the distance between Starbucks outlets (which based on my Seattle visit is about 50m too :).

Geo-fencing applications can use iterative values - smaller as the target approaches (or might be approaching, based on uncertainty) boundaries.

In most cases, you want the LS to provide you a location, any location, even if it can't meet your ideal goals.  You might be able to do something.  After all, maximum likelihood estimation works in other scenarios as well.

> you use for a single measurement).

The FCC mandate is:

  Network based: 300m@95%, 100m@67%
  Handset based: 150m@95%, 50m@67%

That's why I say that the numbers were wrong...  And yes, the mandate is for a large sample set, based on the expected (centroid) value of each result against a measured ground truth.  For a single position with 100m radius and 95%, this means that whatever generated the information is declaring that 95% of the time, the measurand/Target is within 100m of the indicated centre point.

> Okay, so if you are a U.S. PSAP, what DO you ask for?

No calls please.

Seriously?  A U.S. PSAP wants the best location information possible.  They want the person who is generating the location information to throw everything they've got at it.  And from experience, U.S. mobile phone carriers do exactly that.

However, to focus on the emergency case misses the point of documents like loc-filters and location-quality.  Emergency is too black and white.  You do better to talk to the guys who write the LBS applications when you do these sorts of documents. 

Cheers,
Martin

---

From: "Brian Rosen" <[email-masked]>
Date: Thu, 17 Jul 2008 21:11:14 -0400

I'm not arguing that measurements have no error.
I'm not disputing that the LBS vendors have settled on confidence and
uncertainty as the way they will express error.
I'm not arguing IETF protocols should ignore error.

I am suggesting that apps don't know what values to use for uncertainty and
confidence.  They consistently want "best you can do".

If you really, truly have multiple mechanisms that can be used pretty much
at an apps whim (even if there are cost implications to the app), which is
pretty rare so far, then the notion that you insist the app tell you what
uncertainty it will accept without having the LG tell the app what
uncertainty it can achieve makes no engineering sense to me.

What I see out there is much simpler.  The system has a spec, the app
expects the system to meet the spec, and that's the end of the story.  We
should be able to make our protocols work in such simple systems with
minimal fuss.

We're getting closer on confidence.  The NENA/APCO filing suggests everyone
use the same confidence value, which I believe will be 95%.  That's the
suggestion we seem to be heading towards here (with some exceptions).

I'm not opposed to allowing an app to request a specific uncertainty,
especially if it can use simple metrics (e.g. within 3 meters) as long as
there is a distinguished value or other mechanism that means "best you can
do".

It has always made sense to me to return the error of the measurement.

I want to be able to filter on error, although 90% of the problem is solved
with "send me an update when you have the best answer you can get", as long
as we can specify a max time for that.  I think it does make as much sense
to filter on the error as it makes sense to send the error spec to the LG.
In fact, since I don't distinguish between the LG and the LS, since we don't
currently specify protocols between the LG and the LS, I think they are
nearly the same thing.

Brian

> From: Dawson, Martin [mailto:[email-masked]]
> Sent: Thursday, July 17, 2008 8:23 PM
> To: Brian Rosen; Thomson, Martin
> Cc: [email-masked]
> I might hate the fact that sunshine can give me skin cancer, but the
> physics is undeniable - even if I put "hate" in upper case.
> 
> All location-based applications that get geodetic areas of uncertainty
> have to deal with the fact that:
> 
> a) The location of the target can only be provided as an area that the
> target may be in (uncertainty)
> b) There is a possibility that the target isn't even in that area (100%
> - confidence)
> 
> If we're honest, the same kind of "uncertainty" exists for civic
> location as well.
> 
> An application that ignores this information isn't as good as one that
> optimizes its procedures based on this information. Despite what you
> say, I don't actually believe that PSAPs don't have procedures that deal
> with this fact. They may not be formalised through the ODC, they may be
> by convention, and might not even be documented. However, I'm confident
> they don't just dispatch to the centre point and then give up if the
> caller isn't there. In any case, an emergency application that doesn't
> proceduralise this fact isn't as good as one that does.
> 
> Actually, this week's filing from APCO and NENA in response to the FCC
> NPRM demonstrates that they well recognize that accuracy can't be
> arbitrarily demanded and, indeed, are recommending that the FCC back off
> on arbitrary accuracy demands. I've attached the filing - in the,
> perhaps vain, hope that it will survive the list server...
> 
> Bottom line - I don't think "hate" qualifies as reasonable justification
> for a requirement.
> 
> With respect to the FCC mandate versus what the query protocol might
> support, there is an obvious difference. The mandate just states that
> the cellular network is statistically within the uncertainty requirement
> across all the locations they return. There is no requirement to
> "tailor" the confidence on a per-request basis. As such, you'll find
> that, for example, the radius of uncertainty returned from a cell-based
> location may just be based on the distance from the tower at which the
> signal nominally drops by 3dB (half power). This has no particular
> relationship with where in that area of coverage most calls actually
> originate or, more subtly, where most 9-1-1 calls originate. If the
> carrier performance was evaluated on how accurately their provided
> uncertainty actually matched a nominal confidence (rather than just the
> statistical variance based on the centre points provided) then they
> would need to tailor those cell boundaries more carefully. However, that
> would have to be against a specific confidence, it's not like their
> systems are designed to generate a location based on an arbitrary
> required level of confidence. The algorithms in some of their LG
> components my have the necessary wherewithal embedded in them (e.g.
> UTDOA, fingerprinting, GPS) since convergence threshold parameters in
> those algorithms can be driven by confidence. However, it's by no means
> always possible to do that and it's by no means common that
> implementations allow these convergence parameters to be controlled on a
> per-request basis.
> 
> Cheers,
> Martin
> 
> > > an achievable error could be from a particular LG (or LS, since that
> is
> > > what

---

From: "Dawson, Martin" <[email-masked]>
Date: Thu, 17 Jul 2008 19:23:02 -0500

Cheers,
Martin

-----Original Message-----

fixed
95%
it's
relationship,
> _want_.  I can appreciate that.
Well, it's more of a requirement.  It's also very related to "they
ignore
it".  Since the data they get has variable confidence, the uncertainty
doesn't mean the same thing.

I'm specifically suggesting, somewhat naively, that there not be a
variable
for confidence, but it simply be set to 95%.  

The reality might be that the text in all the documents say:
Confidence MUST be set to 95% except for certain existing devices which
will
not currently provide measurements with such a value.
for
> <http://tools.ietf.org/html/draft-thomson-geopriv-uncertainty>
Yeah, pretty nice piece of work.  I do need to send you some comments,
but I
still like it a lot.

> -> RNC -> SAS - all the protocols have QoP parameters of various
sorts.
In the protocols with which I am familiar, you cannot specify
uncertainty,
you can only receive it when you get a measurement.  OTOH, I have
limited
experience outside the emergency call system.  In that system, both the
E2
and ALI protocols have a way to return uncertainty, but no way to
request a
specific uncertainty (or confidence).

not

and
> this falls into the "want" basket.
It's the only thing we have.  This is the source of endless frustration
to
me and the PSAP folks.  We want to put in "6 cm uncertainty, 100%
confidence".  It's the only spec we actually have.   We have a
regulation
that says 100 meters, 95% of the time.  What do you want a PSAP to put
in
the request?

This is the nut of the problem I have.  The user goal is (nearly always)
infinite precision, 100% confidence.  If the system didn't deliver them
location that met some purchase, legal or other spec, they wouldn't be
using
it in the first place.  There IS no other spec other than "best you can
do".

Clearly, as we have discussed several times, you sometimes have an
expressable time constraint (best you can do in 10 seconds).  I can't
think
of any application where there is a meaningful value that could be put
into
a request other than "best you can do", but you insist we come up with
some
numbers.

I'd really like to get off this merry go round.  Your end says "tell me
your
spec".  My end says "best you can do" or the spec is zero uncertainty
and
100% confidence is all we got.

If
> that you seem to be reinforcing.  I'm officially confused now.
I refuse to get drawn into the difference between an LS to watcher and
LG to
watcher and/or LS to LG to watcher.  We may need another protocol for LS
to
LG.  So far, we don't have one.  When we are in that circumstance, we
ignore
the difference between LG and LS; assume a proprietary protocol, or a
single
box.  The watcher tells the LS, the LS tells the LG.  

We talked about a similar problem in a NENA call today: if you use the
enter/exit boundary filter, at what rate should the LG take measurements
to
give you the desired result (how long after you entered/left the
boundary
are you willing to wait to be told you entered or left)?  I suggest that
you
can use minRate as a hint, and perhaps we should say that.  We could
also
have a specific parameter for the purpose.

return
okay
See above

determine
is
find

> exact statement has been made many times in the past.

Actually, you just said it wasn't the same thing above.  You said that
100 m
95% of the time (the FCC mandate) was not the same as 95 meter circular
radius uncertainty with 95% confidence.  You said it was a statistical
calculation over a large number of measurements (implying it's not the
spec
you use for a single measurement).

Brian

---

From: [email-masked]
Date: Tue, 22 Jul 2008 14:32:41 -0400 (EDT)

Some additional information that may be of use in discussion related to
uncertainty. http://www.intamap.org/uncertml/uncertml.php

The referenced document describes some very recent work on how to describe
and document uncertainty -

>From the document

UncertML is an XML schema for describing uncertain information. UncertML
provides a framework to allow uncertainty to be propagated through many
applications including online risk management chains. The uncertainty that
can be described in UncertML ranges from summaries such as simple
statistics (e.g. the mean and variance of an observation) to far more
complex representations such as parametric distributions at each point of
a regular grid, perhaps generated by some geostatistical interpolation
process. More detailed requirements for UncertML are given in Sections 2 &
3.

UncertML uses GML.

The document has been brought into the OGC process for discussion and
standardization and is part of our Data Quality work.

Hope this helps.

Carl

---

From: "Carl Reed" <[email-masked]>
Date: Tue, 29 Jul 2008 14:26:12 -0600

Just to stir the discussion a bit more . . .

The following is from the team that has been working UnCertML.

"We did take a look at the [Geopriv] Location measurement error stuff - they 
have the most complex description of how to create multivariate 
distributions / error statistics I have ever seen. I think the notation and 
terminology must be domain specific - it took me a while to figure out what 
was the aim - I feel there are far simpler ways (as you might expect we 
would suggest UncertML to be used here, since the model for representing the 
error is very flexible but semantically precise)."

If a PhD that specializes in statistics and probabilistic treatments of 
systems and processes has a hard time understanding what GeoPRIV is trying 
to solve WRT location measurement error, then there is an issue. However, as 
the authors stated, they are not telecommunications domain experts.

Regards

Carl

----- Original Message ----- 
From: "Dawson, Martin" <[email-masked]>
To: "Brian Rosen" <[email-masked]>; "Thomson, Martin" 
<[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, July 17, 2008 6:23 PM
Subject: Re: [Geopriv] Location measurement error

Cheers,
Martin

-----Original Message-----

fixed
95%
it's
relationship,

for

not

and

If

return
okay
See above

determine
is
find

Brian

--------------------------------------------------------------------------------

---

From: "Dawson, Martin" <[email-masked]>
Date: Tue, 29 Jul 2008 15:54:31 -0500

Hi Carl,

Can you elaborate on which particular document(s) this review is
applicable to?

Cheers,
Martin

-----Original Message-----
From: Carl Reed [mailto:[email-masked]] 
Sent: Wednesday, 30 July 2008 6:26 AM
To: Dawson, Martin; Brian Rosen; Thomson, Martin
Cc: [email-masked]
Subject: Re: [Geopriv] Location measurement error

"We did take a look at the [Geopriv] Location measurement error stuff -
they 
have the most complex description of how to create multivariate 
distributions / error statistics I have ever seen. I think the notation
and 
terminology must be domain specific - it took me a while to figure out
what 
was the aim - I feel there are far simpler ways (as you might expect we 
would suggest UncertML to be used here, since the model for representing
the 
error is very flexible but semantically precise)."

If a PhD that specializes in statistics and probabilistic treatments of 
systems and processes has a hard time understanding what GeoPRIV is
trying 
to solve WRT location measurement error, then there is an issue.
However, as 
the authors stated, they are not telecommunications domain experts.

Regards

Carl

----- Original Message ----- 
From: "Dawson, Martin" <[email-masked]>
To: "Brian Rosen" <[email-masked]>; "Thomson, Martin" 

Cheers,
Martin

-----Original Message-----

fixed
95%
it's
relationship,

for

not

and

If

return
okay
See above

determine
is
find

Brian

------------------------
------------------------
[mf2]

--------

---

From: "Carl Reed" <[email-masked]>
Date: Tue, 29 Jul 2008 14:56:35 -0600

I will find out.

Thanks

Carl

----- Original Message ----- 
From: "Dawson, Martin" <[email-masked]>
To: "Carl Reed" <[email-masked]>; "Brian Rosen" 
<[email-masked]>; "Thomson, Martin" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, July 29, 2008 2:54 PM
Subject: RE: [Geopriv] Location measurement error

---

From: "Thomson, Martin" <[email-masked]>
Date: Wed, 30 Jul 2008 13:58:03 -0500

I'd also like to request that you put those who did the review in contact with me.  I'm interested in listening to their concerns.  Perhaps there is something there that can be improved upon.

Cheers,
Martin

> Sent: Tuesday, 29 July 2008 9:57 PM
> >> > 2. When pressed to deal with it, all watchers want confidence set
> to
> > at
> >> > They don't want any shape that doesn't look like that.  There is
> no
> >> > need,
> >> > decides to report.  The notion that the error spec goes in either
> a
> >> > request
> >> get pushed down the whole chain.  For example: Application -> GMLC -
> >
> > MSC
> >> If you are talking FCC mandates, you are still getting them wrong.
> In
> > any
> >> better estimate if the LG happens to get one.  That is my point -
> and
> > one
> >> subscription request.  I can't contest that.  It misses the point,
> but
> > in
> >> essentially pointless.  Unless the LG is given instructions on what
> to
> > do,
> >> > the measurement mechanism's achievable error spec is, you don't
> have
> > a
> >> > It occurs to me to ask if "within a 100 meters, 95% of the time"
> is
> >> > actually
> >> > confidence.  It sounds like the same thing.  Is it exactly the
> same
> >> > thing?
> >>
> ---
> ---
> ---
> ---------------------------
> ---------------------------
> > [mf2]
> >
> >
>