Subject: Re: [Geojson] Extensibility of GeoJSON

From: Jeff Yutzler <[email-masked]>
Date: Sat, 21 Nov 2015 10:37:33 -0500

> From: Martin Daly <[email-masked]>
> Interoperability is inversely proportional to extensibility
Thank you for that bit of theory, but in the real world it is the lack of
an extension mechanism that is harming interoperability. Properties are
colliding. Common solutions to common problems are not being shared.

> Just use “properties”, which is both free form (within the constraints of
JSON), and is expected by all software that consumes GeoJSON. It is also
hierarchical, so a “simplestyle” object would not “clutter up”
“properties”, and would avoid key collisions.

Note that in the absence of guidance, this has not happened. The current
state is a direct consequence of inaction on GeoJSON's part. Treating the
properties object as a free-for-all benefits no one.

> TIFF (ie Adobe) anointed some image compressions (analogous to “geometry”
in GeoJSON) as baseline and some, oxymoronically, as standard extensions.
The effect is that all of these anointed compressions are baseline ones.
Anyone wishing to use other compression algorithms is allowed to do so, but
they can’t call it TIFF.

I am not aware of anyone who intends to replace GeoJSON geometry with
something else[1]. The examples from Sean are much more benign - properties
that could be useful (if they are done in a common way) but are by no means
essential. Sean's five guidelines in combination with a key that defines
the extensions in use (take them or leave them) would not destroy the
sanctity of GeoJSON, they would help preserve it.

-Jeff

[1] For the time being, let's ignore TopoJSON which somehow manages to
declare itself as an extension of GeoJSON even though it is not actually
GeoJSON compliant.

---

From: Martin Daly <[email-masked]>
Date: Sat, 21 Nov 2015 15:52:00 +0000

Your assertion re. interoperability is not borne out by the success and uptake of GeoJSON, so you’ll have to try harder.

Don’t confuse your lack of awareness with what’s happening in the real world. This is hardly a black swan, having been mentioned on lists and Github several times:

https://github.com/dret/GeoJSON-X

Sean’s examples are not benign, and nor is the solution. It imposes behaviour on consuming and producing software (maintaining all unexpected keys at the FeatureCollection and Feature level). That directly affects existing implementations, which would be rendered incompatible by such a change.

I fail to see why all implementations (there’s more to life than ogr2ogr you know) should have to change just because some people made obviously poor choices about where to put feature properties. There has always been a place to put properties: the clue is in the name.

The properties object is a free-for-all, and always has been. To pretend otherwise is fatuous.

Martin

From: GeoJSON [mailto:[email-masked]] On Behalf Of Jeff Yutzler
Sent: 21 November 2015 15:38
To: [email-masked]
Subject: Re: [Geojson] Extensibility of GeoJSON

> From: Martin Daly <[email-masked]<mailto:[email-masked]>>
Thank you for that bit of theory, but in the real world it is the lack of an extension mechanism that is harming interoperability. Properties are colliding. Common solutions to common problems are not being shared.

> Just use “properties”, which is both free form (within the constraints of JSON), and is expected by all software that consumes GeoJSON. It is also hierarchical, so a “simplestyle” object would not “clutter up” “properties”, and would avoid key collisions.

Note that in the absence of guidance, this has not happened. The current state is a direct consequence of inaction on GeoJSON's part. Treating the properties object as a free-for-all benefits no one.

> TIFF (ie Adobe) anointed some image compressions (analogous to “geometry” in GeoJSON) as baseline and some, oxymoronically, as standard extensions. The effect is that all of these anointed compressions are baseline ones. Anyone wishing to use other compression algorithms is allowed to do so, but they can’t call it TIFF.

I am not aware of anyone who intends to replace GeoJSON geometry with something else[1]. The examples from Sean are much more benign - properties that could be useful (if they are done in a common way) but are by no means essential. Sean's five guidelines in combination with a key that defines the extensions in use (take them or leave them) would not destroy the sanctity of GeoJSON, they would help preserve it.

-Jeff

[1] For the time being, let's ignore TopoJSON which somehow manages to declare itself as an extension of GeoJSON even though it is not actually GeoJSON compliant.

---

From: tõnis kärdi <[email-masked]>
Date: Sat, 21 Nov 2015 21:18:53 +0200

Whoa. This discussion is becoming rather heated, so I'll add more fuel...

Using "properties" for the means of encoding any extension data will always
run the risk of key collisions. If they are grouped under a single key then
the probability will be lower but there's still a chance.

Introducing values into Feature.properties that are themselves objects will
already have the side-effect as "links: [object Object],[object Object]" in
https://github.com/opengeospatial/owscontext/blob/master/json/dev/trax/json/wfs_100entries.json
for I guess quite a many existing applications. Although, reading the spec
at a glance, it seems to be ok for a feature property to be an object:

"A feature object must have a member with the name "properties". The value
> of the properties member is an object (any JSON object or a JSON null
> value)."

"In GeoJSON, an object consists of a collection of name/value pairs -- also
> called members. For each member, the name is always a string. Member values
> are either a string, number, object, array or one of the literals: true,
> false, and null."

So are the existing implementations that tooltip with [object Object]
already now incompatible, or did I just misinterpret it?

"marker-width", "marker-color" and the rest used in simplestyle that can
used for styling features on a map. I think these are not in essence
feature "properties" as such (like tree height, crown radius, number of
leaves, diameter at breast height etc for a tree), but rather a description
of a particular way of rendering the feature data in a specific
application. It's basically mixing your data with metadata without really
keeping track what is what. It's rather carto-centric - in many cases
GeoJSON is used just for exchanging/storing data that might never see light
on a map but still be used for spatial analysis etc.

So the question is - when skimming through feature.properties and I get a
key called "color" what am i supposed to make of it? How am i supposed to
understand the meaning of it? As interoperability has to do with common
understanding then I think a mechanism for guaranteeing it is a good idea.
Especially since GeoJSON does not have self-describable data-model (and I'm
NOT saying that it should!)

Another thing, "properties" works for a feature member, but say I need to
encode something extra on FeatureCollection level, without having to repeat
the same thing over and over again with every feature. For example describe
style for a whole collection of features as uniformous.

Anyway. I think Sean has a point that having no extensibility is not an
option. The five points seem fine although I am a little in doubt about

> Implementations [..] *SHOULD* take sufficient care to ensure that
> extension properties used within the original document are preserved and
> serialized appropriately.

as to what that exactly means. If I require GeoJSON data from some source,
parse it, do something with it and then reserialize the result then the
result should preferably contain all source extensions? If that is the case
then it will be hard to achieve if i'm not directly using/processing
extension data aswell. E.g an implementation that takes in linestring
FeatureCollections and performs feature union operations over a list of
properties and then outputs the unioned FeatureCollection will not be able
to preserve all per feature extensions (like style etc) unless it
explicitly processes them along the way.

All the best,
Tõnis

2015-11-21 17:52 GMT+02:00 Martin Daly <[email-masked]>:
[...]
>
> Sean’s examples are not benign, and nor is the solution. It imposes
> behaviour on consuming and producing software (maintaining all unexpected
> keys at the FeatureCollection and Feature level). That directly affects
> existing implementations, which would be rendered incompatible by such a
> change.
>
>
>
> I fail to see why all implementations (there’s more to life than ogr2ogr
> you know) should have to change just because some people made obviously
> poor choices about where to put feature properties. There has always been a
> place to put properties: the clue is in the name.
>
>
>
> *From:* GeoJSON [mailto:[email-masked]] *On Behalf Of *Jeff
> Yutzler
>
>
>
> > Just use “properties”, which is both free form (within the constraints
> of JSON), and is expected by all software that consumes GeoJSON. It is also
> > TIFF (ie Adobe) anointed some image compressions (analogous to
> “geometry” in GeoJSON) as baseline and some, oxymoronically, as standard
> extensions. The effect is that all of these anointed compressions are
> baseline ones. Anyone wishing to use other compression algorithms is
> allowed to do so, but they can’t call it TIFF.
>
>
>

---

From: Martin Daly <[email-masked]>
Date: Sat, 21 Nov 2015 20:15:09 +0000

> Using "properties" for the means of encoding any extension data will always run the risk of key
> collisions. If they are grouped under a single key then the probability will be lower but there's still a chance.

I fail to see how moving data out of “properties” and into “feature” would help to avoid key clashes. All that’s different is the parent.

> So are the existing implementations that tooltip with [object Object] already now incompatible, or did I just misinterpret it?

XML/GML mechanically translated into JSON/GeoJSON is never going to be attractive to look at, so I'm going to pass. Sorry.

> "marker-width", "marker-color" and the rest used in simplestyle that can used for styling features on a map. 

The contents of simplestyle are not at issue. It is what it is; let’s not be distracted by it.

> So the question is - when skimming through feature.properties and I get a key called "color" what am i
> supposed to make of it? How am i supposed to understand the meaning of it? As interoperability has
> to do with common understanding then I think a mechanism for guaranteeing it is a good idea. Especially
> since GeoJSON does not have self-describable data-model (and I'm NOT saying that it should!)

What are you supposed to do with a key called “height”? Is it the height of a building footprint, the height of a cell tower, the height of a person, the height of a drone? What is it relative to?

Equally “color” [sic] could be a physical property of a vehicle that you are tracking (just as "crown radius" is a property of a tree), or it could be a hint about how to draw the feature.

They both have the same semantics without information from the supplier of the data: none. (If anyone mentions RDF I'll scream.)

> Another thing, "properties" works for a feature member, but say I need to encode something extra
> on FeatureCollection level, without having to repeat the same thing over and over again with every
> feature. For example describe style for a whole collection of features as uniformous.

I’d be in 100% in favour of an optional “properties” on a FeatureCollection. That is an excellent idea.

> Anyway. I think Sean has a point that having no extensibility is not an option. The five points seem
> fine although I am a little in doubt about Implementations [..] SHOULD take sufficient care to ensure
> that extension properties used within the original document are preserved and serialized appropriately.

Personally, I don’t see anyone saying “I can’t use GeoJSON because …” as a problem. Not at all. I don't see GeoJSON as the basis for a universal encoding of all possible geographic data, and I sincerely hope that no-one else does either. That way madness lies.

Martin

---

From: tõnis kärdi <[email-masked]>
Date: Mon, 23 Nov 2015 15:38:06 +0200

> The contents of simplestyle are not at issue. It is what it is; let’s not be distracted by it.

I'm not saying that simplestyle is an issue, I merely used it as an
example. For key collisions I was rather talking about collisions with
feature's (physical) property names which are not often up to me to
decide. If i dump my db table to GeoJSON and wanted to use some
default styling ops or pass on metadata, then i'd have to keep my
fingers crossed that feature property names will not collide with the
already defined keys. Or else enforce in my datasets that these and
these keys are "reserved" and cannot be used as column names.

> Equally “color” [sic] could be a physical property of a vehicle that you are tracking (just as "crown radius" is a property of a tree), or it could be a hint about how to draw the feature.

If "height" is in feature.properties then in the absence of a data
specifiaction I will assume it is the height of the feature that's
encoded. If the feature is a building then it's height-of-a-building.
If the feature is a tree then I will assume it is the height of the
tree. Just as with "color". But there's something I can be sure of -
it will not reference the height or color of the marker/icon/image
that's supposed to depict it. I have no problems with "default
rendering information" being present for a feature/featurecollection,
just please encode it somehow that I can discard it easily if i'm only
interested in a feature, it's geometry and physical properties.

> If anyone mentions RDF I'll scream.

+1 on that

> Personally, I don’t see anyone saying “I can’t use GeoJSON because …” as a problem. Not at all. I don't
> see GeoJSON as the basis for a universal encoding of all possible geographic data, and I sincerely
> hope that no-one else does either. That way madness lies.

But in turn, hasn't GeoJSON picked up so successfully _because_ it
basically makes little assertions as to what can be encoded so that in
fact it already has turned into a "a universal encoding of all
possible geographic data". The problem I see with it is that in the
absence of pointers of how to do things "right"
interoperability-issues get a hit: this is a thing that vaguely looks
like GeoJSON, smells like GeoJSON, but still there's something fishy
about it.

I think it was Martin who said that "you can do whatever, just don't
call it GeoJSON". I'm all for that and I see the "extension framework"
as a means for the data producer to explicitly declare "I'm doing a
whatever thing, although it might resemble GeoJSON. Tread carefully."

I really haven't thought this through but in it's simplest form - and
picking up on an optional FeatureColelction.properties - it could be
something in the lines of:

{
    "type":"Feature", # or "type":"FeatureCollection"
    "properties": {"this":"that", "foo":{"type":"Extension",
"bar":<whatever rocks your socks off>}}
    ...
}

if it was encoded in "properties". If one's really sadistic then they
could also have a geometry.properties and for example encode the
rendering color, size, fill-opacity; date-of-survey, GPS unit serial
number or whatever for every single vertex of a linestring or a
polygon. Just don't expect it to be processable auto-magically by
standard means. If you aim for interoperability, better design
decisions need to be made - maybe you can achieve the same thing
without the extension-frenzy.

Cheerio,
Tõnis

---

From: Sean Gillies <[email-masked]>
Date: Tue, 24 Nov 2015 14:31:12 -0700

On Sat, Nov 21, 2015 at 12:18 PM, tõnis kärdi <[email-masked]> wrote:

> Using "properties" for the means of encoding any extension data will
> always run the risk of key collisions. If they are grouped under a single
> key then the probability will be lower but there's still a chance.
>
> Introducing values into Feature.properties that are themselves objects
> will already have the side-effect as "links: [object Object],[object
> Object]" in
> "In GeoJSON, an object consists of a collection of name/value pairs --
>> also called members. For each member, the name is always a string. Member
>> values are either a string, number, object, array or one of the literals:
>> true, false, and null."
>
>

Tõnis, I would say that, yes, OWS Context GeoJSON and the practice of
printing out a table of all feature properties (as GitHub does) are
incompatible. Implementers don't know how to present such properties, but
don't feel free to hide them, either. It's not quite as bad a situation as
when an XML parser quits on encountering foreign markup, but it's not a
good situation.

> NOT saying that it should!)

Interop based on a common understanding of feature types (tree vs building,
&c) is beyond GeoJSON's scope. GeoJSON is about coarser interop based on a
common understanding of geometry, feature, and feature collection objects.

> style for a whole collection of features as uniformous.

Yes. A timestamp for a collection or a permalink to the internet resource
it represents or the query that produced it are my primary use cases. I do
not like the suggestion elsewhere in this thread to give feature
collections their own special "properties" object as it propagates the
can't-depict-can't-hide situation I described above.

> explicitly processes them along the way.

Calvin Metcalf asked a highly related question. Could we agree that
reserialization would mean writing out a feature with the same identity
("id") as the input? Go ahead and transform to your heart's content, but
don't assert that the result is the same thing as the input thing.

-- 
Sean Gillies