Subject: [Geojson] Extensibility of GeoJSON

From: Sean Gillies <[email-masked]>
Date: Fri, 20 Nov 2015 15:04:14 -0700

Hi all,

Consensus on and documentation of how to extend GeoJSON is part of our
group's charter. Let's get right to it.

>From my point of view, having no extensibility story is not an option.
GeoJSON is being extended already. Here are 5 examples off the top of my
head:

- https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md
- https://github.com/mapbox/carmen/blob/master/carmen-geojson.md
- https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md
- http://wiki.openstreetmap.org/wiki/Overpass_turbo/GeoJSON
- https://github.com/opengeospatial/owscontext

I see the simplestyle spec, in which styling directives are carried in a
Feature's "properties" object (akin to inline styles in HTML), as a product
of the original GeoJSON spec having no extensibility story. With no
consensus on whether new "style" or "class" objects were valid GeoJSON or
could expect support from other software, the simplestyle authors saw
mixing style and content as the only viable option. The result: feature
properties bloat and key collisions. If every application extends GeoJSON
by not extending it, by dumping everything into the "properties" object,
we'll have a mess.

As a starting point for this extensibility discussion, let's begin with
what's going on in Activity Streams 2.0:

http://www.w3.org/TR/activitystreams-core/#extensibility

Activity Streams 2.0 is based on JSON-LD and GeoJSON is not based on
JSON-LD. Still, there's plenty of common ground. I think the first
paragraph of AS 2's 6.0 translates straight to GeoJSON.

In Activity Streams 2.0, an "extension" is any property not defined by the
> Activity Vocabulary. Consuming implementations that encounter unfamiliar
> extensions MUST NOT stop processing or signal an error and MUST continue
> processing the items as if those properties were not present. Note that
> support for extensions can vary across implementations and no normative
> processing model for extensions is defined. Accordingly, implementations
> that rely too heavily on the use of extensions may experience reduced
> interoperability with other implementations.
>

Also I think section 6.1 on the re-serializations of extensions is
something we should adapt to GeoJSON.

Implementations that parse and then reserialize Activity Streams 2.0
> documents that contain extension properties *SHOULD* take sufficient care
> to ensure that extension properties used within the original document are
> preserved and serialized appropriately.
>

Related: the GDAL package and its ogr2ogr program will, in version 2.1,
preserve and re-serialize GeoJSON extensions. It's going to be a great
model for other implementations.

Section 6.2 in AS 2.0, about aggregation, is also relevant to GeoJSON.

Implementations that collect and aggregate Activity Streams objects from
> multiple sources may often need to deal with conflicting or incompatible
> extensions.
>

These three paragraphs seem to me to be a pretty good core for GeoJSON's
extensibility guidelines. On top of them, I'd like to express that GeoJSON
should not be extended in the following ways:

- The set of GeoJSON types ("FeatureCollection", "Feature", "Point",
"LineString", &c) should not be extended.

- The position construct at the heart of GeoJSON's multidimensional
geometry "coordinates" array (
https://tools.ietf.org/html/draft-ietf-geojson-00#section-2.1.1) should not
be extended beyond longitude, latitude, elevation, and linear referencing M.

Are there any objections to adding a new extensibility section to the draft
based on the 5 points above? Are there any extra considerations that would
need to be added?

-- 
Sean Gillies

---

From: Calvin Metcalf <[email-masked]>
Date: Fri, 20 Nov 2015 22:53:52 +0000

Thinking on the reserialization paragraph there could be some ambiguity
during transforms, say I got the centroid of a polygon and turned it into a
point but kept all the properties am I obligated to keep all the
extensions? I'd probably want to if they related to the feature, but not if
they related to the geometry (shape area would no longer make sense as an
extension, but a time value might). Maybe language that programs which
modify the geometry MAY just preserve the feature level extensions.

Also is measure as a 4th coordinate ever mentioned in the spec currently?
(a quick search doesn't find it) Do we really want to codify it into the
spec?

On Fri, Nov 20, 2015, 5:04 PM Sean

Gillis?

<[email-masked]> wrote:

Hi all,

- https
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>://
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>
github.com
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>/
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>
mapbox
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>/
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>
simplestyle-spec
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>
/blob/master/1.1.0/
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>
README.md
<https://github.com/mapbox/simplestyle-spec/blob/master/1.1.0/README.md>
- https <https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>://
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>github.com
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>/
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>mapbox
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>/
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>carmen
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>
/blob/master/
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>carmen-
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>geojson.md
<https://github.com/mapbox/carmen/blob/master/carmen-geojson.md>
- https
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>
://
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>
github.com
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>/
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>
geocoders
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>/
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>
geocodejson
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>
-spec
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>
/blob/master/draft/
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>
README.md
<https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md>
- http:// <http://wiki.openstreetmap.org/wiki/Overpass_turbo/GeoJSON>
wiki.openstreetmap.org
<http://wiki.openstreetmap.org/wiki/Overpass_turbo/GeoJSON>
/wiki/Overpass_turbo/
<http://wiki.openstreetmap.org/wiki/Overpass_turbo/GeoJSON>GeoJSON
<http://wiki.openstreetmap.org/wiki/Overpass_turbo/GeoJSON>
- https <https://github.com/opengeospatial/owscontext>://
<https://github.com/opengeospatial/owscontext>github.com
<https://github.com/opengeospatial/owscontext>/
<https://github.com/opengeospatial/owscontext>opengeospatial
<https://github.com/opengeospatial/owscontext>/
<https://github.com/opengeospatial/owscontext>owscontext
<https://github.com/opengeospatial/owscontext>

http:// <http://www.w3.org/TR/activitystreams-core/#extensibility>www.w3.org
<http://www.w3.org/TR/activitystreams-core/#extensibility>/TR/
<http://www.w3.org/TR/activitystreams-core/#extensibility>
activitystreams-core
<http://www.w3.org/TR/activitystreams-core/#extensibility>/#extensibility
<http://www.w3.org/TR/activitystreams-core/#extensibility>

- The position construct at the heart of GeoJSON's multidimensional
geometry "coordinates" array (https
<https://tools.ietf.org/html/draft-ietf-geojson-00#section-2.1.1>://
<https://tools.ietf.org/html/draft-ietf-geojson-00#section-2.1.1>
tools.ietf.org
<https://tools.ietf.org/html/draft-ietf-geojson-00#section-2.1.1>/html/
<https://tools.ietf.org/html/draft-ietf-geojson-00#section-2.1.1>
draft-ietf-geojson-00#section-2.1.1
<https://tools.ietf.org/html/draft-ietf-geojson-00#section-2.1.1>) should
not be extended beyond longitude, latitude, elevation, and linear
referencing M.

-- 
Sean Gillies

_______________________________________________
GeoJSON mailing list
[email-masked]
https <https://www.ietf.org/mailman/listinfo/geojson>://
<https://www.ietf.org/mailman/listinfo/geojson>www.ietf.org
<https://www.ietf.org/mailman/listinfo/geojson>/mailman/
<https://www.ietf.org/mailman/listinfo/geojson>listinfo
<https://www.ietf.org/mailman/listinfo/geojson>/
<https://www.ietf.org/mailman/listinfo/geojson>geojson
<https://www.ietf.org/mailman/listinfo/geojson>

---

From: Stefan Hagen <[email-masked]>
Date: Sat, 21 Nov 2015 08:58:11 +0100

Great Suggestion I think. Let's go for it!

--
Stefan.

> Am 20.11.2015 um 23:04 schrieb Sean Gillies <[email-masked]>:
> 
> Hi all,
> 
> Consensus on and documentation of how to extend GeoJSON is part of our group's charter. Let's get right to it.
> 
> From my point of view, having no extensibility story is not an option. GeoJSON is being extended already. Here are 5 examples off the top of my head:
> 
> I see the simplestyle spec, in which styling directives are carried in a Feature's "properties" object (akin to inline styles in HTML), as a product of the original GeoJSON spec having no extensibility story. With no consensus on whether new "style" or "class" objects were valid GeoJSON or could expect support from other software, the simplestyle authors saw mixing style and content as the only viable option. The result: feature properties bloat and key collisions. If every application extends GeoJSON by not extending it, by dumping everything into the "properties" object, we'll have a mess.
> 
> As a starting point for this extensibility discussion, let's begin with what's going on in Activity Streams 2.0:
> 
> Activity Streams 2.0 is based on JSON-LD and GeoJSON is not based on JSON-LD. Still, there's plenty of common ground. I think the first paragraph of AS 2's 6.0 translates straight to GeoJSON.
> 
>> In Activity Streams 2.0, an "extension" is any property not defined by the Activity Vocabulary. Consuming implementations that encounter unfamiliar extensions MUST NOT stop processing or signal an error and MUST continue processing the items as if those properties were not present. Note that support for extensions can vary across implementations and no normative processing model for extensions is defined. Accordingly, implementations that rely too heavily on the use of extensions may experience reduced interoperability with other implementations.
> 
> Also I think section 6.1 on the re-serializations of extensions is something we should adapt to GeoJSON.
> 
>> Implementations that parse and then reserialize Activity Streams 2.0 documents that contain extension properties SHOULD take sufficient care to ensure that extension properties used within the original document are preserved and serialized appropriately.
> 
> Related: the GDAL package and its ogr2ogr program will, in version 2.1, preserve and re-serialize GeoJSON extensions. It's going to be a great model for other implementations. 
> 
>> Implementations that collect and aggregate Activity Streams objects from multiple sources may often need to deal with conflicting or incompatible extensions.
> 
> These three paragraphs seem to me to be a pretty good core for GeoJSON's extensibility guidelines. On top of them, I'd like to express that GeoJSON should not be extended in the following ways:
> 
> - The set of GeoJSON types ("FeatureCollection", "Feature", "Point", "LineString", &c) should not be extended.
> 
> - The position construct at the heart of GeoJSON's multidimensional geometry "coordinates" array (https://tools.ietf.org/html/draft-ietf-geojson-00#section-2.1.1) should not be extended beyond longitude, latitude, elevation, and linear referencing M.
> 
> Are there any objections to adding a new extensibility section to the draft based on the 5 points above? Are there any extra considerations that would need to be added? 
> 
> -- 
> Sean Gillies
> https://www.ietf.org/mailman/listinfo/geojson

---

From: Martin Daly <[email-masked]>
Date: Sat, 21 Nov 2015 13:55:30 +0000

Interoperability is inversely proportional to extensibility, so I am implacably opposed to this going too far (which it will).

Just use “properties”, which is both free form (within the constraints of JSON), and is expected by all software that consumes GeoJSON. It is also hierarchical, so a “simplestyle” object would not “clutter up” “properties”, and would avoid key collisions.

Domains of interest that want to do something else are welcome to borrow the “type” and ”geometry” constructs, and to build their own domain-specific JSON around them, just don’t pretend that it is GeoJSON. I’m all for that. However, no-one’s favourite extension is more important than the core standard.

TIFF, for example, has tags, which are analogous to GeoJSON “properties”. TIFF consuming software might know about special tags, eg GeoTIFF, or EXIF, or it might not, but anything unexpected can still be parsed, ignored, but presented to the user as a companion to the image (which is always standardised).

TIFF (ie Adobe) anointed some image compressions (analogous to “geometry” in GeoJSON) as baseline and some, oxymoronically, as standard extensions. The effect is that all of these anointed compressions are baseline ones. Anyone wishing to use other compression algorithms is allowed to do so, but they can’t call it TIFF.

Page 9 of TIFF Baseline 6.0 (http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf):

“[Check] the Adobe WWW home page (http://www.adobe.com) for future developements) will contain proposed TIFF extensions from other companies that are not approved by Adobe as part of Baseline TIFF.

These proposals typically represent specialized uses of TIFF that do not fall within the domain of publishing or general graphics or picture interchange. Generally, these features will not be widely supported. If you do write files that incorporate these extensions, be sure to either not call them TIFF files or mark them in some way so that they will not be confused with mainstream TIFF files.”

TIFF works, and has done for nearly 30 years. Let’s learn from that, and not from the kind of mindset that proposes “Active and Healthy Ageing Mark-up Language” (and that is not something that I made up), because that, dear reader, is the future that extensions will inevitably bring.

Martin

From: GeoJSON [mailto:[email-masked]] On Behalf Of Sean Gillies
Sent: 20 November 2015 22:04
To: [email-masked]
Subject: [Geojson] Extensibility of GeoJSON

Hi all,

--
Sean Gillies

---

From: Erik Wilde <[email-masked]>
Date: Sun, 22 Nov 2015 09:42:37 +0100

hello sean.

thanks for the initiative, and i am sure that this is both one of the 
most important and most contentious issues to get consensus on. the 
thread so far already has shown that.

On 2015-11-20 23:04, Sean Gillies wrote:
> Are there any objections to adding a new extensibility section to the
> draft based on the 5 points above? Are there any extra considerations
> that would need to be added?

the one thing i am missing that i think is really important is clear 
language that extensions cannot change the semantics of any of the 
standard fields. i'll send separate feedback on the the suggestions you 
have made, but i think this is where often extensibility gets "misused" 
and starts to create real interop problems, when people invent anything 
that changes the meaning of things that should be stable concepts.

cheers,

dret.

-- 
erik wilde | mailto:[email-masked] |
            | http://dret.net/netdret    |
            | http://twitter.com/dret    |

---

From: Erik Wilde <[email-masked]>
Date: Sun, 22 Nov 2015 16:54:50 +0100

> Also is measure as a 4th coordinate ever mentioned in the spec
> currently? (a quick search doesn't find it) Do we really want to codify
> it into the spec?

what's mentioned is that there can be more numbers following the 
(optional) elevation. it's not mentioned why there is the "numbers only" 
constraint, and what the historical reason is. this issue has been 
recorded already:

https://github.com/geojson/draft-geojson/issues/57

cheers,

dret.

---

From: Martin Daly <[email-masked]>
Date: Mon, 23 Nov 2015 10:51:05 +0000

> thanks for the initiative, and i am sure that this is both one of the most
> important and most contentious issues to get consensus on. the thread so far
> already has shown that.

Sorry, but you cannot assert that an extension mechanism is "important" just like that. It may be to you, but you are only one of many. You, and any others, have to make a case for its importance, not just state your opinion of that importance as fact.

Martin

---

From: Erik Wilde <[email-masked]>
Date: Mon, 23 Nov 2015 12:01:21 +0100

> Sorry, but you cannot assert that an extension mechanism is "important" just like that. It may be to you, but you are only one of many. You, and any others, have to make a case for its importance,…

all i (and https://github.com/geojson/draft-geojson/issues/59) am saying 
is that the extensibility language probably could use some improvement, 
because it is not very clear. i am not making any claims about how, 
that's for the group to decide. but the fact is that people are and will 
be extending GeoJSON, and it would be an improvement for the spec to be 
more clear about if and how that can/should be done. i think that's what 
sean's initial suggestions in this thread were all about.

cheers,

dret.

---

From: Carl Reed <[email-masked]>
Date: Mon, 23 Nov 2015 09:17:25 -0700

Any thought on considering how other groups have defined rules/guidance for
extensions?

For example, JSON RPC Spec states:

Method names that begin with rpc. are reserved for system extensions, and
MUST NOT be used for anything else. Each system extension is defined in a
related specification. All system extensions are OPTIONAL.

And then there is additional guidance on the rules for an extension.

Anything useful here?

http://jsonapi.org/extensions/

Or would checking the KML extension model have any value?

http://docs.opengeospatial.org/is/12-007r2/12-007r2.html#25

On Fri, Nov 20, 2015 at 3:04 PM, Sean Gillies <[email-masked]>
wrote:

> -
> https://github.com/geocoders/geocodejson-spec/blob/master/draft/README.md
>> documents that contain extension properties *SHOULD* take sufficient
>> care to ensure that extension properties used within the original document
>> are preserved and serialized appropriately.
>>
>
> https://tools.ietf.org/html/draft-ietf-geojson-00#section-2.1.1) should
> draft based on the 5 points above? Are there any extra considerations that
> would need to be added?
>
> --
> Sean Gillies
>

-- 
Carl Reed, PhD
Carl Reed and Associates

Mobile: 970-402-0284

---

From: Sean Gillies <[email-masked]>
Date: Tue, 24 Nov 2015 13:14:55 -0700

On Sun, Nov 22, 2015 at 1:42 AM, Erik Wilde <[email-masked]> wrote:

> important and most contentious issues to get consensus on. the thread so
> far already has shown that.
>
> language that extensions cannot change the semantics of any of the standard
> fields. i'll send separate feedback on the the suggestions you have made,
> but i think this is where often extensibility gets "misused" and starts to
> create real interop problems, when people invent anything that changes the
> meaning of things that should be stable concepts.
>
> cheers,
>
> dret.
>

I agree about not letting extensions change semantics and would be happy to
include such language with whatever specific examples we find.

The complement to this is to nail down the existing semantics more firmly.

-- 
Sean Gillies

---

From: Sean Gillies <[email-masked]>
Date: Tue, 24 Nov 2015 14:42:49 -0700

> (a quick search doesn't find it) Do we really want to codify it into the
Good catch. No, let's don't mention M in the spec.

Tõnis also asked about reserialization and I took a stab at it in a
different email. Needs some work as it hinges on an under-defined concept
of feature identity :(

-- 
Sean Gillies

---

From: Sean Gillies <[email-masked]>
Date: Tue, 24 Nov 2015 16:04:14 -0700

Hi Martin,

On Sat, Nov 21, 2015 at 6:55 AM, Martin Daly <[email-masked]>
wrote:
>
> Interoperability is inversely proportional to extensibility, so I am
implacably opposed to this going too far (which it will).

I am also opposed to this going too far.

Let me remind you and everyone else that the GeoJSON spec we published in
2008 left room for extensions but didn't provide any guidelines about how
to make good extensions. The first item in the list under section 2 (
http://geojson.org/geojson-spec.html#geojson-objects) is:

   - The GeoJSON object may have any number of members (name/value pairs).

I'm not going to call the lack of guidelines a failure. Eight years ago, we
simply didn't know what extensibility of a JSON format looked like. But we
did leave the door open for future developments and extensions by stating
that GeoJSON objects may have any number of members and still be GeoJSON.
We did this deliberately. The draft adopted by this WG (
https://tools.ietf.org/html/draft-ietf-geojson-00#section-2) says:

   - The GeoJSON object MAY have any number of other members.
   Implementations MUST ignore unrecognized members.

The JSON doc at the top of
https://raw.githubusercontent.com/mapbox/carmen/master/carmen-geojson.md
has a "query" property not described in the GeoJSON format spec (see
https://github.com/mapbox/carmen/blame/master/carmen-geojson.md#L7) and is
still 100% comforming GeoJSON. This is where we are today. Extensions
exist. In my opinion, the fewer of them, the better. I don't see how we'll
achieve a stable state without pinning down GeoJSON's extensibility story.

>
>
>
> Just use “properties”, which is both free form (within the constraints of
JSON), and is expected by all software that consumes GeoJSON. It is also
hierarchical, so a “simplestyle” object would not “clutter up”
“properties”, and would avoid key collisions.

This is one possible extension guideline. I don't like it as much as the
option to squirrel application-specific attributes elsewhere, but as the
spec is written implementers can do either or both.

>
>
>
> Domains of interest that want to do something else are welcome to borrow
the “type” and ”geometry” constructs, and to build their own
domain-specific JSON around them, just don’t pretend that it is GeoJSON.
I’m all for that. However, no-one’s favourite extension is more important
than the core standard.

I strongly disagree that applications should be able to borrow "type" and
"geometry" and intend to strictly rule out extensions that add new types,
change the hierarchy of types, or extend geometry coordinates.

>
>
>
> TIFF, for example, has tags, which are analogous to GeoJSON “properties”.
TIFF consuming software might know about special tags, eg GeoTIFF, or EXIF,
or it might not, but anything unexpected can still be parsed, ignored, but
presented to the user as a companion to the image (which is always
standardised).
>
>
>
> TIFF (ie Adobe) anointed some image compressions (analogous to “geometry”
in GeoJSON) as baseline and some, oxymoronically, as standard extensions.
The effect is that all of these anointed compressions are baseline ones.
Anyone wishing to use other compression algorithms is allowed to do so, but
they can’t call it TIFF.
>
>
>
> Page 9 of TIFF Baseline 6.0 (
http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf):
>
>
>
> “[Check] the Adobe WWW home page (http://www.adobe.com) for future
developements) will contain proposed TIFF extensions from other companies
that are not approved by Adobe as part of Baseline TIFF.
>
>
>
> These proposals typically represent specialized uses of TIFF that do not
fall within the domain of publishing or general graphics or picture
interchange. Generally, these features will not be widely supported. If you
do write files that incorporate these extensions, be sure to either not
call them TIFF files or mark them in some way so that they will not be
confused with mainstream TIFF files.”
>
>
>
> TIFF works, and has done for nearly 30 years. Let’s learn from that, and
not from the kind of mindset that proposes “Active and Healthy Ageing
Mark-up Language” (and that is not something that I made up), because that,
dear reader, is the future that extensions will inevitably bring.

I'm not sure if TIFF is such a great analogy. Baseline TIFF includes
optional sub-images, right? Aren't GeoJSON extensions just like sub-images?
Analogies are terrible, just like war.

Ultimately, it seems to me that if you want to make GeoJSON non-extendable
you're going to have to suggest an edit to the definition of a "GeoJSON
object" that explicitly prohibits members other than those in the core
GeoJSON vocabulary. Instead, let's write some sound guidelines to reduce
the number of pointless and harmful extensions.

Yours,

--
Sean Gillies

---

From: Martin Daly <[email-masked]>
Date: Wed, 25 Nov 2015 11:41:59 +0000

> The first item in the list under section 2 (http://geojson.org/geojson-spec.html#geojson-objects) is:
> The GeoJSON object may have any number of members (name/value pairs).
> ...
> The draft adopted by this WG (https://tools.ietf.org/html/draft-ietf-geojson-00#section-2) says:
> The GeoJSON object MAY have any number of other members. Implementations MUST ignore unrecognized members.

The WG draft wording is precisely my point. If you want interoperability, use "properties"; if you don't care, use other members. That is, interoperability  = 1 / extensibility. QED.

The suggested change (the wording from "activitystreams") shifts the burden on the implementer from 0 - "MUST ignore" - right up to 11 - "should take sufficient care to ensure that extension properties used within the original document are preserved and reserialized appropriately".

What is "sufficient"? How do I measure that? Preserved for how long? A process? A thread? An XMLHttpRequest? A day? A week? A year? Forever? And what does "appropriately" mean? Exactly as they were read, or without losing data, or what? Again, what is the measure of "appropriateness"?

ogr2ogr is only a great model for ogr2ogr. Just as we can't get distracted by the details of simplestyle, we shouldn't get distracted by the details of ogr2ogr.

> I don't see how we'll achieve a stable state without pinning down GeoJSON's extensibility story.

We have a stable state already, and the RFC makes it more stable: people can either choose interoperability, or extensibility. I still have not seen a sensible, reasonable, proportionate way to get both. The "preserve and reserialize" wording certainly isn't it.

> This is one possible extension guideline. I don't like it as much as the option to squirrel application-specific attributes
> elsewhere, but as the spec is written implementers can do either or both.

It's fine that they can do both. But there are consequences of not using "properties". If, to date, people did not realise those consequences, then that is a pity, and we should have been more precise, or strict, or whatever, in making those consequences clear. The fact that we weren't does not mean that we should row back and embrace all those wrong design decisions made elsewhere. Those specs can either be revised to match the new normal or carry on regardless.

> I strongly disagree that applications should be able to borrow "type" and "geometry"

But it is OK for GeoJSON-LD to borrow OGC WKT? We can't possibly stop people doing it (cf the Twitter API), but should be clear that if they do, then what they produce is not GeoJSON.

> and intend to strictly rule out extensions that add new types, change the hierarchy of types, or extend geometry coordinates.

Excellent.

> I'm not sure if TIFF is such a great analogy. ... Analogies are terrible, just like war.

Analogies are never great: they are only ever "like", not "equal to". As in, the Turkish air force shooting down a Russian fighter jet is like Gavrilo Princip shooting Archduke Franz Ferdinand, which is wrong because WW3 won't be starting soon, right? RIGHT?

> Ultimately, it seems to me that if you want to make GeoJSON non-extendable you're going to have to
> suggest an edit to the definition of a "GeoJSON object" that explicitly prohibits members other than
> those in the core GeoJSON vocabulary. Instead, let's write some sound guidelines to reduce the
> number of pointless and harmful extensions.

No extra "non-extendable" wording is required: the "MUST ignore" already says it all. The only sound guidelines that are needed are: if you use "properties" you maximise interoperability; if you add other members you will lose interoperability (of those other members).

Martin

---

From: Allan Doyle <[email-masked]>
Date: Wed, 25 Nov 2015 09:37:58 -0500

I’m having trouble figuring out how to jump into this email thread because I’m not sure I want to do a point-by-point response to any one email.

My thoughts are this:

1. Let’s make sure that the spec itself is tight enough that there’s no question whether any instance of claimed GeoJSON is or is not GeoJSON.

2. Because the spec document is about a *format* and not a protocol, I don’t see any need to specify behaviors of consumers or producers. If I look at a sequence of bits claiming to be GeoJSON, is that a valid claim or not (see #1)? In case you can’t tell, I’d come down against the “MUST ignore” bit in the current version but I’m not going to worry too much about it.

3. If a process wants to stop processing when it sees something it doesn’t like or if a process wants to consume and transform GeoJSON but not preserve the extended parts, declaring that to be good or bad is not in scope for this spec.

4. If people want to talk about process behavior, that should definitely not be normative.

Now for the extensibility/interoperability part.

Our job is to ensure interoperability of the core GeoJSON. We can’t ensure interoperability of extensions that we don’t specify. We can write rules and suggestions until we’re blue in the face and our fingers wear out, but in the end, people are going to extend GeoJSON in ways that make sense for them that we may not have even thought about.

The spec already says there can’t be other types. It allows people to use a “properties” element. If the group thinks that all extensions must go into “properties”, that sounds wrong, particularly since that would instantly invalidate existing extensions.

If people want to provide suggestions on how to sanely extend GeoJSON, maybe that’s fine, but that should go into an extension section and that section should be informative. 

	Allan

> On Nov 25, 2015, at 6:41 AM, Martin Daly <[email-masked]> wrote:
> 

-- 
Allan Doyle
[email-masked]
+1.617.417.8911 (Cell)

---

From: Erik Wilde <[email-masked]>
Date: Wed, 25 Nov 2015 16:09:43 +0100

On 2015-11-24 21:14, Sean Gillies wrote:
> On Sun, Nov 22, 2015 at 1:42 AM, Erik Wilde <[email-masked]
> <mailto:[email-masked]>> wrote:
>     standard fields. i'll send separate feedback on the the suggestions
>     you have made, but i think this is where often extensibility gets
>     "misused" and starts to create real interop problems, when people
>     invent anything that changes the meaning of things that should be
>     stable concepts.
> I agree about not letting extensions change semantics and would be happy
> to include such language with whatever specific examples we find.

maybe that's almost the most important thing to make very clear: that no 
matter how exactly extensibility is defined and/or constrained, 
extensions MUST NOT change the semantics of standard fields.

btw, that's how RFC 6906 profiles are defined as well: you must make 
sure that somebody not understanding your profile can safely interpret a 
resource according to the fundamental semantics of the media type, 
completely ignoring the profile and whatever it added.

http://dret.typepad.com/dretblog/2013/03/on-profiles.html

cheers,

dret.

---

From: Allan Doyle <[email-masked]>
Date: Wed, 25 Nov 2015 10:18:44 -0500

I’m totally +1 on this.

> On Nov 25, 2015, at 10:09 AM, Erik Wilde <[email-masked]> wrote:
> 
> maybe that's almost the most important thing to make very clear: that no matter how exactly extensibility is defined and/or constrained, extensions MUST NOT change the semantics of standard fields.
> 
> btw, that's how RFC 6906 profiles are defined as well: you must make sure that somebody not understanding your profile can safely interpret a resource according to the fundamental semantics of the media type, completely ignoring the profile and whatever it added.
>

---

From: Carl Reed <[email-masked]>
Date: Wed, 25 Nov 2015 09:47:02 -0700

Allen -

WRT to comments on extensions, Thanks!

This is why I provided a reference to the KML 2.3 extension mechanism
clause. There is an understanding that the KML standard cannot control who
does extensions for solving what problem (they tend to be domain specific).
KML specifies how (rules) extensions can be done so that they are developed
and deployed in a consistent manner.

Carl

On Wed, Nov 25, 2015 at 7:37 AM, Allan Doyle <[email-masked]>
wrote:

> I’m having trouble figuring out how to jump into this email thread because
> I’m not sure I want to do a point-by-point response to any one email.
>
> My thoughts are this:
>
> 1. Let’s make sure that the spec itself is tight enough that there’s no
> question whether any instance of claimed GeoJSON is or is not GeoJSON.
>
> 2. Because the spec document is about a *format* and not a protocol, I
> don’t see any need to specify behaviors of consumers or producers. If I
> look at a sequence of bits claiming to be GeoJSON, is that a valid claim or
> not (see #1)? In case you can’t tell, I’d come down against the “MUST
> ignore” bit in the current version but I’m not going to worry too much
> about it.
>
> 3. If a process wants to stop processing when it sees something it doesn’t
> like or if a process wants to consume and transform GeoJSON but not
> preserve the extended parts, declaring that to be good or bad is not in
> scope for this spec.
>
> Our job is to ensure interoperability of the core GeoJSON. We can’t ensure
> interoperability of extensions that we don’t specify. We can write rules
> and suggestions until we’re blue in the face and our fingers wear out, but
> in the end, people are going to extend GeoJSON in ways that make sense for
> them that we may not have even thought about.
>
> The spec already says there can’t be other types. It allows people to use
> a “properties” element. If the group thinks that all extensions must go
> into “properties”, that sounds wrong, particularly since that would
> instantly invalidate existing extensions.
>
> If people want to provide suggestions on how to sanely extend GeoJSON,
> maybe that’s fine, but that should go into an extension section and that
> section should be informative.
>
>         Allan
>
> >> The first item in the list under section 2 (
> >> The draft adopted by this WG (
> >> The GeoJSON object MAY have any number of other members.
> > The WG draft wording is precisely my point. If you want
> interoperability, use "properties"; if you don't care, use other members.
> That is, interoperability  = 1 / extensibility. QED.
> >
> > The suggested change (the wording from "activitystreams") shifts the
> burden on the implementer from 0 - "MUST ignore" - right up to 11 - "should
> take sufficient care to ensure that extension properties used within the
> original document are preserved and reserialized appropriately".
> >
> > What is "sufficient"? How do I measure that? Preserved for how long? A
> process? A thread? An XMLHttpRequest? A day? A week? A year? Forever? And
> what does "appropriately" mean? Exactly as they were read, or without
> losing data, or what? Again, what is the measure of "appropriateness"?
> >
> > ogr2ogr is only a great model for ogr2ogr. Just as we can't get
> distracted by the details of simplestyle, we shouldn't get distracted by
> the details of ogr2ogr.
> >
> > We have a stable state already, and the RFC makes it more stable: people
> can either choose interoperability, or extensibility. I still have not seen
> a sensible, reasonable, proportionate way to get both. The "preserve and
> reserialize" wording certainly isn't it.
> >
> >> This is one possible extension guideline. I don't like it as much as
> the option to squirrel application-specific attributes
> > It's fine that they can do both. But there are consequences of not using
> "properties". If, to date, people did not realise those consequences, then
> that is a pity, and we should have been more precise, or strict, or
> whatever, in making those consequences clear. The fact that we weren't does
> not mean that we should row back and embrace all those wrong design
> decisions made elsewhere. Those specs can either be revised to match the
> new normal or carry on regardless.
> >
> > But it is OK for GeoJSON-LD to borrow OGC WKT? We can't possibly stop
> people doing it (cf the Twitter API), but should be clear that if they do,
> then what they produce is not GeoJSON.
> >
> >> and intend to strictly rule out extensions that add new types, change
> the hierarchy of types, or extend geometry coordinates.
> >
> > Excellent.
> >
> > Analogies are never great: they are only ever "like", not "equal to". As
> in, the Turkish air force shooting down a Russian fighter jet is like
> Gavrilo Princip shooting Archduke Franz Ferdinand, which is wrong because
> WW3 won't be starting soon, right? RIGHT?
> >
> > No extra "non-extendable" wording is required: the "MUST ignore" already
> says it all. The only sound guidelines that are needed are: if you use
> "properties" you maximise interoperability; if you add other members you
> will lose interoperability (of those other members).
> >
> > Martin
> >

-- 
Carl Reed, PhD
Carl Reed and Associates

Mobile: 970-402-0284

---

From: Martin Daly <[email-masked]>
Date: Wed, 25 Nov 2015 16:50:37 +0000

Carl,

KML feels like a bad reference because there is (realistically) only one client. As long as you target GE, you are safe. The same is not true of GeoJSON.

Martin

From: GeoJSON [mailto:[email-masked]] On Behalf Of Carl Reed
Sent: 25 November 2015 16:47
To: Allan Doyle <[email-masked]>
Cc: [email-masked]
Subject: Re: [Geojson] Extensibility of GeoJSON

Allen -

This is why I provided a reference to the KML 2.3 extension mechanism clause. There is an understanding that the KML standard cannot control who does extensions for solving what problem (they tend to be domain specific). KML specifies how (rules) extensions can be done so that they are developed and deployed in a consistent manner.

Carl

On Wed, Nov 25, 2015 at 7:37 AM, Allan Doyle <[email-masked]<mailto:[email-masked]>> wrote:
I’m having trouble figuring out how to jump into this email thread because I’m not sure I want to do a point-by-point response to any one email.

My thoughts are this:

        Allan

> On Nov 25, 2015, at 6:41 AM, Martin Daly <[email-masked]<mailto:[email-masked]>> wrote:
>
> [email-masked]<mailto:[email-masked]>

--
Allan Doyle
[email-masked]<mailto:[email-masked]>
+1.617.417.8911<tel:%2B1.617.417.8911> (Cell)

_______________________________________________
GeoJSON mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/geojson

--
Carl Reed, PhD
Carl Reed and Associates

Mobile: 970-402-0284

---

From: Sean Gillies <[email-masked]>
Date: Wed, 25 Nov 2015 12:34:08 -0700

Allan, all:

Here's my outline of an extensibility section:
https://github.com/geojson/draft-geojson/pull/117.

One thing it does not cover in its present form: profiles.

Martin, what do you object to specifically in that section? Do you just
object to the "e word" on general principles? I don't think that's grounds
for keeping this section out of the draft. Extensibility is a thing in the
world and good stewardship of the format means to me that we take care to
disallow extensions that harm the format, whether that be changing the
semantics of geometric objects or changing the sense and order of
coordinates. It also means to me that we make benign extensions possible so
that entirely new formats not be created needlessly.

On Wed, Nov 25, 2015 at 8:18 AM, Allan Doyle <[email-masked]>
wrote:

> matter how exactly extensibility is defined and/or constrained, extensions
> MUST NOT change the semantics of standard fields.
> >

-- 
Sean Gillies

---

From: Allan Doyle <[email-masked]>
Date: Wed, 25 Nov 2015 14:35:42 -0500

I have a comment up there already…

> On Nov 25, 2015, at 2:34 PM, Sean Gillies <[email-masked]> wrote:
> 
> Allan, all:
> 
> Here's my outline of an extensibility section: https://github.com/geojson/draft-geojson/pull/117 <https://github.com/geojson/draft-geojson/pull/117>.
> 
> Martin, what do you object to specifically in that section? Do you just object to the "e word" on general principles? I don't think that's grounds for keeping this section out of the draft. Extensibility is a thing in the world and good stewardship of the format means to me that we take care to disallow extensions that harm the format, whether that be changing the semantics of geometric objects or changing the sense and order of coordinates. It also means to me that we make benign extensions possible so that entirely new formats not be created needlessly.
> 
> On Wed, Nov 25, 2015 at 8:18 AM, Allan Doyle <[email-masked] <mailto:[email-masked]>> wrote:
> I’m totally +1 on this.
> 
> > On Nov 25, 2015, at 10:09 AM, Erik Wilde <[email-masked] <mailto:[email-masked]>> wrote:
> >
> >> On Sun, Nov 22, 2015 at 1:42 AM, Erik Wilde <[email-masked] <mailto:[email-masked]>
> >> <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
> > http://dret.typepad.com/dretblog/2013/03/on-profiles.html <http://dret.typepad.com/dretblog/2013/03/on-profiles.html>
> >
> > cheers,
> >
> > dret.
> >
> 
> -- 
> Sean Gillies

---

From: Carl Reed <[email-masked]>
Date: Wed, 25 Nov 2015 13:13:33 -0700

Playing the standards geek -

Profiles are restricted subsets of a standard. Period. No extensions. Thus,
a GeoJSON profile would specify rules as to how to use a subset of the
existing GeoJSON standard. This definition is true in the W3C, ISO, OGC,
OASIS, and so on. OGC GML Simple Features is a profile. In these same
standards organizations, extensions assume adherence to the "core" (in our
case, GeoJSON) with no changes to the core but additional
properties/elements are specified to create an extension. In OGC land,
CityGML has a core which is adhered to and extensions to deal with textures
and other properties required by rendering engines.

Regards

Carl

On Wed, Nov 25, 2015 at 8:09 AM, Erik Wilde <[email-masked]> wrote:

> btw, that's how RFC 6906 profiles are defined as well: you must make sure
> that somebody not understanding your profile can safely interpret a

-- 
Carl Reed, PhD
Carl Reed and Associates

Mobile: 970-402-0284

---

From: Erik Wilde <[email-masked]>
Date: Wed, 25 Nov 2015 21:29:29 +0100

On 2015-11-25 21:13, Carl Reed wrote:
> Profiles are restricted subsets of a standard. Period. No extensions.
> Thus, a GeoJSON profile would specify rules as to how to use a subset of
> the existing GeoJSON standard.

i don't want to spend time fighting about words. i understand where 
you're coming from (i am standards geek myself) and see your point. but 
RFC 6906 profiles are a slightly different kind of thing, since they do 
allow "extensions" (as long, and that's essential, as the extensions 
fall into the realm of what a standard allows in its extension model).

so they may be a bit different from how you define "profiles". this 
might mean you're unhappy with the terminology, but that's not something 
that we should discuss. RFC 6906 just talks about "profiles", and 
assumes that those can use extensions.

but: RFC 6906 also says that any profile cannot use extensions that 
break the standard; they have to be defined in a way so that they work 
withing the scope of the standard, and do not break anybody who's not 
supporting the profile. so maybe the RFC 6906 profiles are not as far 
from the profiles that you know after all.

cheers,

dret.

---

From: Martin Daly <[email-masked]>
Date: Wed, 25 Nov 2015 20:38:35 +0000

> Here's my outline of an extensibility section: https://github.com/geojson/draft-geojson/pull/117.

I've added a comment and an alternative, snappier, outline.

> Martin, what do you object to specifically in that section? Do you just object to the "e word" on general principles?
> I don't think that's grounds for keeping this section out of the draft. Extensibility is a thing in the world and good
> stewardship of the format means to me that we take care to disallow extensions that harm the format, whether
> that be changing the semantics of geometric objects or changing the sense and order of coordinates. It also
> means to me that we make benign extensions possible so that entirely new formats not be created needlessly.

The "e word", as it shall henceforth be known, confers a kind of respectability that a few extra object members (as your and my suggested wordings both allow, and no more) scarcely deserve. Who is going to police whether or not an "e word" follows the rules? What happens if an "e word" doesn't follow the rules? I believe that anything more than additional members, as suggested elsewhere, will inevitably be necessarily complex, and therefore against one of the fundamental tenets of GeoJSON since the very beginning, when I was still a young man.

I also believe that the "maintain unknown members" is only practical in the very simplest of scenarios, and its reputation been traduced by Allan much more eloquently than I could manage.

Martin

---

From: Carl Reed <[email-masked]>
Date: Wed, 25 Nov 2015 13:43:41 -0700

Erik -

Neither do I :-)

I think we are saying the same thing - just from a slightly different
perspective. Sure, one can extend a profile as long a the rules for
extensions are followed!

Regards

Carl

On Wed, Nov 25, 2015 at 1:29 PM, Erik Wilde <[email-masked]> wrote:

> i don't want to spend time fighting about words. i understand where you're
> coming from (i am standards geek myself) and see your point. but RFC 6906
> profiles are a slightly different kind of thing, since they do allow
> "extensions" (as long, and that's essential, as the extensions fall into
> the realm of what a standard allows in its extension model).
>
> so they may be a bit different from how you define "profiles". this might
> mean you're unhappy with the terminology, but that's not something that we
> should discuss. RFC 6906 just talks about "profiles", and assumes that
> those can use extensions.
>
> but: RFC 6906 also says that any profile cannot use extensions that break
> the standard; they have to be defined in a way so that they work withing
> the scope of the standard, and do not break anybody who's not supporting
> the profile. so maybe the RFC 6906 profiles are not as far from the
> profiles that you know after all.
>
> cheers,
>
> dret.
>
> --

-- 
Carl Reed, PhD
Carl Reed and Associates

Mobile: 970-402-0284