Subject: [Geojson] Being more specific about immutable semantics

From: Sean Gillies <[email-masked]>
Date: Mon, 8 Feb 2016 11:29:54 -0700

We've got consensus that extensions must not change the semantics of
GeoJSON types.

https://github.com/geojson/draft-geojson/blob/master/middle.mkd#semantics-of-geojson-properties-and-types-are-not-extensible

I'd like this to be more specific to steer implementers away from producing
monsters like this:

{
  "features":[
    {
     "features": [],
      "geometry": {
      "coordinates": [0, 0],
        "type": "Point"
    },
    "properties": {
     "name": "Null Island"
    },
    "type": "Feature"
    }
    ],
  "geometry": {
    "coordinates": [123.0, 345.0],
    "type": "Point"
  },
  "properties": {
    "name": "Dinagat Islands"
  },
  "type": "FeatureCollection"
}

It's a FeatureCollection with semantics of Feature and Geometry containing
a Feature with semantics of FeatureCollection.

Concretely: I propose to add to the "Extending GeoJSON" section a
definition of the GeoJSON properties, enumerating "id", "type", "features",
&c. And add to the "Semantics of GeoJSON properties and types are not
extensible" section something like the following:

- A Feature or Geometry object MUST NOT contain a "features" property.
- A FeatureCollection or Geometry object MUST NOT contain a "geometry" or
"properties" object.
- A FeatureCollection or Feature object MUST NOT contain a "coordinates"
property.

I'll prepare a PR for review unless there are objections.

-- 
Sean Gillies

---

From: Calvin Metcalf <[email-masked]>
Date: Mon, 08 Feb 2016 18:54:51 +0000

I could see some legitimate wants for extensions to do this, especially
feature collections having geometry or properties.  But more importantly
why should we be prohibit this?  What exactly do we gain by making that
invalid json?

On Mon, Feb 8, 2016 at 1:29 PM Sean Gillies <[email-masked]> wrote:

> I'd like this to be more specific to steer implementers away from
> producing monsters like this:
>
> {
>   "features":[
>     {
>      "features": [],
>       "geometry": {
>       "coordinates": [0, 0],
>         "type": "Point"
>     },
>     "properties": {
>      "name": "Null Island"
>     },
>     "type": "Feature"
>     }
>     ],
>   "geometry": {
> _______________________________________________
> GeoJSON mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/geojson
>

---

From: Sean Gillies <[email-masked]>
Date: Mon, 8 Feb 2016 12:29:30 -0700

I'm not saying that extensions must not add new set-like or geometry or
attribute extension values to GeoJSON objects, only that they must not use
the GeoJSON keys "features", "geometry", and "properties" to do so. They
may not bend core GeoJSON properties and create non-extension extensions. A
clear distinction between core and extension properties helps implementers
and I'm proposing to make the distinction more clear than it is now.

On Mon, Feb 8, 2016 at 11:54 AM, Calvin Metcalf <[email-masked]>
wrote:

>> It's a FeatureCollection with semantics of Feature and Geometry
>> containing a Feature with semantics of FeatureCollection.
>>

-- 
Sean Gillies

---

From: Erik Wilde <[email-masked]>
Date: Mon, 8 Feb 2016 11:39:42 -0800

hello.

On 2016-02-08 11:29, Sean Gillies wrote:
> attribute extension values to GeoJSON objects, only that they must not
> use the GeoJSON keys "features", "geometry", and "properties" to do so.
> They may not bend core GeoJSON properties and create non-extension
> extensions. A clear distinction between core and extension properties
> helps implementers and I'm proposing to make the distinction more clear
> than it is now.
> On Mon, Feb 8, 2016 at 11:54 AM, Calvin Metcalf
> <[email-masked] <mailto:[email-masked]>> wrote:
>     I could see some legitimate wants for extensions to do this,
>     especially feature collections having geometry or properties.  But
>     more importantly why should we be prohibit this?  What exactly do we
>     gain by making that invalid json?

i think i like sean's approach. if i understand him correctly, he wants 
to make it more clear not only that extensions cannot change the core 
vocabulary, but also that extensions cannot reuse it in ways not covered 
by the spec.

if that's indeed what sean is saying, then i think what we gain by this 
is that we disallow the core vocabulary to be used in unpredictable ways 
that are not covered by the spec. that's a good thing, i would argue, 
because otherwise consumers see the core vocabulary being used, but have 
no easy way to find out what it actually means, and also have to be 
careful enough to be better safe than sorry and ignore that usage of the 
core vocabulary, even though it is core, because it is used in contexts 
where it is has no core-defined meaning.

cheers,

dret.

-- 
erik wilde | mailto:[email-masked] |
            | http://dret.net/netdret    |
            | http://twitter.com/dret    |

---

From: Sean Gillies <[email-masked]>
Date: Mon, 8 Feb 2016 12:46:18 -0700

Oops, sent too quickly.

Here's a nice property of more distinct semantics: improved support for
efficient scanning and highly optimized single-pass parsing of GeoJSON.

When lexically sorted, `"features: ["` precedes `"type":
"FeatureCollection"`, and often by many megabytes. If we restrict
"features" to FeatureCollection it can be used as the signal of a
FeatureCollection by implementations that don't want to be constrained to
loading the entire GeoJSON into memory.

On Mon, Feb 8, 2016 at 12:29 PM, Sean Gillies <[email-masked]>
wrote:

>>> - A FeatureCollection or Geometry object MUST NOT contain a "geometry"
>>> or "properties" object.

-- 
Sean Gillies

---

From: Erik Wilde <[email-masked]>
Date: Mon, 8 Feb 2016 11:52:19 -0800

just to be sure:

On 2016-02-08 11:46, Sean Gillies wrote:
> "FeatureCollection"`, and often by many megabytes.

but we're not making any assumptions about GeoJSON using sorted keys 
only, right? so there's no way to optimize without parsing the whole thing.

cheers,

dret.

---

From: Calvin Metcalf <[email-masked]>
Date: Mon, 08 Feb 2016 19:53:17 +0000

ok you've sold me thanks for clarifying, I might suggest loosening up the
restriction on properties on a FeatureCollection as I bet there are in the
wild documents that do that currently (I'm pretty sure I've done that
myself).

On Mon, Feb 8, 2016 at 2:46 PM Sean Gillies <[email-masked]> wrote:

>> On Mon, Feb 8, 2016 at 11:54 AM, Calvin Metcalf <[email-masked]
>> > wrote:
>>
>>>> - A FeatureCollection or Feature object MUST NOT contain a
>>>> "coordinates" property.
>>>>

---

From: Calvin Metcalf <[email-masked]>
Date: Mon, 08 Feb 2016 19:55:05 +0000

There is no assumption of keys being sorted here, he's saying that if they
were lexically sorted then type would come after features and if features
were restricted to feature collections then you could parse earlier.

---

From: Sean Gillies <[email-masked]>
Date: Mon, 8 Feb 2016 13:11:22 -0700

On Mon, Feb 8, 2016 at 12:52 PM, Erik Wilde <[email-masked]> wrote:

> but we're not making any assumptions about GeoJSON using sorted keys only,
> right? so there's no way to optimize without parsing the whole thing.
>
>
> cheers,
>
> dret.
>

Definitely not making assumptions about order. I'm just pointing out (as
Phillip Hallam-Baker did in the JSON list) that GeoJSON's "type" can arrive
way too late in a byte stream to be useful to a parser and that "features"
is the thing for parsers to key on.

I don't want GeoJSON to cater to a specific kind of parser, just want to
point out a benefit for SAX-like parsers of making "features" exclusive.

-- 
Sean Gillies

---

From: Sean Gillies <[email-masked]>
Date: Mon, 8 Feb 2016 15:24:42 -0700

Thanks for the feedback! PR is at
https://github.com/geojson/draft-geojson/pull/142.

-- 
Sean Gillies

---

From: Sean Gillies <[email-masked]>
Date: Wed, 10 Feb 2016 10:43:12 -0700

Would it be good to add that foreign objects in GeoJSON MUST NOT have
"features", "geometry", "properties" properties or is that none of
GeoJSON's business? It would help SAX style parsers, but I don't want to
cater and JSON objects are (in Crockford's vision) independent namespaces.

On Mon, Feb 8, 2016 at 3:24 PM, Sean Gillies <[email-masked]> wrote:

> ...

-- 
Sean Gillies

---

From: Erik Wilde <[email-masked]>
Date: Wed, 10 Feb 2016 09:50:26 -0800

> cater and JSON objects are (in Crockford's vision) independent namespaces.

i think that it makes sense to define that GeoJSON properties MUST NOT 
be used in ways other than explicitly allowed by the spec. anything else 
leaves such usages undefined, and potentially conflicting across 
different ways in which people may start using GeoJSON properties in 
contexts that are not covered by the spec.

from the spec writing point of view, we should also ask: what's the 
recommended processing model if an implementation encounters such a use? 
should it ignore or fail? i vote to make that a "MUST ignore".

---

From: "Stefan Hagen" <[email-masked]>
Date: Fri, 12 Feb 2016 08:31:50 +0100

On 2016-FEB-10 18:50:26 +01:00, Erik Wilde <<[email-masked]>> wrote:

> i think that it makes sense to define that GeoJSON properties MUST NOT be used in ways other than explicitly allowed by the spec. anything else leaves such usages undefined, and potentially conflicting across different ways in which people may start using GeoJSON properties in contexts that are not covered by the spec.
> 
> from the spec writing point of view, we should also ask: what's the recommended processing model if an implementation encounters such a use? should it ignore or fail? i vote to make that a "MUST ignore".
> +1 on MUST ignore or a phrase with a plain ignore (thus not pushing a MUST if the prose is clear enough), *but* ...

TL;DR Is there a simple proposal for specifying the necessary and sufficient extension rules for the future?

Details:

The tricky part to me w.r.t. extension is the idea, we could foresee the future and explicitly allow extension in all places relevant (to the future extended use) thus not leaving out regions of interest in future and over detailing allowed usage in then uninteresting parts.

In OData and other protocol specs we implement namespacing concepts to overcome this, like all so and so tokens starting with a dollar sign ($) are reserved for future (spec) use or the like.

This is not easy in JSON, where the "keys" of the objects (maps/dicts) are named only "strings" as part of there grammar member element production in RFC 7159 and there are disjoint traditions in associating naming concepts with those.

I hope for no further bike shredding when we try to explicate the allowed extension points and mechanisms.

Best,
Stefan.

---

From: Carl Reed <[email-masked]>
Date: Fri, 12 Feb 2016 09:11:50 -0700

How about changing the wording so that the requirement is a MUST instead of
a MUST NOT?

Perhaps easier to understand.

Cheers

Carl

On Wed, Feb 10, 2016 at 10:50 AM, Erik Wilde <[email-masked]> wrote:

> i think that it makes sense to define that GeoJSON properties MUST NOT be
> used in ways other than explicitly allowed by the spec. anything else
> leaves such usages undefined, and potentially conflicting across different
> ways in which people may start using GeoJSON properties in contexts that
> are not covered by the spec.
>

-- 
Carl Reed, PhD
Carl Reed and Associates

Mobile: 970-402-0284

---

From: Erik Wilde <[email-masked]>
Date: Fri, 12 Feb 2016 17:31:10 +0100

hello carl.

On 2016-02-12 17:11, Carl Reed wrote:
> How about changing the wording so that the requirement is a MUST instead
> of a MUST NOT? Perhaps easier to understand.

the main idea is to prohibit the use of properties that are defined in 
the GeoJSON context outside of their context, right? so just starting 
from this general idea, i think a "MUST NOT" is the way to go. nothing 
is keeping us from adding language that clarifies/explains this.

for the "MUST" idea, i don't see this being easier to understand. what i 
can come up with is something like this:

"GeoJSON properties MUST be used in the context for which they are defined."

that reads a bit redundant. but maybe that's a bad suggestion. but if 
that's what we say, then it is harder to understand (i think) than 
explicitly saying what we really want to say:

"GeoJSON properties MUST NOT be used in contexts other than the ones 
they are defined for."

but i guess this is mostly a matter of specification language 
preference. if the goal is to keep developers from doing something, i 
think the best way to explain that to them is to make that explicit.

thanks and cheers,

dret.

---

From: Allan Doyle <[email-masked]>
Date: Fri, 12 Feb 2016 14:43:00 -0500

Are we disallowing nesting a GeoJSON object inside another? Here’s a GeoJSON LineString called “foo” embedded in a Feature.

{ "type": "Feature",
  "geometry": {
      "type": "Polygon",
      "coordinates": [
          [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0],
            [100.0, 1.0], [100.0, 0.0] ]
      ]
  },
  "properties": {
      "prop0": "value0",
      "prop1": {"this": "that"},
      "foo": {
          "type": "Feature",
          "geometry": {
              "type": "LineString",
              "coordinates": [
                  [102.0, 0.0], [103.0, 1.0], [104.0, 0.0], [105.0, 1.0]
              ]
          },
          "properties": {
              "prop0": "value0",
              "prop1": 0.0
          }
      }
  }
}

> On Feb 12, 2016, at 11:31 AM, Erik Wilde <[email-masked]> wrote:
> 
> hello carl.
> 
> the main idea is to prohibit the use of properties that are defined in the GeoJSON context outside of their context, right? so just starting from this general idea, i think a "MUST NOT" is the way to go. nothing is keeping us from adding language that clarifies/explains this.
> 
> for the "MUST" idea, i don't see this being easier to understand. what i can come up with is something like this:
> 
> that reads a bit redundant. but maybe that's a bad suggestion. but if that's what we say, then it is harder to understand (i think) than explicitly saying what we really want to say:
> 
> "GeoJSON properties MUST NOT be used in contexts other than the ones they are defined for."
> 
> but i guess this is mostly a matter of specification language preference. if the goal is to keep developers from doing something, i think the best way to explain that to them is to make that explicit.
> 
> thanks and cheers,
> 
> dret.
> 
> --