Subject: Craeted proposal: PaceProvideSchema

From: Norman Walsh <[email-masked]>
Date: Sat, 10 Jul 2004 09:01:35 -0400

This Pace supports my comment that a schema should be provided as part
of the Atom format spec. Concretely, I suggest that both a RELAX NG
grammar and a W3C XML Schema should be provided.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | Wink at small faults; for thou has
http://nwalsh.com/            | great ones.--Thomas Fuller (II)

---

From: Danny Ayers <[email-masked]>
Date: Sat, 10 Jul 2004 18:24:42 +0200

>grammar and a W3C XML Schema should be provided.

I personally reckon it's almost certainly a good idea, but I don't think 
convincing case has yet been made: what is the purpose of the 
grammar/schema? The rationale in the Pace is that it will "aid readers" 
- how so?

Another obvious question is: should every conformant Atom document be 
valid against the schema?

A perhaps more interesting question is: will every schema-valid document 
be a conformant Atom document?

If the answer to either is "Yes", then there's very good justification 
for a reference to schemas. The latter would mean code generated from 
the schema(s) could be relied on to produce conformant Atom (assuming a 
sensible generator and hiding RFC 3023 in the basement).

A possible scenario which might work with the latter is that the XML 
Schema mandates the element order, but the spec prose doesn't.

Cheers,
Danny.

-- 

Raw
http://dannyayers.com

---

From: Tim Bray <[email-masked]>
Date: Sat, 10 Jul 2004 10:57:50 -0700

On Jul 10, 2004, at 9:24 AM, Danny Ayers wrote:

> I personally reckon it's almost certainly a good idea, but I don't 
> think convincing case has yet been made: what is the purpose of the 
> grammar/schema? The rationale in the Pace is that it will "aid 
> readers" - how so?

I think one normative schema (but two feels like overkill) would be 
immensely valuable, first of all because writing a schema is helpful in 
shaking out corner cases and making sure the verbal hand-waving 
actually captures clean clear syntax constraints.

> valid against the schema?

Yes, but that doesn't mean you'd normally validate at run-time.  The 
value-add here is that when something breaks (as it will) the schema 
will help the parties sort out where the breakage is.  Exactly in the 
same way that web-page designers routinely validate their HTML when 
sorting out weird rendering breakage, but nobody validates HTML in 
production.

> A perhaps more interesting question is: will every schema-valid 
> document be a conformant Atom document?

Probably not, because there are going to be semantic constraints that a 
schema can say nothing about... e.g. are unique ID's really unique, 
globally?

> Schema mandates the element order, but the spec prose doesn't.

That would be unacceptable, the schema and the prose have to be 
consistent. -Tim

---

From: Tim Bray <[email-masked]>
Date: Sat, 10 Jul 2004 12:00:04 -0700

On Jul 10, 2004, at 11:48 AM, Mark Nottingham wrote:

> In practice, that's tricky to achieve.

I disagree.  The syntax constraints you express in the schema are a 
small, tractable subset of the general constraints that appear in the 
prose.

> In other words, if there are requirements buried in the schema, they 
> have to be reflected in the prose; we can't rely on the schema alone 
> to state them.

Agreed.  But, that's orthogonal to whether the schema is normative. -Tim

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 10 Jul 2004 11:48:55 -0700

On Jul 10, 2004, at 10:57 AM, Tim Bray wrote:
> immensely valuable, first of all because writing a schema is helpful 
> in shaking out corner cases and making sure the verbal hand-waving 

What happens when the schema and the prose requirements conflict?

My inclination would be to make the schema non-normative. In other 
words, if there are requirements buried in the schema, they have to be 
reflected in the prose; we can't rely on the schema alone to state 
them.

--
Mark Nottingham     http://www.mnot.net/

---

From: Asbjørn Ulsberg <[email-masked]>
Date: Sun, 11 Jul 2004 01:06:47 +0200

> grammar/schema? The rationale in the Pace is that it will "aid readers"  

It would make it a thousand times simpler for e.g. me to write my own Atom  
consumer and producer tool. Afaik, there are a lot of home-grown  
syndaction tools out there. A normative schema would help the creators of  
those to make a better tool. That is a big +1 in my humble opinion.

> valid against the schema?

Valid: yes, validated every second: no.

> be a conformant Atom document?

Hopefully. But I guess XSD at least isn't powerful enough to express all  
the quirks of Atom. Those quirks will, should I think, be edge cases for  
producers or consumers to wound up in.

-- 
Asbjørn Ulsberg         -=|=-        [email-masked]
«He's a loathsome offensive brute, yet I can't look away»

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 10 Jul 2004 16:50:56 -0700

> small, tractable subset of the general constraints that appear in the 

I predict that it will be very difficult to have a one-to-one 
relationship between the constraints in an XML Schema and prose, and 
furthermore, that small differences and ambiguities between them will 
conflict despite our best efforts. This will lead to a long period of 
sorting out conflicting normative statements in the prose and schema, 
and errata releases of the specification after we've thought it final.

Specifying something twice is not a good idea; if you give people two 
different readings of a constraint, they'll pick one and ignore the 
other.

Since people seem to feel strongly about making the schema as well as 
the prose normative, let's take a look at them after the next round of 
drafts and see where this approach gets us. If we find ourselves 
sinking significant resources into aligning them, however, I'd ask that 
we reconsider such a decision.

---

From: Dare Obasanjo <[email-masked]>
Date: Sat, 10 Jul 2004 17:23:31 -0700 (PDT)

> and errata releases of the specification after we've

Based on my experiences with working with W3C XML
Schema and normative schmeas in various specifications
over the past three years I tend to agree with Mark. 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #95
My dungeon will have its own qualified medical staff complete with bodyguards. That way if a prisoner becomes sick and his cellmate tells the guard it's an emergency, the guard will fetch a trauma team instead of opening up the cell for a look.

__________________________________
Do you Yahoo!?
Take Yahoo! Mail with you! Get it on your mobile phone.
http://mobile.yahoo.com/maildemo

---

From: Sam Ruby <[email-masked]>
Date: Sat, 10 Jul 2004 20:51:37 -0400

> over the past three years I tend to agree with Mark. 

Ditto.

To be clear: presuming that there are adequate volunteers, having one or 
more non-normative schema(s) included in the specification would be a 
very nice thing to have.

- Sam Ruby

---

From: James M Snell <[email-masked]>
Date: Sat, 10 Jul 2004 20:55:55 -0600

+1 on this.  There will be no way to produce a normative XML Schema, but 
it would be extremely useful to have a non-normative schema that at least 
describes the fundamental elements and attributes.

- James M Snell
  [email-masked]
  http://www.ibm.com
  (877) 511-5082 / Office
  930-1979 / Tie Line

Dare Obasanjo <[email-masked]> 
Sent by: [email-masked]
07/10/2004 05:23 PM

To
Mark Nottingham <[email-masked]>, Tim Bray <[email-masked]>
cc
Atom Syntax <[email-masked]>, [email-masked], Norman Walsh 
<[email-masked]>
Subject
Re: Craeted proposal: PaceProvideSchema

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #95
My dungeon will have its own qualified medical staff complete with 
bodyguards. That way if a prisoner becomes sick and his cellmate tells the 
guard it's an emergency, the guard will fetch a trauma team instead of 
opening up the cell for a look.

---

From: Tim Bray <[email-masked]>
Date: Sat, 10 Jul 2004 20:06:38 -0700

On Jul 10, 2004, at 5:23 PM, Dare Obasanjo wrote:

>> relationship between the constraints in an XML
>> Schema and prose

Well, if the consensus is against a normative schema I can live with 
that, but I suggest that Dare's problems may be more rooted in the 
specifics of XML Schema than the general notion of a normative schema.

Having said that, I'm inclined to declare rough consensus in favor of 
writing at least one schema, and the jury being out on whether it can 
be made usefully normative.  Anyone disagree?

We seem to have multiple eager volunteers to do the schemas; to them I 
counsel holding on a couple of revs in the draft till things stabilize. 
-Tim

---

From: Norman Walsh <[email-masked]>
Date: Sun, 11 Jul 2004 10:34:39 -0400

/ Tim Bray <[email-masked]> was heard to say:
| Having said that, I'm inclined to declare rough consensus in favor of
| writing at least one schema, and the jury being out on whether it can
| be made usefully normative.  Anyone disagree?

For the record, I'm completely comfortable with a non-normative
schema. Two normative descriptions of something is bound to lead to
trouble and the prose has to be normative.

| We seem to have multiple eager volunteers to do the schemas; to them I
| counsel holding on a couple of revs in the draft till things
| stabilize. -Tim

I agree, except that tinkering with the schema this morning has
already pointed me to a couple of places where the spec is less than
clear. So I think I'll finish the exercise, even if it gets thrown
away it will have added value, I think.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | The fact of having been born is bad
http://nwalsh.com/            | augury for immortality.-- Santayana

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 11 Jul 2004 08:23:18 -0700

On Jul 11, 2004, at 7:34 AM, Norman Walsh wrote:
> schema.

+1

---

From: Norman Walsh <[email-masked]>
Date: Sun, 11 Jul 2004 12:24:05 -0400

/ Norman Walsh <[email-masked]> was heard to say:
[...]
| atomDateConstruct =
|    atomCommonAttributes,
|    (xsd:date | xsd:dateTime | xsd:gYearMonth | xsd:gYear)

Urk. This is a cut-and-paste bug. I think we mean xsd:dateTime *only* here.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | Everything should be made as simple as
http://nwalsh.com/            | possible, but no simpler.

---

From: Norman Walsh <[email-masked]>
Date: Sun, 11 Jul 2004 12:21:05 -0400

/ Norman Walsh <[email-masked]> was heard to say:
| This Pace supports my comment that a schema should be provided as part
| of the Atom format spec. Concretely, I suggest that both a RELAX NG
| grammar and a W3C XML Schema should be provided.

Ok, I started with the draft-...-00.txt spec and (re)built a RELAX NG
Grammar for it. That exercise led to several questions that I've
posted today. Dave and I have agreed offline to work together on
building the grammar and I said I'd take another stab at it.

A few notes:

1. I tried hard to implement exactly what the spec currently says

2. I intentionally structured the grammar like the spec. This may or
may not be a good thing in the long run. I expect the whole thing
could be made to fit on a single page with a little condensing.

3. I created patterns from some types that we might eventually
consider trying to constrain (like media types and language tags), but
I've made no effort to define the constraints.

4. I've added Schematron rules in a few places to test for things that
a grammar based schema can't practically test.

5. The schema can be converted to XSD with some approximation as XSD
is inherently less capable of expressing some kinds of constraints.

Here it is.

# -*- Relax NG -*-

namespace local = ""
namespace atom = "http://purl.org/atom/ns#"
namespace s = "http://www.ascc.net/xml/schematron"

start = atomFeed | atomEntry

# Attribute definitions

atomCommonAttributes =
   attribute xml:base { atomUri }?,
   attribute xml:lang { atomLanguageTag }?

atomVersionAttribute = attribute version { text }

# Common Atom Constructs

atomContentConstruct =
   atomCommonAttributes,
   attribute type { atomRegisteredMediaType }?,
   attribute mode { "xml" | "escaped" | "base64" }?,
   (text|anyElement)*

atomPersonConstruct =
   atomCommonAttributes,
   (element atom:name { text }
    & element atom:url { atomUri }?
    & element atom:email { atomEmailAddress }?)

atomDateConstruct =
   atomCommonAttributes,
   (xsd:date | xsd:dateTime | xsd:gYearMonth | xsd:gYear)

atomLinkConstruct =
   atomCommonAttributes,
   attribute rel {
      "alternate"
    | "start"
    | "next"
    | "prev"
    | "service.edit"
    | "service.post"
    | "service.feed" },
   attribute type { atomMediaType },
   attribute href { atomUri },
   attribute hreflang { atomLanguageTag }?,
   attribute title { text }?,
   empty

# atom:feed
# TODO: Test for multiple atom:link/@rel='alternate' with the same @type
# The following tests are simple to do, but my validator is giving me trouble.
# TODO: Debug and add them back
#       Test for at least one atom:link/@rel='alternate'
#       Test for atom:author or all atom:entry have atom:author

atomFeed =
   element atom:feed {
   atomCommonAttributes,
   atomVersionAttribute,
   (atomTitle
    & atomLink+
    & atomAuthor?
    & atomContributor*
    & atomTagline?
    & atomId?
    & atomGenerator?
    & atomCopyright?
    & atomInfo?
    & atomModified
    & atomEntry*
    & anyElement*)
}

# atom:title

atomTitle = element atom:title { atomContentConstruct }

# atom:link

atomLink = element atom:link { atomLinkConstruct }

# atom:author

atomAuthor = element atom:author { atomPersonConstruct }

# atom:contributor

atomContributor = element atom:contributor { atomPersonConstruct }

# atom:tagline

atomTagline = element atom:tagline { atomContentConstruct }

# atom:id

atomId = element atom:id { atomUri }

# atom:generator

atomGenerator = element atom:generator {
   atomCommonAttributes,
   atomVersionAttribute?,
   attribute url { atomUri }?,
   text
}

# atom:copyright

atomCopyright = element atom:copyright { atomContentConstruct }

# atom:info

atomInfo = element atom:info { atomContentConstruct }

# atom:modified
# TODO: Test for a timezone that SHOULD be UTC

atomModified = element atom:modified { atomDateConstruct }

# atom:entry
# TODO: Test for multiple atom:link @rel='alternate' with the same @type

atomEntry =
   [
      s:rule [
         context = "/atom:entry"
         s:assert [
            test = "@version"
            "The version attribute is required on standalone atom:entry elements."
         ]
      ]
      s:rule [
         context = "atom:entry"
         s:assert [
            test = "atom:link[@rel='alternate']"
            "An atom:entry must have at least one link element with a rel attribute of 'alternate'."
         ]
      ]
      s:rule [
         context = "atom:entry"
         s:assert [
            test = "atom:author or ../atom:author"
            "An atom:entry must have an atom:author if the parent atom:feed does not."
         ]
      ]
   ]
   element atom:entry {
      atomCommonAttributes,
      atomVersionAttribute?,
      (atomTitle
       & atomLink+
       & atomAuthor?
       & atomContributor*
       & atomId?
       & atomModified
       & atomIssued?
       & atomCreated?
       & atomSummary?
       & atomContent?
       & atomCopyright?
       & anyElement*)
   }

# atom:issued

atomIssued = element atom:issued { atomDateConstruct }

# atom:created
# TODO: Test for a timezone that SHOULD be UTC

atomCreated = element atom:created { atomDateConstruct }

# atom:summary

atomSummary = element atom:summary { atomContentConstruct }

# atom:content

atomContent =
   [
      s:rule [
         context = "atom:content"
         s:assert [
            test = "@type != 'multipart/alternative' or not(@mode)"
            "Multipart/alternative content cannot have a mode attribute."
         ]
      ]
      s:rule [
         context = "atom:content"
         s:assert [
            test = "@type = 'multipart/alternative' and *[@type = 'multipart/alternative']"
            "Multipart/alternative content cannot have multipart/alternative children."
         ]
      ]
      s:rule [
         context = "atom:content"
         s:assert [
            test = "@type != 'multipart/alternative' or child::*"
            "Multipart/alternative content must have children content elements."
         ]
      ]
   ]
   element atom:content { atomContentConstruct }

# Low-level simple types

# TODO: can anything more specific be said about these types?

atomMediaType = text
atomRegisteredMediaType = text
atomLanguageTag = text
atomUri = text
atomEmailAddress = text

# Extensibility

anyForeignElement =
   element * - (atom:* | local:*)
   {
      (attribute * { text }
       | text
       | anyForeignElement)*
   }

anyForeignAttribute =
   attribute * - (atom:* | local:* | xml:*) { text }

anyElement =
   element * - local:*
   {
      (attribute * { text }
       | text
       | anyElement)*
   }

# EOF

                                        Be seeing you,
                                          norm

---

From: Danny Ayers <[email-masked]>
Date: Mon, 12 Jul 2004 10:43:33 +0200

Tim Bray wrote:

> be made usefully normative. 

+1

> counsel holding on a couple of revs in the draft till things 
> stabilize. -Tim

I agree that I don't think we can call either way on making any schema 
normative without more work, so it makes sense to put that on hold. But 
regarding informative schemas - I'd say the more different kinds the 
better*, although it would be useful to have one reference version of 
each type.

I'm not sure of the IETF way of doing the documentation - are there any 
conventions for informative references?

* I'd include RDF/OWL schema/ontology(s) - as an informative resource 
the draft I did based on format 0.3 has already been used as a starting 
point for Henry's BlogEd work.

Cheers,
Danny.

-- 

Raw
http://dannyayers.com

---

From: Bill de hÓra <[email-masked]>
Date: Mon, 12 Jul 2004 16:40:33 +0100

> conventions for informative references?

I suspect the way to do this would be to invert dependencies. 
Publish an Atom/OWL RFC referring to an Atom RFC, not the other way 
around.

cheers
Bill

---

From: Paul Hoffman / IMC <[email-masked]>
Date: Mon, 12 Jul 2004 09:18:36 -0700

At 10:43 AM +0200 7/12/04, Danny Ayers wrote:
>I'm not sure of the IETF way of doing the documentation - are there 
>any conventions for informative references?

Absolutely. In fact, they're required. The "References" section of an 
Internet Draft splits the references into "Normative" and 
"Non-normative". The latter is often called "informative".

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Bill de hÓra <[email-masked]>
Date: Tue, 13 Jul 2004 14:16:48 +0100

>     & anyElement*)

Why not use anyForeignElement here?

cheers
Bill

---

From: Norman Walsh <[email-masked]>
Date: Tue, 13 Jul 2004 09:55:01 -0400

/ Bill de hÓra <[email-masked]> was heard to say:
|
|>     & anyElement*)
|> }
|
| Why not use anyForeignElement here?

Because that's not what the -00 draft says. I think it should be anyForeignElement.

                                        Be seeing you,
                                          norm

-- 
Norman Walsh <[email-masked]> | Great men too make mistakes, and many
http://nwalsh.com/            | among them do it so often that one is
                              | almost tempted to call them little
                              | men.-- Lichtenberg

---

From: "MURATA Makoto (FAMILY Given)" <[email-masked]>
Date: Wed, 21 Jul 2004 01:34:00 +0900

> a grammar based schema can't practically test.

This is a great example.  For those who do not know Schematron, 
here are some good introductions.

http://www.xml.com/pub/a/2004/02/11/relaxtron.html
http://www.xml.com/pub/a/2003/11/12/schematron.html

Cheers,
-- 
MURATA Makoto (FAMILY Given) <[email-masked]>