Subject: PaceReformedContent3

From: Tim Bray <[email-masked]>
Date: Mon, 27 Sep 2004 23:10:59 -0700

I just cooked this up, it embodies the "HTML"/"XHTML" notion, plus 
incorporating comments from Martin and Graham.  I'm not (yet) proposing 
that it replace PaceReformedContentRedux, but I'd appreciate a few 
pairs of eyes being cast over it to catch obvious typos and thinkos.  
http://www.intertwingly.net/wiki/pie/PaceReformedContent3

Here's the Abstract:

This third in the series of proposals starting with the original 
PaceReformedContent provides special "HTML" and "XHTML" values for the 
@type attribute which have the same effect as "text/html" and 
"application/xhtml+xml" in previous iterations but don't claim to be 
media types as they stand.

  Also, provided a value "TEXT" to allow explicitly specifying the 
default; this is good markup design.

  Also, introduced the notion of "discrete media types" from RFC2045.

  Also, forbade the use of "TEXT", "HTML", and "XHTML" in @type when 
"src=" is provided, i.e. the content is remote.

  Also, reorganized the text to be longer but more readable by 
replicating the HTML-specific rules for both "HTML" and "XHTML".

  Took out the exhortations about well-formedness; this is handled at a 
higher level in the definition of atom.

  Added a note that if you have <content type="application/foo+xml">, 
you'd expect there to be a single child element which is the root 
element of the payload.

  Tried to be more explicit in the "src=" discussion per Graham's 
comments.

  Fixed the examples per Martin's comments (I hope).

  The comment from Ulsberg/McLeod calling for media-type parameters has 
been preserved to avoid losing it; someone needs to write a Pace on 
this subject.

---

From: Martin Duerst <[email-masked]>
Date: Tue, 28 Sep 2004 21:28:40 +0900

At 23:10 04/09/27 -0700, Tim Bray wrote:

>that it replace PaceReformedContentRedux, but I'd appreciate a few pairs 
>of eyes being cast over it to catch obvious typos and thinkos.

- "the content may be Web Resource identified by" ->
   "the content may be the Web Resource identified by"

-  "1.If the value is "TEXT", the content of atom:content MUST NOT contain
       child elements. Such text is intended to be":
    there is a formatting error here.

- "3. If the value of "type" begins with "text/" the content of
    atom:content MUST be markup-free text containing no child elements.":
   Does that mean NCRs are not allowed? I hope not. I wonder why only
   this item says "markup-free".

- In 4., maybe explicitly say that there is no way to include DOCTYPE
   and other prolog stuff, and that if that is necessary, the doc needs
   to be external (use src).

- Maybe say explicitly that 3. before 4. means that text/xml and text/...+xml
   would be handled as text/*, but they are strongly discouraged anyway.

- This may already be a comment regarding substance:
   I think XHTML should be listed before HTML, and any other
   means to try to indicate that XHTML is the 'right'/'preferred'
   thing should be taken.

- "The content SHOULD be XHTML text and markup": The SHOULD here, and
   in some other places, may better be a MUST.

>  Fixed the examples per Martin's comments (I hope).

type="HTML" isn't needed for
    <title type="HTML">Example Feed</title>
I don't mind some 'overengineered' examples, but they should be
towards the end, and clearly marked as such.

Same for:
    <summary type="HTML">A lengthy discourse on do&amp;rsquo;s and
        don&amp;rsquo;t&amp;rsquo;s for the HTML &amp;lt;style>
      element</summary>

<!-- if it were type="HTML" you'd have to say "&amp;lt;hr/>" -->
or maybe &amp;lt;hr> ?

<title type="XHTML">How to use &lt;hr /></title>
    same overengineering again, would be fine without type attribute

I don't think the gerbil example is a good idea, it may cross
the brutality threshold for some people/regions.

Regards,    Martin.

---

From: Graham Parks <[email-masked]>
Date: Tue, 28 Sep 2004 08:57:11 -0700

>  Tried to be more explicit in the "src=" discussion per Graham's 

There ares still far too many occurances of "When the "src" attribute is [not] present". Ideally it should be reduced to two sections headed "present" and "not present".

Graham

---

From: Tim Bray <[email-masked]>
Date: Tue, 28 Sep 2004 10:02:59 -0700

>   "the content may be the Web Resource identified by"

Rewritten

>    there is a formatting error here.

Fixed

>   this item says "markup-free".

Fixed

>   to be external (use src).

I wrote it in, but it felt awkward so I took it out; I think this is 
implementors'-guide stuff.

>   would be handled as text/*, but they are strongly discouraged anyway.

Once again, I tried, but couldn't find anything short and clear. 
Suggestions?

>   means to try to indicate that XHTML is the 'right'/'preferred'

Definitely a comment on substance.  Need a Pace for that one.

>   in some other places, may better be a MUST.

The trouble is that verifying whether it's really XHTML, and is really 
valid as direct content of xhtml:body, is an expensive test, and the 
penalty for failing is not that high, most agents are forgiving.  But 
if a want of people want MUST, I'm OK.

> towards the end, and clearly marked as such.

Fixed

>        don&amp;rsquo;t&amp;rsquo;s for the HTML &amp;lt;style>

Nope.  Because this shows how you have to double-escape, and use of the 
hard-wired HTML &rsquo; entity

> <!-- if it were type="HTML" you'd have to say "&amp;lt;hr/>" -->

Cute, but you wouldn't *have* to say that.

>    same overengineering again, would be fine without type attribute

OK

> the brutality threshold for some people/regions.

Anybody is welcome to supply a proposed replacement :) -Tim

---

From: Tim Bray <[email-masked]>
Date: Tue, 28 Sep 2004 10:02:23 -0700

On Sep 28, 2004, at 8:57 AM, Graham Parks wrote:

> There ares still far too many occurances of "When the "src" attribute 
> is [not] present". Ideally it should be reduced to two sections headed 
> "present" and "not present".

I agree; perhaps, if we have consensus on what it says, we could ask 
our editors (who are a fairly brainy lot) to consider re-ordering this 
section a bit.  Right now the document has a 2-level hierarchy where 
each element gets a section, but no attributes get sections.  I suspect 
that 5.10 atom:content would benefit from breaking this convention. 
-Tim

---

From: Martin Duerst <[email-masked]>
Date: Wed, 29 Sep 2004 08:52:43 +0900

Ah, yeah, and something I forgot yesterday:

Things such as "TEXT" seem to give the impression that
this is case-sensitive. However, my recollection is that
media types are case-insensitive (but I don't remember
having seen them in upper-case at all).
Anyway, the ghist is: say clearly how casing works for @type.

Regards,    Martin.

---

From: Tim Bray <[email-masked]>
Date: Tue, 28 Sep 2004 20:18:00 -0700

> Anyway, the ghist is: say clearly how casing works for @type.

Right, media types are case-insensitive, but this isn't, as you pointed 
out, a media type, it's an atom "type" attribute, something else 
entirely, with three literal pre-specified values: TEXT, HTML, XHTML.  
So type="TEXT/HTML" would be perfectly legal, but still wouldn't 
conflict. -Tim

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Wed, 29 Sep 2004 22:43:11 +0200

>http://www.intertwingly.net/wiki/pie/PaceReformedContent3

I am opposed to overload the type attribute in this way, statements like
"the difference between HTML and XHTML is that with HTML you write all
markup escaped" make no sense when read out of context and are generally
confusing. The attribute would also be rather different from the type
attribute in various document formats such as VoiceXML and SVG when used
for similar functionality. It is very different from the type attribute
proposed for XHTML 2.0...

There are too many technical flaws in the proposal to properly review it
a number of examples are invalid or would not work as expected, for
example

  ...
  <content type="image/jpeg" src="http://example.com/pix/ct.jpg" />
  ...

The proposal states

  atom:content MAY have a "type" attribute, When present, the value MAY
  be one of "TEXT", "HTML", or "XHTML". Failing that, it MUST be the
  name of an Internet Media Type which is a "discrete type" as defined
  in section 5 of [RFC2045].

The value in the example is not one of the pre-defined values and not a
discrete-type as defined in RFC 2045, it would need to be e.g. "image".

  ...
  <content type="image/gif">
  R0lGODlhBQAFAJEAAAAAAP////8AAAAAACwAAAAABQAFAAACBYyPJslRADs=
  </content>
  ...

The proposal states

  For all other values of "type", the content of atom:content MUST be
  a valid base64 encoding, which when decoded SHOULD be suitable for
  handling by software that knows the indicated media type.

#xA #x20 #x20 ... #xA #x20 #x20 cannot occur in valid base64 as defined
by RFC 3548.

  ...
  <entry>
    <!-- if it were type="HTML" you'd have to say "&amp;lt;hr/>" -->
    <title>How to use &lt;hr /></title>
  ...
    <content>&lt;hr /> is a strictly graphical element which draws a
    horizontal line. &lt;hr />, &lt;hr/>, and &lt;hr>&lt/hr> are all
    equivalent strictly speaking, but the first form is the most
    common.</content>
  ...

The proposal states

  The HTML markup SHOULD be such that it could validly appear directly
  within an HTML <BODY> element.

#PCDATA cannot be used directly within <body> in HTML 4.01 Strict, so
the example would either be non-conforming after the suggested change
or the proposal is way too vague to be meaningful in this regard.

It also seems odd that the escaped-markup mode is limited to HTML
content, don't you think some users might want to use it with XHTML?
I further note that e.g.

  <content text="text/vnd.wap.wml">
    <wml ...>...</wml>
  </content>

would be illegal just like

  <content text="text/xhtml+xml">
    <p>...</p>
  </content>

and

  <content text="text/xml">
    <foo />
  </content>

and even

  <content text="application/xml">
    <foo />
  </content>

And it is impossible to include any XML content directly that requires
the presence of a document type declaration as escaped-markup mode is
not available and

  <content text="...+xml">
    <!DOCTYPE ...
  </content>

would be illegal.

I am opposed to the inconsistent casing, most pre-defined Atom syntax
uses lower-case, having something UPPER-CASE inbetween is ugly and not
going to work, people will write it lower-case regardless.

---

From: Tim Bray <[email-masked]>
Date: Wed, 29 Sep 2004 15:38:42 -0700

> proposed for XHTML 2.0...

I take it you prefer the approach in the current draft where we have 
separate "mode" and "type" attributes?  See discussion below.

> discrete-type as defined in RFC 2045, it would need to be e.g. "image".

Oops, you're right, I realize in re-reading 2045 that the correct 
terminology should be

"an internet media type in which, to use the terms of Section 5 of 
[RFC2045], the top-level is a discrete type".

I'll fix the Pace.

> #xA #x20 #x20 ... #xA #x20 #x20 cannot occur in valid base64 as defined

I was puzzled by this comment given that I often see base64 data 
surrounded by whitespace of various kinds and I know from experience 
that most base64 software routinely ignores whitespace.  Having 
reviewed RFC3548 section 2.3, what we need to do is to specify that in 
Atom, it's OK to follow the start-tag with whitespace, precede the 
end-tag with whitespace, and separate lines with #A.

> or the proposal is way too vague to be meaningful in this regard.

That's a good point.  I suggest modifying the proposal to say "can 
appear directly within HTML <DIV> or <xhtml:div>".  Or do you have a 
better idea?

> content, don't you think some users might want to use it with XHTML?

You raise this again in a more general way with other media-types 
below, I'll discuss there.

> would be illegal

Why?

> just like

Agreed, they should use "XHTML".

>   <content text="text/xml">

Oh, I see, because we say that if it begins with "text" it can't have 
child elements. Bah, I *hate* "text/xml".  Either we explicitly forbid 
"text/xml" or we change the rule so this is treated as ending in 
"+xml".

>   <content text="application/xml">

I see, because we say that if it doesn't end in +xml it should be 
base64.  Once again, either we explicitly forbid application/xml or we 
say that "*/xml" is treated as "*+xml".

I propose modifying the pace to treat */xml the same as */*+xml.

> not available

Agreed.  I think that our discussion had led us to consensus that 
escaped-markup mode was reserved for "legacy HTML", and that if you 
really wanted a complete XML document, that's what the "src=" was for.  
The fact is that XML is a lousy packaging format for XML, sigh.

If we went back to the two-attribute approach, then you'd be able to 
use the escaped-markup mode for all kinds of XML including XHTML as you 
note above.  I think we had consensus that we preferred the simplicity 
of this approach, even if it means you can't have a <!DOCTYPE> inside 
<atom:content>.  If the group wants to change its mind again I'd be 
disappointed but wouldn't stand in the way.

> going to work, people will write it lower-case regardless.

I was looking for something that was obviously not a media-type to the 
casual eye and was reasonably short.  What would you suggest as an 
alternative? -Tim

---

From: Dare Obasanjo <[email-masked]>
Date: Wed, 29 Sep 2004 16:12:56 -0700 (PDT)

> The fact is that XML is a lousy packaging format for

+1 

Quite frankly I think embedding XML directly in
atom:content is a non-scenario mostly introduced by
geeks on this list than something that most publishers
or content producers intend to do. 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #23
I will keep a special cache of low-tech weapons and train my troops in their use. That way -- even if the heroes manage to neutralize my power generator and/or render the standard-issue energy weapons useless -- my troops will not be overrun by a handful of savages armed with spears and rocks.

_______________________________
Do you Yahoo!?
Declare Yourself - Register online to vote today!
http://vote.yahoo.com

---

From: Martin Duerst <[email-masked]>
Date: Wed, 29 Sep 2004 13:05:37 +0900

I'm not worried about any conflicts. I just want the Pace
to clearly state that all these values are case-insensitive
(or whatever else you think they are). If I have overlooked
a statement to that effect, sorry.

Regards,    Martin.

At 20:18 04/09/28 -0700, Tim Bray wrote:

>out, a media type, it's an atom "type" attribute, something else entirely, 
>with three literal pre-specified values: TEXT, HTML, XHTML.
>So type="TEXT/HTML" would be perfectly legal, but still wouldn't conflict. 
>-Tim

---

From: Tim Bray <[email-masked]>
Date: Wed, 29 Sep 2004 16:22:46 -0700

> a statement to that effect, sorry.

Not quite right.  Media-types *are* case-insensitive (blecch) but the 
reserved words atom specifies for special type= values can and should 
be case-sensitive. -Tim

---

From: Martin Duerst <[email-masked]>
Date: Thu, 30 Sep 2004 08:35:52 +0900

At 15:38 04/09/29 -0700, Tim Bray wrote:

>>confusing.

I guess that could be fixed by more careful wording.

>>The attribute would also be rather different from the type

Any pointers?

>That's a good point.  I suggest modifying the proposal to say "can appear 
>directly within HTML <DIV> or <xhtml:div>".  Or do you have a better idea?

We probably need the same changes for 'XHTML'.

>You raise this again in a more general way with other media-types below, 
>I'll discuss there.
>
>"text/xml" or we change the rule so this is treated as ending in "+xml".

I think forbidding text/xml is the right way to go.

>base64.  Once again, either we explicitly forbid application/xml or we say 
>that "*/xml" is treated as "*+xml".

We should say that this is treated like +xml.

>escaped-markup mode was reserved for "legacy HTML", and that if you really 
>wanted a complete XML document, that's what the "src=" was for.

I agree. Escaping a whole XML document just to be able to include
the 'doctype' is a really bad idea. Having things with doctypes
external should be good enough. If having doctypes is really
very important, I'd prefer somebody come up with a special
escaping syntax that only escapes the doctype part. We could
even do this with something like:
    <content type='' doctype=''>
[just an idea that would have to be baked a bit more]

>If we went back to the two-attribute approach, then you'd be able to use 
>the escaped-markup mode for all kinds of XML including XHTML as you note above.

If the only/main use case for this is the doctype, then I'm -1, or even -2.

>  I think we had consensus that we preferred the simplicity of this 
> approach, even if it means you can't have a <!DOCTYPE> inside 

The casing question is separate. There is no problem with
people writing 'html' or 'xhtml' in lower case if we make @type
case-insensitive. Because media types are already (as far as I know)
case-insensitive, that's probably the easiest way to go anyway.
See my other mail.

Regards,    Martin.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Thu, 30 Sep 2004 01:44:16 +0200

>separate "mode" and "type" attributes?  See discussion below.

That would make more sense, yes.

>[RFC2045], the top-level is a discrete type".

The wording is not all that clear to me either, but I should note that
conformance to the Atom specification would depend on unstable content
if the lexical space of the attribute is defined in terms of discrete-
type as per RFC 2045 (it refers to the ietf-token symbol which stands
for RFC'd and IANA-registered strings only). That's not easily testable.

>appear directly within HTML <DIV> or <xhtml:div>".  Or do you have a 

Say I'd be willing to integrate support for Atom and/or HTML/XHTML in
Atom documents to the W3C Markup Validator, I would not know whether
this means

  * HTML 2.0
  * HTML 3.2
  * HTML 4.0
     * Strict
     * Transitional
  * HTML 4.01
     * Strict
     * Transitional
  * XHTML 1.0
     * Strict
     * Transitional
  * XHTML 1.1
  * XHTML Basic 1.0
  * XHTML Mobile 1.0
  * XHTML Print 1.0
  * ISO-HTML
  * ...

So I would need to make my own decision (which will be wrong since all
of them would be too loose and only one of them too strict, etc.) or I
decide to avoid the headache and drop the idea of adding Atom support.
I prefer machine-testable conformance requirements, a clear reference
to a single HTML or XHTML specification would help a lot.

><atom:content>.

I am afraid I do not really see how this approach adds simplicity. The
nature of something and the storage method of it are different concepts,
keeping those separate seems easier to understand to me. Maybe you could
point me to a document that describes the complexity involved with using
two attributes here?

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Thu, 30 Sep 2004 01:51:11 +0200

* Martin Duerst wrote:
>>>[@type in XHTML 2.0]

<http://www.w3.org/TR/xhtml2/xhtml2.html#adef_embedding_type>:

  This attribute specifies the allowable content types of the relevant
  src URI. At its most general, it is a comma-separated list of media
  ranges with optional accept parameters, as defined for HTTP in section
  14.1 of [RFC2616] as the field value of the accept request header of
  HTTP.

  In its simplest case, this is just a media type, such as "image/png"
  or "application/xml", but it may also contain asterisks, such as
  "image/*" or "*/*", or lists of acceptable media types, such as
  "image/png, image/gif, image/jpeg".

  The user agent must combine this list with its own list of acceptable
  media types by taking the intersection, and then use the resulting
  list as the field value of the accept request header when requesting
  the resource using HTTP.

  For instance, if the attribute specifies the value "image/png,
  image/gif, image/jpeg", but the user agent does not accept images of 
  type "image/gif" then the resultant accept header would contain
  "image/png, image/jpeg". 

  A user agent should imitate similar behavior when using other methods
  than HTTP. For instance, when accessing files in a local filestore, <p
  src="logo" type="image/png, image/jpeg"> might cause the user agent
  first to look for a file logo.png, and then for logo.jpg.

  If this attribute is not present, "*/*" is used for its value.

---

From: Dare Obasanjo <[email-masked]>
Date: Wed, 29 Sep 2004 17:15:13 -0700 (PDT)

> point me to a document that describes the complexity

It is more complex because there are two attributes to
process instead of one to know how to dispatch on the
children of the atom:content element. As someone who
implemented this for Atom 0.3 this seemed like
unnecessary complexity especially when the content
would most likely be either HTML or XHTML. 

Then there is the fact that both attributes can be
combined in ways that don't make logical sense but
would in fact be valid (e.g. mode='xml' &
type='text/html' or mode='text/plain' and
mode='base64' from Atom 0.3).  

=====

---

From: Eric Scheid <[email-masked]>
Date: Thu, 30 Sep 2004 10:38:36 +1000

> mode='base64' from Atom 0.3).

the latter isn't nonsense if your text/plain content contains illegal-in-XML
characters (like Form Feeds).

e.

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Thu, 30 Sep 2004 03:05:27 +0200

>would most likely be either HTML or XHTML. 

But you need to post-process this one attribute in order to determine
the two bits of information you need to process the actual content. I
still do not understand how that is simpler, you would have code like

  my $type = $content->getAttributeNS(undef, "type");

  if    ($type eq "TEXT")     { ... }
  elsif ($type eq "XHTML")    { ... }
  elsif ($type eq "HTML")     { ... }
  elsif ($type =~ m(^text/)i) { ... }
  elsif ($type =~ m(\+xml$)i) { ... }
  else                        { ... }

versus

  my $type = $content->getAttributeNS(undef, "type");
  my $mode = $content->getAttributeNS(undef, "mode");

To me the latter seems simpler... Even if you could save a few lines
of code in some implementation, it would not seem very relevant to me.

>mode='base64' from Atom 0.3).  

I do not think that these do not make any sense. Maybe my plain text
content contains a form feed or other control character not allowed
in XML, using base64 for it would make sense in this case. It might
not be very useful, but it would not really hurt either.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 30 Sep 2004 10:38:43 +0900

>characters (like Form Feeds).

The typical example of these being internet-drafts :-(.

Regards,   Martin.

---

From: Martin Duerst <[email-masked]>
Date: Thu, 30 Sep 2004 08:39:29 +0900

At 16:22 04/09/29 -0700, Tim Bray wrote:

>reserved words atom specifies for special type= values can and should be 
>case-sensitive. -Tim

So you are saying that type='TEXT/plain' or type="IMAGE/SVG+XML' are
legal, but type='html' or type='xhtml' aren't? I have nothing against
using different casing in the examples to make the distinction clear,
but I think it is an extremely bad idea to have an attribute where
some of the values are case-sensitive and others are not. I haven't
yet heard of such a beast in any spec, and I have a feel for why not.

Regards,   Martin.

---

From: Tim Bray <[email-masked]>
Date: Wed, 29 Sep 2004 19:46:41 -0700

On Sep 29, 2004, at 4:39 PM, Martin Duerst wrote:

> legal, but type='html' or type='xhtml' aren't?

Exactly.  You check for a literal match against "TEXT", "HTML", or 
"XHTML" and if all three fail then you hand it to a media-type handler, 
which presumably knows about the rules for media types.  Why is it 
better if I have to do case-folding before checking?  -Tim

---

From: Graham Parks <[email-masked]>
Date: Thu, 30 Sep 2004 13:25:33 +0100

Tim Bray wrote:

> You check for a literal match against "TEXT", "HTML", or "XHTML" and 
> if all three fail then you hand it to a media-type handler, which 
> presumably knows about the rules for media types.  Why is it better if 
> I have to do case-folding before checking?

At this point I think it's time to say that requiring or encouraging 
capital letters will end up as a major point of confusion once the 
public get their hands on it. It also seems a little patronising.

Graham