Subject: [p3-payload] Media-Type -- Two Values, One Cup Anti-Pattern?

From: Ray Polk <[email-masked]>
Date: Fri, 6 Jan 2012 01:59:10 -0700

Hi,

I had a question about the evolution of media types.

Early media types constrained themselves strictly to describing the format
(syntax) of the representation (image/png, application/xml).  As time wore
on, though, media types sprang up that contained both syntactic & semantic
type information (see the application/vnd.openxmlformats-*+xml series for
an example).  Negative ramifications seem obvious (add json, yaml => m*n
types, parsing, querying, ...); in fact, in other contexts (DB & object
fields), placing two values into a single cup/bucket is such a basic
mistake with such obvious drawbacks, it's rarely even discussed as an issue.

Does anyone else see this as a problem?

Media types of this sort are being created to solve a problem, though.  I
believe they may indicate an unsatisfied need to specify not only syntactic
type, but also semantic type in a uniform way.  Which would present the
more uniform interface?

   - Each implementer creating custom headers & custom registries for these
   data types
   - Header & registry defined by a fantastic standards body :-)

At any rate, thanks for all your hard work & genius.  Apologies if this
isn't the proper forum/format/approach/timing/anythingelse.

Regards,
Ray

---

From: "Roy T. Fielding" <[email-masked]>
Date: Fri, 6 Jan 2012 13:05:45 -0800

On Jan 6, 2012, at 12:59 AM, Ray Polk wrote:

> Early media types constrained themselves strictly to describing the format (syntax) of the representation (image/png, application/xml).

No, media types have always described the intended MUA (MIME user agent)
processing.  They are named after data formats because most of the time
there is a 1:1 correlation between format and intended processing mechanism.
Regardless, for any given data format there are at least two and often
three different media types that might apply.  For example, most HTML
is equally valid as text/html, text/plain, or application/octet-stream.
The only difference is how the sender intends that the recipient
should process the message.

....Roy

---

From: Ray Polk <[email-masked]>
Date: Tue, 17 Jan 2012 14:27:54 -0700

Thanks for the correction.  To be sure I understand, I'll paraphrase your
response:

Media types are a key used by user agents to look up processing
instructions with a registered authority (iana for http).  Though these
processing instructions might include anything, many (all?) include format
information; further, many include _only_ format information.  As a result,
a practice arose of naming the key after the format.

Perhaps my mistake confused the main point I was trying to make, though.  I
wasn't concerned with the number of media types that are applicable to an
end point.  My concern was with the ramifications of including data
semantics in the processing mechanism along with the format information.

By combining the two, the number of total registered types can/could
increase geometrically.  The vnd.openxmlformats-*+xml formats are an
example of that potential.  Imagine if someone registers openjsonformats &
openyamlformats.  Then they want both text and binary formats.  The list of
~75 types at the end of this email becomes 400+.

Now imagine the growth as such a practice spreads to other domains AND as
the number of popular text formats grows...

Endpoints and clients dealing with format+datatype media types would be
repeating themselves over and over again.  Imagine the Accept headers...

To put it another way, can I register vnd.cat-jpeg? :-)  Should I?

Sure, a design that allows endpoints to supply user agents a key to lookup
processing instructions maximizes flexibility.  Catch-all text columns,
member variables and void pointers all also maximize flexibility.

When in practice, though, ~100% of those catch-alls include format
information.  Isn't it time for a more strongly typed header/column/member?
 (to go along witth the catch all)

-Ray

(for reference, these are the Media Types to which I'm referring above when
I say "openxmlformats":

<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.custom-properties+xml>
vnd.openxmlformats-officedocument.customXmlProperties+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.customXmlProperties+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.drawing+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.drawing+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.drawingml.chart+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.drawingml.chart+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.drawingml.chartshapes+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.drawingml.chartshapes+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.drawingml.diagramColors+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.drawingml.diagramColors+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.drawingml.diagramData+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.drawingml.diagramData+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.drawingml.diagramLayout+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.drawingml.diagramLayout+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.drawingml.diagramStyle+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.drawingml.diagramStyle+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.extended-properties+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.extended-properties+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.commentAuthors+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.commentAuthors+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.comments+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.comments+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.handoutMaster+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.handoutMaster+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.notesMaster+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.notesMaster+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.notesSlide+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.notesSlide+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.presentation<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.presentation>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.presentation.main+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.presentation.main+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.presProps+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.presProps+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.slide<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.slide>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.slide+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.slide+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.slideLayout+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.slideLayout+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.slideMaster+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.slideMaster+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.slideshow<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.slideshow>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.slideshow.main+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.slideshow.main+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.slideUpdateInfo+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.slideUpdateInfo+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.tableStyles+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.tableStyles+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.tags+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.tags+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.template<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml-template>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.template.main+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.template.main+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.presentationml.viewProps+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.presentationml.viewProps+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.calcChain+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.calcChain+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.chartsheet+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.chartsheet+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.comments+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.comments+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.connections+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.connections+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.dialogsheet+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.dialogsheet+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.externalLink+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.externalLink+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.pivotCacheDefinition+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.pivotCacheDefinition+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.pivotCacheRecords+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.pivotCacheRecords+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.pivotTable+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.pivotTable+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.queryTable+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.queryTable+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.revisionHeaders+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.revisionHeaders+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.revisionLog+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.revisionLog+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.sharedStrings+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.sharedStrings+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.sheet<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.sheet>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.main+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.sheetMetadata+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.sheetMetadata+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.styles+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.styles+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.table+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.table+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.tableSingleCells+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.tableSingleCells+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.template<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml-template>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.template.main+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.template.main+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.userNames+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.userNames+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.volatileDependencies+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.volatileDependencies+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.spreadsheetml.worksheet+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.theme+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.theme+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.themeOverride+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.themeOverride+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.vmlDrawing<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.vmlDrawing>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.comments+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.comments+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.document<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.document>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.document.glossary+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.document.glossary+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.endnotes+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.endnotes+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.fontTable+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.fontTable+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.footer+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.footer+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.footnotes+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.footnotes+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.numbering+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.numbering+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.settings+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.settings+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.styles+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.styles+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.template<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml-template>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.template.main+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.template.main+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-officedocument.wordprocessingml.webSettings+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-officedocument.wordprocessingml.webSettings+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-package.core-properties+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-package.core-properties+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-package.digital-signature-xmlsignature+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-package.digital-signature-xmlsignature+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]
vnd.openxmlformats-package.relationships+xml<http://www.iana.org/assignments/media-types/application/vnd.openxmlformats-package.relationships+xml>
[Murata <http://www.iana.org/assignments/contact-people.htm#Murata>]

)

On Fri, Jan 6, 2012 at 2:05 PM, Roy T. Fielding <[email-masked]> wrote:

> > Early media types constrained themselves strictly to describing the
> format (syntax) of the representation (image/png, application/xml).
>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 17 Jan 2012 13:54:21 -0800

On Jan 17, 2012, at 1:27 PM, Ray Polk wrote:

> Thanks for the correction.  To be sure I understand, I'll paraphrase your response:
> 
> Media types are a key used by user agents to look up processing instructions with a registered authority (iana for http).  Though these processing instructions might include anything, many (all?) include format information; further, many include _only_ format information.  As a result, a practice arose of naming the key after the format.
> 
> Perhaps my mistake confused the main point I was trying to make, though.  I wasn't concerned with the number of media types that are applicable to an end point.  My concern was with the ramifications of including data semantics in the processing mechanism along with the format information.

I understood that.  However, that concern left the barn some 20 years ago.

> By combining the two, the number of total registered types can/could increase geometrically.  The vnd.openxmlformats-*+xml formats are an example of that potential.  Imagine if someone registers openjsonformats & openyamlformats.  Then they want both text and binary formats.  The list of ~75 types at the end of this email becomes 400+.

They could, but in practice they don't.  The reason is simply that getting
people to agree on a common processing mechanism to use on receipt of a
message is much harder than registering a type.

> Now imagine the growth as such a practice spreads to other domains AND as the number of popular text formats grows...
> 
> Endpoints and clients dealing with format+datatype media types would be repeating themselves over and over again.  Imagine the Accept headers...
> 

If you have a need for the recipient to process that type is a way
that is different than how they would normally process jpeg, then
the answer is yes.  For example, text/html and text/plain are two
different ways to process the same data.

> Sure, a design that allows endpoints to supply user agents a key to lookup processing instructions maximizes flexibility.  Catch-all text columns, member variables and void pointers all also maximize flexibility.
> 
> When in practice, though, ~100% of those catch-alls include format information.  Isn't it time for a more strongly typed header/column/member?  (to go along witth the catch all)

The format information is not the most significant information needed
by the recipient in order to process a MIME message.  While someone
could come up with a better design, I suppose, it is certainly not
within scope for HTTPbis.  Regardless, since the exact same sequence
of octets *is* processed differently depending on the media type metadata,
it is fair to say that format information is not the key concern even
if most data formats have a type named after them.

....Roy

---

From: Ray Polk <[email-masked]>
Date: Thu, 19 Jan 2012 09:53:53 -0700

Maybe discussion out band is less apt to receive a response?

...and does anyone else have any thoughts in this area?  Am I talking
nonsense?  Or is everyone just scared to gainsay Dr. Fielding?  ;-)

To summarize my position:

Two+ values in one cup/bucket/variable/column/header is bad.

The best place for catch-alls like that are when one cannot anticipate the
required types (like say...when you're first writing a spec/api).  We now
know the sort of instructions most Media Types dictate.

Since all Media Type processing instructions constrain the format.  We
could all greatly benefit from format (syntactic type) (and maybe datatype
(semantic type)) being divorced from the other less uniform processing
instructions.

How would one add that to an existing protocol though?  Addition to the
existing spec?  New RFC?

-Ray

(btw - I don't mean to be negative.  I'm a huge REST/Fielding & HTTP
fanboy.  I just think this aspect may be able to benefit from an update.)

---------- Forwarded message ----------
From: Ray Polk <[email-masked]>
Date: Tue, Jan 17, 2012 at 3:34 PM
Subject: Re: [p3-payload] Media-Type -- Two Values, One Cup Anti-Pattern?
To: "Roy T. Fielding" <[email-masked]>

On Tue, Jan 17, 2012 at 2:54 PM, Roy T. Fielding <[email-masked]> wrote:

> > My concern was with the ramifications of including data semantics in the
> processing mechanism along with the format information.
>

Oh to be 20 years older (how often do you hear that? :D )

> message is much harder than registering a type.

I would posit that it's not done more because it's a bad idea.  The fact
that it's relatively easy to register a type probably leads to spurious
types more than it suppresses them.

> > Now imagine the growth as such a practice spreads to other domains AND
> as the number of popular text formats grows...
> >
> > Sure, a design that allows endpoints to supply user agents a key to
> lookup processing instructions maximizes flexibility.  Catch-all text
> columns, member variables and void pointers all also maximize flexibility.
> >
> within scope for HTTPbis.

Can you suggest a better forum for such a discussion?  A better vehicle for
such an effort?

Regardless, since the exact same sequence
> if most data formats have a type named after them.

Are you referencing your text/html & text/plain example above?  The format
information is THE key.  It dictates the two methods of processing the
data.  The fact that they both apply to the same bits is simply an artifact
of text/xml being a subtype of text/plain.

To prove your point, you'd need the same bits being processed differently
due to instructions _other_ than format; not because of the format.

---

From: "Eric J. Bowman" <[email-masked]>
Date: Thu, 19 Jan 2012 10:25:28 -0700

Ray Polk wrote:
>
> Or is everyone just scared to gainsay Dr. Fielding?  ;-)
> 

Historically, I'd have to say no, that ain't it.  Silence can also mean
consensus has already been achieved on what he's telling you.

-Eric

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 20 Jan 2012 01:01:03 +0700

On 19/01/2012, at 11:53 PM, Ray Polk wrote:

> ...and does anyone else have any thoughts in this area?  Am I talking nonsense?  Or is everyone just scared to gainsay Dr. Fielding?  ;-)
> 

Yes, but (as I think Roy covered), it's not practical to change things now. We're not trying for perfect, just interoperable.

Cheers,

--
Mark Nottingham
http://www.mnot.net/

---

From: Ray Polk <[email-masked]>
Date: Thu, 19 Jan 2012 11:23:59 -0700

I look at this group like the founding fathers of the internet.  To think
founding fathers would settle for "just interoperable" when there's a
chance for "better AND interoperable" makes me kind of sad.

Obviously doing away with the existing paradigm of content negotiation
isn't practical.  I wasn't trying to propose that.  It's not how anyone
makes backward compatible changes anyway, right?

1) One could add spec defined optional headers analogous to those for Media
Type:

Accept-Format / Content-Format
Accept-Data-Type / Content-Data-Type

With IANA registries for each.  Eventually this might become an alternative
to Accept / Content-Type.

2) One could extend Media Types to include the notion of format somehow.
 Add to that the idea of format hierarchies or something.

format/text/xml/atomsvc (don't need the yucky + xml anymore)

Then add an optional spec defined header for datatype OR "additional
processing instructions beyond format"

3) something one of you smarter and more experienced guys than me can think
of

If a better design is possible, isn't it our duty to at least _try_?

-Ray the tryhard

On Thu, Jan 19, 2012 at 11:01 AM, Mark Nottingham <[email-masked]> wrote:

> Yes, but (as I think Roy covered), it's not practical to change things
> now. We're not trying for perfect, just interoperable.
>
> Cheers,
>
>
> --
> Mark Nottingham
> http://www.mnot.net/
>
>
>
>
>

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 20 Jan 2012 15:59:47 +1300

On 20/01/2012 7:23 a.m., Ray Polk wrote:
> I look at this group like the founding fathers of the internet.  To 
> think founding fathers would settle for "just interoperable" when 
> there's a chance for "better AND interoperable" makes me kind of sad.
>
> isn't practical.  I wasn't trying to propose that.  It's not how 
> anyone makes backward compatible changes anyway, right?
>
> 1) One could add spec defined optional headers analogous to those for 
> Media Type:
>
> With IANA registries for each.  Eventually this might become an 
> alternative to Accept / Content-Type.
>
> 2) One could extend Media Types to include the notion of format 
> somehow.  Add to that the idea of format hierarchies or something.
>

IMHO the place for adding such instructions is not in the HTTP tokens, 
but in the fields provided by the representation.

XML itself is perfectly capable of specifying in one of its tags the 
data type that will be produced after unwrapping/processing. So this "+" 
syntax is not actually adding "+xml" to a type as it is more along the 
lines of adding redundant "something+" to the text/xml and 
application/xml types alongside how the XML coded representation stores 
that meta data.

This is in no way the fault of HTTP protocol, but of those designing the 
sub-protocols not taking into account the 2-value repercussions and 
nesting options clearly enough.
... or perhapse they did, and decided that registering a nested type 
"text/a+b" was the best way to go given the nature of how a and b interact.

Either way it seems way beyond the bounds of HTTP to re-define what is 
or is not allowed to be transferred beyond the basic reserved field 
characters.

AYJ

---

From: "Markus Lanthaler" <[email-masked]>
Date: Fri, 20 Jan 2012 12:11:08 +0800

> "text/a+b" was the best way to go given the nature of how a and b

May I just ask a quick question on this. We are currently working on a
JSON-based media type (JSON-LD) and plan to use the application/ld+json
media type for it. Do you think that's a bad practice?

Unfortunately in JSON there's no way to specify such processing instructions
within the representation as you propose. From looking at the current
registered media types and discussions with a number of people (mostly from
the WHATWG) we thought that's the best practice, so I'm a bit confused now.

A related issue we stumbled into is how we define further "subtypes". E.g.
we will need a way to describe a "JSON-LD frame". The first idea was to use
application/frame-ld+json but some people argued that the best practice
would be to use application/frame+ld+json which looks weird to me.

So, in these concrete examples, what would you (and others) propose to use?

P.S.: I fear I will get a response that says: it's an opaque identifier and
it doesn't matter

--
Markus Lanthaler
@markuslanthaler

---

From: "Eric J. Bowman" <[email-masked]>
Date: Thu, 19 Jan 2012 21:42:22 -0700

"Markus Lanthaler" wrote:
> JSON-based media type (JSON-LD) and plan to use the application/ld
> +json media type for it. Do you think that's a bad practice?
> 

I have no idea the status of that media type as a proposed standard.
If it is, then I still don't know if it's bad practice as I have no
idea what your system even does.

>
> Unfortunately in JSON there's no way to specify such processing
> instructions within the representation as you propose.
>

There's more than one way to use JSON in a system, if you need to do
something out-of-scope to JSON, you should consider a different media
type.

>
> From looking at the current registered media types and discussions
> with a number of people (mostly from the WHATWG) we thought that's
> the best practice, so I'm a bit confused now.
> 

>From looking at the string "application/ld+json", even though I don't
know what "ld" means, I know that the sender's intended processing
model is JSON-based, so I can parse it in javascript and get objects,
fwiw.

>
> A related issue we stumbled into is how we define further "subtypes".
> E.g. we will need a way to describe a "JSON-LD frame". The first idea
> was to use application/frame-ld+json
>

If you aren't going to standardize /frame-ld+json, then the appropriate
syntax and registration procedure is for the vnd. tree, i.e.
application/vnd.frame-ld+json.

>
> but some people argued that the best practice would be to use
> application/frame+ld+json which looks weird to me.
> 

The guidelines are being rewritten to include "+json" etc. right now, I
doubt multiple "+" syntax will be allowed.  What you have there isn't
what suffixes are meant to do, at all.

> So, in these concrete examples, what would you (and others) propose

I don't see any concrete examples to base any sort of proposal on, nor
is this the proper list for that, stick with rest-discuss.

>
> P.S.: I fear I will get a response that says: it's an opaque
> identifier and it doesn't matter
> 

I should hope not.  URIs are opaque, media types map to processing
models; the sender's intended processing model for the payload had
better not be opaque or the Web wouldn't work.

-Eric

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Fri, 20 Jan 2012 05:47:14 +0100

>the WHATWG) we thought that's the best practice, so I'm a bit confused now.

The main criticism with "+json" types that I am aware of was that it's a
bit of an ad-hoc convention that unlike "+xml" was undocumented. That is
getting addressed now though and there are people making serious argu-
ments against "+json" types, they are not posting them to `ietf-types`.
There are plenty of people in the process or registering "+json" types,
I would regard using the convention a best practise.

>would be to use application/frame+ld+json which looks weird to me.

The media type meta data is often lost (or ignored, or not checked for),
so I would recommend against relying on its presence for new types, i.e.
it might be better to use one type for all the subtypes and use an in-
ternal discriminator (but I haven't looked at your use case in detail).
If you do need separate types, then a "+ld" would be odd; "+json" would
indicate that you can handle the data if you can handle JSON, but you
would not be able to use the "+ld" in the same way. Besides, it would
seem to me that `application/ld-frame+json` would be more in keeping
with the terminology.

https://www.ietf.org/mailman/listinfo/ietf-types would be the place to
discuss this in any case as far as the media type perspective goes.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Amos Jeffries <[email-masked]>
Date: Fri, 20 Jan 2012 18:06:16 +1300

> the WHATWG) we thought that's the best practice, so I'm a bit confused now.

May or may not be best practice under the type specs. My point was that 
it is under *those* specs rather than HTTP ones that type syntax points 
being argued fall.

> P.S.: I fear I will get a response that says: it's an opaque identifier and

Pretty much, although "don't care" instead of "doesn't matter" (because 
it might matter, for you).

Its a personal preference on my behalf not to like needless duplication. 
The XML case is particularly offensive in that the type says "ZZ+xml" 
then the very first tag in the XML itself says essentially "wrapped ZZ". 
Which is a waste of bytes in the HTTP layer and lead to this threads 
confusion.

AYJ

---

From: "Markus Lanthaler" <[email-masked]>
Date: Fri, 20 Jan 2012 13:54:09 +0800

> ternal discriminator (but I haven't looked at your use case in detail).

Unfortunately that doesn't seem to be possible. All documents will be valid
JSON but their structure is completely different.

> would not be able to use the "+ld" in the same way. Besides, it would

OK, thanks, that was what I thought as well.

> seem to me that `application/ld-frame+json` would be more in keeping

True! Might be a good alternative

> discuss this in any case as far as the media type perspective goes.

Thank you very much for this link.

Eric, let me include the reply to your mail as well in here:

> model is JSON-based, so I can parse it in javascript and get objects,

That's how it works.. but if you have a JSON-LD parser you can do more with
it, e.g., use hyperlinks.

> what suffixes are meant to do, at all.

OK, that was my opinion as well. Do you have a link at hand where I can find
more information about this rewriting effort?

--
Markus Lanthaler
@markuslanthaler

---

From: "Eric J. Bowman" <[email-masked]>
Date: Thu, 19 Jan 2012 23:08:09 -0700

"Markus Lanthaler" wrote:
> 
> OK, that was my opinion as well. Do you have a link at hand where I
> can find more information about this rewriting effort?
> 

http://www.w3.org/2001/tag/products/mimeweb.html

-Eric

---

From: Ray Polk <[email-masked]>
Date: Fri, 20 Jan 2012 07:29:22 -0700

On Thu, Jan 19, 2012 at 7:59 PM, Amos Jeffries <[email-masked]> wrote:

> IMHO the place for adding such instructions is not in the HTTP tokens, but
> in the fields provided by the representation.
>
> XML itself is perfectly capable of specifying in one of its tags the data
> type that will be produced after unwrapping/processing. So this "+" syntax
> is not actually adding "+xml" to a type as it is more along the lines of
> adding redundant "something+" to the text/xml and application/xml types
> alongside how the XML coded representation stores that meta data.
>
> sub-protocols not taking into account the 2-value repercussions and nesting
> options clearly enough.
> Either way it seems way beyond the bounds of HTTP to re-define what is or
> is not allowed to be transferred beyond the basic reserved field characters.
>
> AYJ
>
>
Sure, you can put whatever you like into the entity body.  But how does the
client know to look for it?  Does IANA have a registry of typical xml
bodies?  json bodies?  What's more...how does the client negotiate what
sort of thing they want from the server?

Currently, this is entirely the purview of MediaType -- opaque Media Type
keys referencing sets of instructions registered with IANA; those
instructions always contain format & increasingly contain datatype.

Why make every single sub protocol bear the burden of specifying format?
(all in different, non standard ways)

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 21 Jan 2012 15:03:34 +0700

This is an interesting discussion, but it's getting off-topic for this list.

The apps-discuss list is likely more suitable.

Regards,

On 20/01/2012, at 9:29 PM, Ray Polk wrote:

> I look at this group like the founding fathers of the internet.  To think founding fathers would settle for "just interoperable" when there's a chance for "better AND interoperable" makes me kind of sad.
> 
> Obviously doing away with the existing paradigm of content negotiation isn't practical.  I wasn't trying to propose that.  It's not how anyone makes backward compatible changes anyway, right?
> 
> 1) One could add spec defined optional headers analogous to those for Media Type:
> 
> With IANA registries for each.  Eventually this might become an alternative to Accept / Content-Type.
> 
> 2) One could extend Media Types to include the notion of format somehow.  Add to that the idea of format hierarchies or something.
> 
> Then add an optional spec defined header for datatype OR "additional processing instructions beyond format"
> 
> IMHO the place for adding such instructions is not in the HTTP tokens, but in the fields provided by the representation.
> 
> XML itself is perfectly capable of specifying in one of its tags the data type that will be produced after unwrapping/processing. So this "+" syntax is not actually adding "+xml" to a type as it is more along the lines of adding redundant "something+" to the text/xml and application/xml types alongside how the XML coded representation stores that meta data.
> 
> This is in no way the fault of HTTP protocol, but of those designing the sub-protocols not taking into account the 2-value repercussions and nesting options clearly enough.
> ... or perhapse they did, and decided that registering a nested type "text/a+b" was the best way to go given the nature of how a and b interact.
> 
> Either way it seems way beyond the bounds of HTTP to re-define what is or is not allowed to be transferred beyond the basic reserved field characters.
> 
> AYJ
> 
> 
> Sure, you can put whatever you like into the entity body.  But how does the client know to look for it?  Does IANA have a registry of typical xml bodies?  json bodies?  What's more...how does the client negotiate what sort of thing they want from the server?
> 
> Currently, this is entirely the purview of MediaType -- opaque Media Type keys referencing sets of instructions registered with IANA; those instructions always contain format & increasingly contain datatype.
> 
> Why make every single sub protocol bear the burden of specifying format? (all in different, non standard ways)

--
Mark Nottingham
http://www.mnot.net/