Subject: [i81] Content Negotiation for media types

From: "Javier Godoy" <[email-masked]>
Date: Sat, 3 Nov 2007 17:48:13 -0300

Though I do not have enough authority as for involving in this issue (after
several gurus have supported the opossite resolution to this issue), I would
like to share my point of view about it.

I agree that listing all supported MIME types in each request makes no sense
now-a-days, at least for fully-featured web-browsers (where the 'active
content' approach fits better, or where there is a user interface and the
preferred variant may be choosed before the request is submitted).  From the
server side, I recall that some programming API (such as Java Servlets 2.5)
do not provide an easy handling of the Accept-* header (they left up to the
programmer the parsing of alternatively accepted tokens and sorting them
according to the related q values), thus encouraging Accept to be ignored.

However, I think that an Accept header may still be useful in other contexts
such as web services, where no active-content may be deployed and where the
client is also a process running without human supervision. For instance,
suppose there is a web-service that provides information as XML
(application/xml) conforming an implementation-specific schema, then the
service is enhanced for providing the same information as RDF
(application/rdf+xml), while keeping the XML variant for compatibility
issues. Of course, these services may negotiate the content in a
non-standard way, but I think it is better to keep this feature standardized
(just for using it when it is meaningful).

Therefore, instead of deprecating Accept (which is a too drastic decision),
it might be regarded as OPTIONAL (on the server-side) because "one vendor
may choose to include the item because a particular marketplace requires it
or because the vendor feels that it enhances the product while another
vendor may omit the same item." (RFC 2119). This rewording conforms the
currenct implementation practice, and I think it is enough for "make it
clear to people reading the spec that it doesn't really work that way in
practice".

Furthermore, if the Accept header is not removed from the revised HTTP
specification, the requierment level of a 406 response when the "Accept"
header can not be satisfied (currently it is a "SHOULD" as per section 14.1)
must be replaced by "MAY" since clients are prepared for receiving content
whose type was not included in the Accept header, and if the server does not
support negotiation based on content-type, they usually ignore the Accept
header (i.e they behave as if Accept: */* were specified, instead of
replying with a 406 status code). In this situation, the "valid reasons in
particular circumstances" (RFC 2119) are too common, then it fits better as
a "vendor decision".

Regards

Javier Godoy

---

From: "Larry Masinter" <[email-masked]>
Date: Sat, 3 Nov 2007 23:30:15 -0700

Clearly "deprecate" was hyperbole. (I can say that since I raised the issue
in the first place.) However, Accept headers have a limited domain of
applicability, e.g., when the client has a limited repertoire of types that
it is actually willing to accept, and this is generally not true on 
typical desktop web browsers (maybe some phones might have such a limitation).

The point about changing the 406 requirement so that it matches reality
should also be added to the issue.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Javier Godoy
Sent: Saturday, November 03, 2007 1:48 PM
To: [email-masked]
Subject: [i81] Content Negotiation for media types

Regards

Javier Godoy

---

From: "Larry Masinter" <[email-masked]>
Date: Sat, 3 Nov 2007 23:57:30 -0700

When we were preparing to move HTTP to Draft Standard, we put together
a 'feature list' so that we could check off whether there were at least
two independent interoperable implementations of every feature. 
See:

http://www.w3.org/Protocols/HTTP/Forum/Reports/ 

This experience and others in other working groups led me to 
propose an (experimental) process:

http://tools.ietf.org/id/draft-ietf-newtrk-interop-reports

For an extensive feature set document (based on that idea) see:

http://tools.ietf.org/id/draft-saintandre-xmpp-feature-set

I don't know if the working group wants to take on preparing
a separate document, but when you're reviewing the document
for clarity, it might be useful to review whether the
"feature list" could easily be constructed.

Larry

---

From: Julian Reschke <[email-masked]>
Date: Sun, 04 Nov 2007 15:00:23 +0100

Larry Masinter wrote:
> ...
> ...

Looking at "Accept"...:

"If no Accept header field is present, then it is assumed that the 
client accepts all media types. If an Accept header field is present, 
and if the server cannot send a response which is acceptable according 
to the combined Accept field value, then the server SHOULD send a 406 
(not acceptable) response." -- 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.1.p.8>

and "406"...:

"Note: HTTP/1.1 servers are allowed to return responses which are not 
acceptable according to the accept headers sent in the request. In some 
cases, this may even be preferable to sending a 406 response. User 
agents are encouraged to inspect the headers of an incoming response to 
determine if it is acceptable." -- 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.10.4.7.p.2>

...it seems that the spec needs to be clarified anyway: the description 
of "Accept" says "SHOULD send 406", but the description of 406 makes it 
sound like something OPTIONAL.

I think the latter is closer to reality...

BR, Julian

---

From: "Mike Dierken" <[email-masked]>
Date: Sun, 4 Nov 2007 10:33:52 -0800

> Clearly "deprecate" was hyperbole. (I can say that since I 
> raised the issue in the first place.) However, Accept headers 
> have a limited domain of applicability, e.g., when the client 
> has a limited repertoire of types that it is actually willing 
> to accept, and this is generally not true on typical desktop 
> web browsers (maybe some phones might have such a limitation).
> 
What about the additional context that a tag such as <img> or <script>
provide? 
Although modern desktop Web browsers handle many formats as compared to some
mobile devices (and that is rapidly changing), the context of URI within a
document may give enough hints that a browser would select a sub-set of all
media types for use in the Accept header. Or am I misunderstanding the issue
altogether?

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 12 Nov 2007 17:11:43 +1100

Yes; I've hit this a few times. I think it revolves around the  
interpretation of "acceptable". Consider:

1) Accept: text/html
2) Accept: text/html;q=1, */*;q=0.1
3) Accept: text/html;q=1, */*;q=0

Many people read the spec as saying that #1 and #3 are equivalent. It  
sounds like we're saying #1 and #2 are closer to the mark -- i.e.,  
that the server can choose to send anything back, even if it's not  
listed, as long as the client didn't explicitly say they didn't want  
that to happen. This seems to be in the spirit of server-driven  
negotiation;

> Server-driven negotiation is advantageous when the algorithm for
>    selecting from among the available representations is difficult to
>    describe to the user agent, or when the server desires to send its
>    "best guess" to the client along with the first response (hoping to
>    avoid the round-trip delay of a subsequent request if the "best
>    guess" is good enough for the user).

So, I could see making changes to the requirement level around 406,  
as well as some clarifications generally around the concept of  
"acceptable."

Cheers,

P.S. In looking into this, I did notice how much jumping around the  
spec was necessary. Just food for thought WRT partitioning...

On 05/11/2007, at 1:00 AM, Julian Reschke wrote:

> client accepts all media types. If an Accept header field is  
> present, and if the server cannot send a response which is  
> acceptable according to the combined Accept field value, then the  
> server SHOULD send a 406 (not acceptable) response." -- <http:// 
> greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.1.p.8>
>
> and "406"...:
>
> "Note: HTTP/1.1 servers are allowed to return responses which are  
> not acceptable according to the accept headers sent in the request.  
> In some cases, this may even be preferable to sending a 406  
> response. User agents are encouraged to inspect the headers of an  
> incoming response to determine if it is acceptable." -- <http:// 
> greenbytes.de/tech/webdav/rfc2616.html#rfc.section.10.4.7.p.2>
>
> ...it seems that the spec needs to be clarified anyway: the  
> description of "Accept" says "SHOULD send 406", but the description  
> of 406 makes it sound like something OPTIONAL.
>

--
Mark Nottingham     http://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 12 Nov 2007 21:54:33 +1100

That sounds like a good topic for discussion in Vancouver. Larry,  
will you be coming?

I'm also reminded of the HTTP requirements summary:
   http://www3.ietf.org/proceedings/99mar/I-D/draft-ietf-http-req- 
sum-00.txt

and obliquely of the work at WS-I;
   http://www.ws-i.org/Profiles/BasicProfile-1.0-2004-04-16.html

Even if it weren't official output, this kind of work could help make  
sure that the requirements, roles and features in the document are  
well-defined.

Thanks,

---

From: "Subbu Allamaraju" <[email-masked]>
Date: Sun, 11 Nov 2007 19:31:14 -0800

> typical desktop web browsers (maybe some phones might have such a

and also HTTP client applications (other than browsers) relying on
content-negotiation.

Subbu

---

From: Henrik Nordstrom <[email-masked]>
Date: Mon, 12 Nov 2007 13:50:04 +0100

> that to happen. This seems to be in the spirit of server-driven  

A conditionally compliant server/application can choose to ignore Accept
entirely and send anything back, but not a fully compliant
implementation.

Obeying to the limits set by the Accept header is a SHOULD level
requirement in the definition of Accept. The slightly conflicting
comment in the definition of 406 is just that, a comment. Doesn't say
anything about compliance level of either implementation or which
request headers the server may ignore, and I read that more as a
reminder to client implementers that it's not a MUST level requirement
for servers to obey to the limits set by Accept or any of the other
Accept-* headers.

But in practice most servers implementations is partially conditionally
compliant here, not always looking at the Accept header.

The changes I see as appropriate here is to change the 406 comment to
clarify that it's directed to client implementers and not servers.
Suggested wording:

      Note: HTTP/1.1 servers are in certain situations allowed to
      return responses which are not acceptable according to the
      accept headers sent in the request. Because of this User
      agents are encouraged to inspect the headers of an incoming
      response to determine if it is acceptable 

Or remove the comment entirely as it's not really about the 406 response
as such..

The paragraph immediately following the comment should also be moved to
a different place. It's not really about 406 but how clients should
process responses in general. No suggestion at the moment.

Regards
Henrik

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 20 Nov 2007 17:15:53 +1100

It's interesting you bring the notion of conditional compliance up,  
since one of the charter goals is to "clarify conformance requirements."

Do people think that having a notion of conditional vs. full  
compliance is useful, as currently used in the spec? I've always been  
a little bit uncomfortable with overloading SHOULD, which RFC2119  
defines as

> 3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there
>    may exist valid reasons in particular circumstances to ignore a
>    particular item, but the full implications must be understood and
>    carefully weighed before choosing a different course.

Using SHOULD to determine conformance levels takes flexibility away  
in a lot of cases; sometimes a SHOULD should be a SHOULD, without  
giving someone the stigma of "conditional conformance."

Cheers,

---

From: Henrik Nordstrom <[email-masked]>
Date: Tue, 20 Nov 2007 22:59:55 +0100

On tis, 2007-11-20 at 17:15 +1100, Mark Nottingham wrote:

> Using SHOULD to determine conformance levels takes flexibility away in
> a lot of cases; sometimes a SHOULD should be a SHOULD, without giving
> someone the stigma of "conditional conformance."

Which fits quite nicely with the conditionally compliant criteria,
except that nearly every existing implementation is conditionally
compliant in one way or another..

An implementation not implementing a relevant SHOULD is not behaving
entirely as expected by the RFC, but should still interoperate with any
other compliant implementation without too much trouble. Therefore the
conditionally compliant level.

Regards
Henrik