Subject: Content Negotiation again

From: Larry Masinter <[email-masked]>
Date: Sat, 9 Jan 2010 08:12:33 -0800

HTTPBIS WG issue 81

http://trac.tools.ietf.org/wg/httpbis/trac/ticket/81

W3C TAG action:
http://www.w3.org/2001/tag/group/track/actions/231

My ACTION-231 from W3C TAG was to propose to HTTP-WG to add a sentence
about "same information", since there have been some questions about
what constituted the "same information", e.g.:

>> is the PNG *representation* derived

>> via conneg from the generic resource

>> <http://sw-app.org/sandbox/house>

>> equivalent to the RDF in Turtle?

And after some wordsmithing, my current proposal is:
=====

Note that in all cases, the supplier of representations has the
responsibility for determining which representations might be
considered to be the "same information".

Looking at section 4 of Payload
 http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p3-payload-08.html#rfc.section.4
there was some question about how to integrate my proposed
rewording into the rest of section 4.

So to be more explicit:

I suggest leaving 4.1 and 4.2 alone , except to change
"HTTP/1.1 defines" to "This specification defines", and dropping
section 4.3 entirely, in lieu of the explicit reference to
RFC 2295. (I think originally section 4.3 was written before
2295 was completed, and a reference seems more appropriate
than a summary.)

========================

HTTP responses include a representation which contains information for
interpretation, whether by a human user or for further processing.
Often, the server has different ways of representing the
same information; for example, in different formats, languages,
or using different character encodings.

HTTP clients and their users might have different or variable
capabilities, characteristics or preferences which would influence
which representation, among those available from the server,
would be best for the server to deliver. For this reason, HTTP
provides mechanisms for "content negotiation" -- a process of
allowing selection of a representation of a given resource,
when more than one is available.

This specification defines two patterns of content negotiation;
"server-driven", where the server selects the representation based
upon the client's stated preferences, and "agent-driven" negotiation,
where the server provides a list of representations for the client to
choose from, based upon their metadata. In addition,  there are
other patterns: some applications use an "active content" pattern,
where the server returns active content which runs on the client
and, based on client available parameters, selects additional
resources to invoke. "Transparent Content Negotiation" [RFC 2295]
has also been proposed.

These patterns are all widely used, and have trade-offs in applicability
and practicality. In particular, when the number of preferences or
capabilities to be expressed by a client are large (such as when many
different formats are supported by a user-agent), server-driven
negotiation becomes unwieldy, and may not be appropriate. Conversely,
when the number of representations to choose from is very large, agent-
driven negotiation may not be appropriate.

Note that caches can be instructed on how to manage server-negotiated
responses using the Vary header [ref], although this mechanism is not
richly descriptive.

The headers "accept:", "accept-language:" and "accept-charset:"
are defined explicitly for content negotiation. In addition,
many systems employ "User-Agent" based negotiation, where different
content is delivered depending on the version of software employed.
In practice, User-Agent based negotiation is fragile, because new
clients are not recognized by the deployed servers or active content.

Note that in all cases, the supplier of representations has the
responsibility for determining which representations might be
considered to be the "same information".

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Jan 2010 16:07:18 +0100

Larry,

thanks for the update. I have attached a diff implementing most of this 
to the Trac entry 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/81/i81.diff>.

Looking at the text that would result:

-- snip --
4.  Content Negotiation

    HTTP responses include a representation which contains information
    for interpretation, whether by a human user or for further
    processing.  Often, the server has different ways of representing the
    same information; for example, in different formats, languages, or
    using different character encodings.

    HTTP clients and their users might have different or variable
    capabilities, characteristics or preferences which would influence
    which representation, among those available from the server, would be
    best for the server to deliver.  For this reason, HTTP provides
    mechanisms for "content negotiation" -- a process of allowing
    selection of a representation of a given resource, when more than one
    is available.

    This specification defines two patterns of content negotiation;
    "server-driven", where the server selects the representation based
    upon the client's stated preferences, and "agent-driven" negotiation,
    where the server provides a list of representations for the client to
    choose from, based upon their metadata.  In addition, there are other
    patterns: some applications use an "active content" pattern, where
    the server returns active content which runs on the client and, based
    on client available parameters, selects additional resources to
    invoke.  "Transparent Content Negotiation" ([RFC2295]) has also been
    proposed.

    These patterns are all widely used, and have trade-offs in
    applicability and practicality.  In particular, when the number of
    preferences or capabilities to be expressed by a client are large
    (such as when many different formats are supported by a user-agent),
    server-driven negotiation becomes unwieldy, and may not be
    appropriate.  Conversely, when the number of representations to
    choose from is very large, agent-driven negotiation may not be
    appropriate.

    Note that caches can be instructed on how to manage server-negotiated
    responses using the Vary header (Section 3.5 of [Part6]), although
    this mechanism is not richly descriptive.

    The headers "Accept:", "Accept-Language:" and "Accept-charset:" are
    defined explicitly for content negotiation.  In addition, many
    systems employ "User-Agent" based negotiation, where different
    content is delivered depending on the version of software employed.
    In practice, User-Agent based negotiation is fragile, because new
    clients are not recognized by the deployed servers or active content.

4.1.  Server-driven Negotiation

    If the selection of the best representation for a response is made by
    an algorithm located at the server, it is called server-driven
    negotiation.  Selection is based on the available representations of
    the response (the dimensions over which it can vary; e.g. language,
    content-coding, etc.) and the contents of particular header fields in
    the request message or on other information pertaining to the request
    (such as the network address of the client).

    Server-driven negotiation is advantageous when the algorithm for
    selecting from among the available representations is difficult to
    describe to the user agent, or when the server desires to send its
    "best guess" to the client along with the first response (hoping to
    avoid the round-trip delay of a subsequent request if the "best
    guess" is good enough for the user).  In order to improve the
    server's guess, the user agent MAY include request header fields
    (Accept, Accept-Language, Accept-Encoding, etc.) which describe its
    preferences for such a response.

    Server-driven negotiation has disadvantages:

    1.  It is impossible for the server to accurately determine what
        might be "best" for any given user, since that would require
        complete knowledge of both the capabilities of the user agent and
        the intended use for the response (e.g., does the user want to
        view it on screen or print it on paper?).

    2.  Having the user agent describe its capabilities in every request
        can be both very inefficient (given that only a small percentage
        of responses have multiple representations) and a potential
        violation of the user's privacy.

    3.  It complicates the implementation of an origin server and the
        algorithms for generating responses to a request.

    4.  It may limit a public cache's ability to use the same response
        for multiple user's requests.

    HTTP/1.1 includes the following request-header fields for enabling
    server-driven negotiation through description of user agent
    capabilities and user preferences: Accept (Section 5.1), Accept-
    Charset (Section 5.2), Accept-Encoding (Section 5.3), Accept-Language
    (Section 5.4), and User-Agent (Section 9.9 of [Part2]).  However, an
    origin server is not limited to these dimensions and MAY vary the
    response based on any aspect of the request, including information
    outside the request-header fields or within extension header fields
    not defined by this specification.

    The Vary header field (Section 3.5 of [Part6]) can be used to express
    the parameters the server uses to select a representation that is
    subject to server-driven negotiation.

4.2.  Agent-driven Negotiation

    With agent-driven negotiation, selection of the best representation
    for a response is performed by the user agent after receiving an
    initial response from the origin server.  Selection is based on a
    list of the available representations of the response included within
    the header fields or entity-body of the initial response, with each
    representation identified by its own URI.  Selection from among the
    representations may be performed automatically (if the user agent is
    capable of doing so) or manually by the user selecting from a
    generated (possibly hypertext) menu.

    Agent-driven negotiation is advantageous when the response would vary
    over commonly-used dimensions (such as type, language, or encoding),
    when the origin server is unable to determine a user agent's
    capabilities from examining the request, and generally when public
    caches are used to distribute server load and reduce network usage.

    Agent-driven negotiation suffers from the disadvantage of needing a
    second request to obtain the best alternate representation.  This
    second request is only efficient when caching is used.  In addition,
    this specification does not define any mechanism for supporting
    automatic selection, though it also does not prevent any such
    mechanism from being developed as an extension and used within
    HTTP/1.1.

    HTTP/1.1 defines the 300 (Multiple Choices) and 406 (Not Acceptable)
    status codes for enabling agent-driven negotiation when the server is
    unwilling or unable to provide a varying response using server-driven
    negotiation.
-- snip --

Note that the introduction now contains two paragraphs that appear to be 
specific to server-driven negotiation, and which repeat information 
which already is present in 4.1.:

-- snip new in 4. --
    Note that caches can be instructed on how to manage server-negotiated
    responses using the Vary header (Section 3.5 of [Part6]), although
    this mechanism is not richly descriptive.

    The headers "Accept:", "Accept-Language:" and "Accept-charset:" are
    defined explicitly for content negotiation.  In addition, many
    systems employ "User-Agent" based negotiation, where different
    content is delivered depending on the version of software employed.
    In practice, User-Agent based negotiation is fragile, because new
    clients are not recognized by the deployed servers or active content.
-- snip new in 4. --

and

-- snip already in 4.1 --
    HTTP/1.1 includes the following request-header fields for enabling
    server-driven negotiation through description of user agent
    capabilities and user preferences: Accept (Section 5.1), Accept-
    Charset (Section 5.2), Accept-Encoding (Section 5.3), Accept-Language
    (Section 5.4), and User-Agent (Section 9.9 of [Part2]).  However, an
    origin server is not limited to these dimensions and MAY vary the
    response based on any aspect of the request, including information
    outside the request-header fields or within extension header fields
    not defined by this specification.

    The Vary header field (Section 3.5 of [Part6]) can be used to express
    the parameters the server uses to select a representation that is
    subject to server-driven negotiation.
-- snip already in 4.1 --

I would recommend that we remove these two from the new text, and merge 
the additional bit about "User-Agent" into the text we already in 4.1.

Feedback appreciated,

Julian

---

From: Larry Masinter <[email-masked]>
Date: Tue, 12 Jan 2010 21:30:26 -0800

> the additional bit about "User-Agent" into the text we already in 4.1.

That's fine. I noted the redundancy and thought it was harmless, but
tightening the text is good.

Larry

---

From: Julian Reschke <[email-masked]>
Date: Wed, 13 Jan 2010 15:57:46 +0100

> tightening the text is good.

OK, I have generated a new patch at 
<http://trac.tools.ietf.org/wg/httpbis/trac/attachment/ticket/81/i81.2.diff>.

And below is the full text for Section 4:

-- snip --
4.  Content Negotiation

       Note: In practice, User-Agent based negotiation is fragile,
       because new clients might not be recognized.

    This specification defines the 300 (Multiple Choices) and 406 (Not
    Acceptable) status codes for enabling agent-driven negotiation when
    the server is unwilling or unable to provide a varying response using
    server-driven negotiation.
-- snip --

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Jan 2010 11:06:13 +0100

Julian Reschke wrote:
> Larry Masinter wrote:
>>> I would recommend that we remove these two from the new text, and 
>>> merge the additional bit about "User-Agent" into the text we already 
>>> in 4.1.
>>
> ...

Applied with <http://trac.tools.ietf.org/wg/httpbis/trac/changeset/745>.

Best regards, Julian