Subject: draft-snell-http-prefer

From: Mark Nottingham <[email-masked]>
Date: Fri, 07 Oct 2011 03:05:45 +0000

Hi James,

I notice that this draft expired a few days ago. Are you still planning 
on persuing it?

If so, it'd be good to motivate the document, as per Roy's comment 
<http://lists.w3.org/Archives/Public/ietf-http-wg/2011JanMar/0294.html>.

A few suggestions:

> Note that the application of a preference by the server MAY affect 
> the caching characteristics of the response.

It'd be good to mention Vary here explicitly.

> 3.  The Preference-Applied Response Header

I'm not sure this is necessary, as the application of the preference 
should be obvious in the response, no?

> The "return-accepted" token indicates that the client prefers that
> the server respond with a 202 Accepted response indicating that the
> request has been accepted for processing.

I think the use case here is that a 202 is preferred *if* the response 
takes too long to generate, by some arbitrary measure (under control of 
the server), not that the 202 is unconditionally preferred over a 200 
(for example).

If so, it'd be good to explain.

> 5.  The "return-content" Preference

I think this needs to explain that if the response isn't normally a 
representation of the resource, you need to include a Content-Location 
header with the resource's URI (see the httpbis drafts).

Cheers,

-- 
Mark Nottingham
http://www.mnot.net/

---

From: James Snell <[email-masked]>
Date: Sun, 16 Oct 2011 23:26:30 -0700

My apologies for missing this Mark, comments inline

On Thu, Oct 6, 2011 at 8:05 PM, Mark Nottingham <[email-masked]> wrote:
> Hi James,
>
> I notice that this draft expired a few days ago. Are you still planning on
> persuing it?
>

Yes.

> <http://lists.w3.org/Archives/Public/ietf-http-wg/2011JanMar/0294.html>.

Definitely noted. I will update the draft with appropriate coverage for this.

One point that I will note (here and in the draft) although the draft
itself currently only defines preferences involving the types of
responses preferred by the client, the extensibility of the header
makes it possible to leverage this mechanism to apply to any
potentially optional feature or behavior employable by the server. One
example that has come up in a real scenario is a server that allows
clients to request a certain level of error handling capability and
detail in an error response. For instance, when PUTing a resource, the
client could ask for "Prefer: strict-validation" or "Prefer:
relaxed-validation", either of which could impact the choices the
server makes when processing the resource and reporting errors.
Obviously the server is still free to do whatever it wants in either
case.

Another real-world use case that has come up is the ability for a
client to request a certain priority for handling a given request. In
the case that came up, if a client wished to request that a server
handle a given request with greater priority than another, it could
include something along the lines of, "Prefer: priority-handling;l=1"
(where l is a non-negative integer [1...*] specifying the handling
priority, 1 being the highest). This obviously only works is fairly
controlled environments, but you get the idea... use of the Prefer
header is in no way bound strictly to the determination of which
response codes and payloads to return.

(Just to note, in this particular case, the priority-handling
preference is combined with the return-accepted preference to
construct what is essentially an asynchronous priority queue for
long-running requests)

A third case that has come up (albeit purely experimental at this
stage) involves the use of Prefer in cometd/long-polling scenarios...
essentially using it as a mechanism for indicating whether the client
would like the server to return a response immediately or hold the
connection until a response is available without requiring the use of
a query string parameters in the request URI. For instance,

GET /my_uri HTTP/1.1
Prefer: no-wait

vs.

GET /my_uri HTTP/1.1
Prefer: wait

Again, the server is obviously free to do what it wants, but in this
particular scenario, when the server received a no-wait preference, it
returned immediately without holding the connection regardless of
whether there was any data to return. When receiving a "wait"
preference, on the other hand, the connection would be held until data
became available or until a certain server specified period of time
had passed.

Obviously, these scenarios are possible through the use and
manipulation of query string parameters, but those, too, have their
own drawbacks.
> A few suggestions:
>
>> Note that the application of a preference by the server MAY affect the
>> caching characteristics of the response.
>

Agreed.

> I'm not sure this is necessary, as the application of the preference should
> be obvious in the response, no?
>

Not necessarily. Imagine, for instance, the case where an API uses a
JSON-encoded resource to report the status of a request as well as to
represent the resource itself. If send Prefer: return-status and get
back a JSON object, I would have no way of knowing prior to parsing
the resource whether the preference was applied. Further, since the
spec explicitly allows a proxy to honor a Preference even if the
origin server does not, the Preference-Applied header can be used by
the intermediary to determine the action taken by the server, to
determine whether it needs to do any additional work at all.

As a side note, the fact that an intermediary can honor a preference
independently also, at least partially, addresses the concern that Roy
raises about whether the mechanism handling the PUT/POST would even be
capable of honoring the Preference. An intelligent proxy could have
mechanisms in place to properly handle the preference regardless of
the capabilities of the mechanism handling the modification.

> I think the use case here is that a 202 is preferred *if* the response takes
> too long to generate, by some arbitrary measure (under control of the
> server), not that the 202 is unconditionally preferred over a 200 (for
> example).
>

Agreed. This is one that definitely needs expansion. I don't believe
we necessarily have to leave it completely under the control of the
server. When specified by itself, e.g. "Prefer: return-accepted"...
then yes, the server decides the maximum acceptable wait time.
However, when specified as "Prefer: return-accepted; wait=1000"
(where "wait" is specified in milliseconds), this would effectively be
interpreted as "If the response will take at least 1000 milliseconds
or more to complete, please return a 202 Accepted Response". Even
specified in this way, as application of the Preferences header is
entirely optional on the part of the server, the server can choose to
do what it wants. For instance, if the server deems the 1000
milliseconds to be to long to wait and returns a 202 after only 500
milliseconds, then it can return a "Preference-Applied:
return-accepted; wait=500" in the response.

> header with the resource's URI (see the httpbis drafts).

Agreed.

> Cheers,

Thanks for the feedback. I'll get that draft updated within the next
day or two.

- James

---

From: "Thomson, Martin" <[email-masked]>
Date: Mon, 17 Oct 2011 14:40:08 +0800

Hi James,

I've been looking at this for a little while and I think that you could address both with an (optional) parameter that sets an upper bound on the wait interval.  This indicates that after this time, the client is likely to close the connection.  I've found some cases where knowing the time limit can affect what the server does in order to produce a result.

Prefer: wait=0 is then equivalent to no-wait.  We've found that being able to set a limit on the wait time is quite useful.  A server (or intermediary) that was aware of the prefer option can then generate a response prior to this time elapsing.

--Martin

---

From: James Snell <[email-masked]>
Date: Mon, 17 Oct 2011 06:36:03 -0700

> Prefer: wait=0 is then equivalent to no-wait.  We've found that being able to set a limit on the wait time is quite useful.  A server (or intermediary) that was aware of the prefer option can then…

Martin, agreed... my feedback to the folks who were putting this
particular implementation together was along the same lines, actually.
In fact, thinking about it further, defining "wait" as a distinct
preference allows us to keep from having to parameterize the
return-accepted preference, we'd simply be able to compose the two:
"Prefer: return-accepted, wait=500".

- James

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Oct 2011 09:42:38 +1100

On 17/10/2011, at 5:26 PM, James Snell wrote:
> potentially optional feature or behavior employable by the server.

Which brings to mind the registry.

Currently, the policy is IESG approval. That's pretty heavyweight, and requires the IESG to know something about Prefer (fairly optimistic ;)

I'd be inclined to make this more something like Specification Required; that's significantly less burdensome (because it doesn't have to be an IETF spec, or even a spec from a standards org), and the time between request and registration can be significantly reduced.

> the resource whether the preference was applied.

If there's a Content-Location header that has the same URI as the resource, it's a representation of that resource. See:
  http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-16#section-6.1

I've always thought of Prefer as a lightweight mechanism that can be used to simplify resources (e.g., so that I don't have to have a "full content" vs. "no content" URI for the same thing). I guess I'm concerned that, by adding an explicit return channel, it's making things potentially more complex; I can see people layering lots of weird things on top of this...

> Further, since the

Whoa. That's kind of a loophole that a truck can drive through. It's one thing to allow a proxy to turn a 200 into a 204; it's another to allow it to act on behalf of the origin server.

This needs to be bolted down; non-transforming intermediaries cannot change things without explicit permission in the protocol, and that's giving them way too much. I'd suggest making this a directive-by-directive decision. 

> the capabilities of the mechanism handling the modification.

I see Roy's concern as already being dealt with by the optionality of the mechanism. I'm much more concerned about giving intermediaries -- especially proxies! -- so much license. 

--
Mark Nottingham   http://www.mnot.net/

---

From: James Snell <[email-masked]>
Date: Mon, 17 Oct 2011 20:26:08 -0700

Comments inline...

Agreed.

> I've always thought of Prefer as a lightweight mechanism that can be used to simplify resources (e.g., so that I don't have to have a "full content" vs. "no content" URI for the same thing). I gues…

Well, again, the Content-Location option only works for that specific
pair of preferences. And yes, Preference-Applied does add complexity
but in some cases there will simply be no way of knowing for sure
whether a preference was honored. In many cases, there simply will not
be any reason to include a Preference-Applied in the response,
regardless of whether it was applied. For instance, in the 200 vs 204
or 202 response code scenarios, as well as the "priority-handling" and
"wait" scenarios.

Thinking about it further just now... perhaps we could get away with
eliminating Preference-Applied and "simply" requiring individual
prefer directives to provide for their own signaling mechanism. Let me
stew on this a bit more and work it through, but I'm starting to feel
swayed on this.

> This needs to be bolted down; non-transforming intermediaries cannot change things without explicit permission in the protocol, and that's giving them way too much. I'd suggest making this a direct…

It's not quite as open-ended as it would look on the surface, which
needs to be called out in the draft. For instance, if I pass along a
"Prefer: return-content", an intermediary is only going to be able to
honor that if it has the ability to retrieve the current
representation of that resource (e.g. from a cache). If I pass along a
"Prefer: return-accepted", it's perfectly reasonable to imagine an
intermediary that implements the async behavior in front of a server
that doesn't support it. Likewise, if I pass along a "Prefer:
priority-handling;l=1" with a request per that async priority queue
example I described, an intermediary choosing the apply that
preference is only doing so within the context of it's own handling of
the request and is not necessarily doing so on behalf of the origin
server. Or, if I pass along a "Prefer: strict-validation" extension,
there'd be no problem with an intelligent intermediary performing the
validation prior to dispatching that request on to the origin server.
In other words, this is not so much a case of the intermediary acting
on behalf of the origin server as it is  making allowances for the
intermediary choosing to honor a preference independently of the
origin server. Hopefully that makes sense, lol. It'll be clearer once
I get a moment to write up the actual description in the spec text
later tonight after these kids are down to bed.

---

From: James Snell <[email-masked]>
Date: Tue, 25 Oct 2011 09:43:15 -0700

Ok... I've published an update

http://www.ietf.org/id/draft-snell-http-prefer-04.txt

I've added much more explanation around the motivation for each
preference directive but I plan on making another round of edits to
tease it out further.

- James