Subject: can GET do for QUERY?

From: "[email-masked]" <[email-masked]>
Date: Tue, 28 Apr 2015 23:38:50 +0200

Earlier today it occurred to me [1] that one could perhaps just
use GET as a query verb, by adding the query to the unused GET
body like this:

GET /contacts HTTP/1.1
Host: example.org
Content-Type: text/query
Accept: text/csv

select surname, givenname, email limit 10
<<<<<

where the result value could be the following if the server understood 
the query language:

HTTP/1.1 206 OK
Content-Type: text/csv
Query-Result: yes

surname, givenname, email
Smith, John, [email-masked]
Jones, Sally, [email-masked]
Dubois, Camille, [email-masked]
<<<<<

One would need a header such as Query-Result in order to be able
to distinguish an answer from a server that had understood the query
from one answered by a server who did not understand it, and so just
returned the whole representation. I am thinking of having it 
return a 206 as the result is a partial representation of the remote
resource.

A server that does not understand the query, and does not even
know about the possibility that GET can have a body would just return
the full list of contacts in csv format.

The advantage of this is that a client could send a query asking for
a precise part of any data to any resource on the web, without knowing
in advance if it had the query capability, but still get back a good 
response to the request if the server did not understand the query.

In the case of SEARCH or QUERY that it is unlikely unless one goes through
a well known proxy that the remote resource understands the QUERY so that one
really always would need to start with a HEAD or OPTIONS before sending a
QUERY. And sending a packet to the other side of the world, is pretty slow.

It also ties QUERY nicely into GET semantics.

Just a thought at this point.

Henry

[1] see last paragraph at end of 
 https://lists.w3.org/Archives/Public/ietf-http-wg/2015AprJun/0256.html

Social Web Architect
http://bblfish.net/

---

From: "[email-masked]" <[email-masked]>
Date: Tue, 28 Apr 2015 23:59:18 +0200

I tried it and it works on my server FWIW

-----------
$ telnet bblfish.net 80
Trying 208.64.60.175...
Connected to bblfish.net.
Escape character is '^]'.
GET / HTTP/1.1
Host: bblfish.net
Content-Type: text/query
Accept: text/csv
Content-Length: 42

select surname, givenname, email limit 10
HTTP/1.1 200 OK
Date: Tue, 28 Apr 2015 22:13:22 GMT
Server: Apache/2.2.16 (Unix) mod_ssl/2.2.16 OpenSSL/0.9.8o DAV/2 PHP/5.2.12 mod_fcgid/2.3.4
Last-Modified: Sat, 07 Mar 2015 16:03:44 GMT
ETag: "36eb-1ddd-510b4f2053ea6"
Accept-Ranges: bytes
Content-Length: 7645
Content-Type: text/html

<!DOCTYPE html>
<html xmlns="hhttp://www.w3.org/1999/xhtml"
....
--------------

> On 28 Apr 2015, at 23:38, [email-masked] wrote:
> 
> ------
> -----
> -----
> ------

Social Web Architect
http://bblfish.net/

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 07:04:29 +0200

> I tried it and it works on my server FWIW

Why not use POST then ? Passing a body in GET could confuse caches and
result in wrong data being retrieved. There are reasons why POST is a
different method.

Willy

---

From: Julian Reschke <[email-masked]>
Date: Wed, 29 Apr 2015 08:36:02 +0200

>> I tried it and it works on my server FWIW

The rational for not using GET definitively should be explained in the 
SEARCH spec.

For once: 
<http://greenbytes.de/tech/webdav/rfc7231.html#rfc.section.4.3.1.p.4>

> result in wrong data being retrieved. There are reasons why POST is a

That rational also needs to be in the spec (for instance: SEARCH is 
safe, POST is not).

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 09:01:41 +0200

Hi Julian,

> That rational also needs to be in the spec (for instance: SEARCH is 

Yes good point. My argument was in fact against the misuse of GET but
you're right, let's be precise :-)

Cheers,
Willy

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 10:12:59 +0200

> On 29 Apr 2015, at 07:04, Willy Tarreau <[email-masked]> wrote:
> 

Because POST is about creating a new resource or appending to one. Here
we are doing neither: we are just asking for a Partial Content of the 
resource requested, in the way RFC 7233 specifies it. We just need a
range request field for query result pages.

This is much better and RESTful for a number of reasons:

• these partial representations of the resource can be cached
• these cached representations can be used by query language aware 
  caches to reconstruct the full representation in some circumstances 
  ( just as with paging )
• it avoids URL explosion for partial representations of the same 
  resource, which have no relation to the original resource that 
  you get with POST.  
• This means DELETE, PUT, PATCH on the original resource invalidate
  the caches, which does not happen in the URL explosion case generated 
  by POST
•  With query urls you have a huge number of URLs referring to resources 
 with exactly the same access control rules as the non query resource. 
 This means much more complex access control rules, more work tracking 
 of rights managements, privacy settings, etc...
• Adding the query in the GET body, allows for semantically well defined 
query languages to be used, that could become globally available across the
web. 

For the type of queries I am interested in that provide partial 
representations of the original resource, GET fits the bill. There
is precedent for partial representations too. Essentially we are suggesting
extending the partial representation concept to allow us to specify the
representation using more complex languages than those provided by byte 
ordering.

I hope that helps show that this is not an abuse of GET, it just extends
existing usage.

Henry

Social Web Architect
http://bblfish.net/

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 10:18:38 +0200

On Wed, Apr 29, 2015 at 10:12:59AM +0200, [email-masked] wrote:
> Adding the query in the GET body, allows for semantically well defined 

You may want to extend but you cannot decide to replace existing products
which will not comply with your extensions unfortunately. There *are* many
deployed products which will not cope well with a body in a GET request
for various reasons and while I agree that from a messaging perspective
it should not be a problem, from an interoperability perspective it's
definitely going to be one.

Willy

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 10:35:03 +0200

> On 29 Apr 2015, at 10:18, Willy Tarreau <[email-masked]> wrote:
> 

One shoud be able to verify this. I tested it with the Apache on my server
and it had no problem, even though it knows nothing about the meaning of
GET requests with a body.

One could further and make a crawl of a large distributed section of the web 
with a  normal GET query and the proposed  one and see what types of errors 
appear. The benefits of the extension can then be compared with the gained value. 

But anyway, keep in mind that I don't expect the usage of clients generating 
GET  requests with query content on arbitrary servers to grow that fast. 
Initially requests will mostly be made from clients that know that the 
servers can accept this request. So this should not be an issue.

Henry

Social Web Architect
http://bblfish.net/

---

From: Michael Sweet <[email-masked]>
Date: Wed, 29 Apr 2015 05:46:01 -0700

Henry,

> On Apr 29, 2015, at 1:35 AM, [email-masked] wrote:
>>> ...

The CUPS HTTP server will treat a trailing message body in a GET request as a pipelined request since a GET request is not defined to contain a message body (and we have NEVER seen a client do so...)

Unfortunately, the wording for this has changed between RFC 2616 and 7230:

   [RFC2616 p.33]
   The presence of a message-body in a request is signaled by the
   inclusion of a Content-Length or Transfer-Encoding header field in
   the request's message-headers. A message-body MUST NOT be included in
   a request if the specification of the request method (section 5.1.1)
   does not allow sending an entity-body in requests. A server SHOULD
   read and forward a message-body on any request; if the request method
   does not include defined semantics for an entity-body, then the
   message-body SHOULD be ignored when handling the request.

vs.:

   [RFC7230 p.27]
   The presence of a message body in a request is signaled by a
   Content-Length or Transfer-Encoding header field.  Request message
   framing is independent of method semantics, even if the method does
   not define any use for a message body.

i.e., RFC 2616 forbids including a message body for methods that do not define one, while RFC 7230 allows it.

I would expect overloading GET in this way to cause interop problems.

_________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: Eric Covener <[email-masked]>
Date: Wed, 29 Apr 2015 09:10:23 -0400

> i.e., RFC 2616 forbids including a message body for methods that do not define one, while RFC 7230 allows it.

"A message-body MUST NOT be included in a request if the specification
of the request method (section 5.1.1) does not allow sending an
entity-body in requests."

IMO "does not allow" and "does not define" are quite a bit far apart.

-- 
Eric Covener
[email-masked]

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 15:09:38 +0200

> On 29 Apr 2015, at 14:46, Michael Sweet <[email-masked]> wrote:
> 
> Henry,
> 

Thanks Michael! 

  RFC7230 overrides RFC2616, and I suppose HTTP2.0 is in lign with RFC7230.
So that means in the longer term things should be ok. It may not be wise
to send GET with a query arbitrarily to web sites right now ( without 
first running tests to find out how big a problem it really is) but over
time it should become better and better. 

  Given that I think we can continue exploring the proposition. The aim
is to build something that is correct over the long term, that fits well
with web architecture, that is efficient, etc... This proposal has all
those advantages. 

The advantage over SEARCH [1] are already a few:

0) that it is clear that the request is scoped to the resource being fetched
1) that the fallback to GET with query is just GET, which should make sure that the RESTful nature of the web is respected and (0) above is
2) that it can function nicely with future enhanced caches, whereas SEARCH does not allow responses to be cacheable
3) that in the longer term it will require only 1 connection to a resource to get the quick answer if available, or a full answer if not, whereas SEARCH would always at least require a GET, HEAD or OPTIONS followed by SEARCH  ( unless all SEARCH goes through one mega resource which we want to avoid )
4) it does not require a new verb to be created such as QUERY or to build on SEARCH 

So I must say I am quite surprised by where last week's reasoning has led me to, given that I started out as a proponent of SEARCH - and even implement my server that way - then moved to thinking about a new verb QUERY, only to find that GET seems to do the job!  :-)

Henry

[1] as proposed for example by draft-snell-search-method 
   http://datatracker.ietf.org/doc/draft-snell-search-method/?include_text=1

Social Web Architect
http://bblfish.net/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 29 Apr 2015 15:25:58 +0200

> 0) that it is clear that the request is scoped to the resource being fetched

To me it's not clear at all.

> 1) that the fallback to GET with query is just GET, which should make sure that the RESTful nature of the web is respected and (0) above is

How would that fallback be helpful?

> 2) that it can function nicely with future enhanced caches, whereas SEARCH does not allow responses to be cacheable

Caching GET+body is exactly as hard as caching SEARCH.

> 3) that in the longer term it will require only 1 connection to a resource to get the quick answer if available, or a full answer if not, whereas SEARCH would always at least require a GET, HEAD or…

SEARCH doesn't require any HEAD, GET or OPTIONS requests. Also, are you 
confusing "connection" with "number of requests"?

> 4) it does not require a new verb to be created such as QUERY or to build on SEARCH

In one case you update definition of GET, in the other case you update 
the definition of SEARCH. The latter looks much easier (process-wise) 
and disruptive (deployment-wise) to me.

> So I must say I am quite surprised by where last week's reasoning has led me to, given that I started out as a proponent of SEARCH - and even implement my server that way - then moved to thinking a…

/me is surprised as well :-)

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 29 Apr 2015 15:35:14 +0200

> i.e., RFC 2616 forbids including a message body for methods that do not define one, while RFC 7230 allows it.

For good reason, see the history in 
<http://tools.ietf.org/wg/httpbis/trac/ticket/19>.

> I would expect overloading GET in this way to cause interop problems.

Indeed.

Best regards, Julian

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 15:43:27 +0200

> On 29 Apr 2015, at 15:25, Julian Reschke <[email-masked]> wrote:
> 

It has to as per definition of GET .

> How would that fallback be helpful?

Because since the client is only interested in download speed optimisation with the with GET + Query proposal - the client wants to get at some part of the information quickly - then if he receives instead the full document he still gets the information, just not as fast.

> Caching GET+body is exactly as hard as caching SEARCH.

In draft-snell-00 it says "The response to a SEARCH request is not cacheable." 
And in our discussions this week I did not see any movement on that point, though I questioned it.
So it looks rather that caching is impossible with SEARCH but built in to GET.

> SEARCH doesn't require any HEAD, GET or OPTIONS requests. Also, are you confusing "connection" with "number of requests"?

yes, sorry, it requires one more request. 

That is important though. If you count the fastest clock cycle of a CPU and map it to the fastest human clock cycle, say 1 second, then the time it would take on that frame of reference to send a packet from California to Europe and back would be 4 years. I am saying this to help get an idea of the latency differences between in CPU cache memory access and going over the web.

Extra requests are wasteful and should be avoided.

> In one case you update definition of GET, in the other case you update the definition of SEARCH. The latter looks much easier (process-wise) and disruptive (deployment-wise) to me.

We're not proposing to updating GET, just enhancing the existing Partial Content feature. 

> /me is surprised as well :-)

I let myself be driven by where arguments lead me to :-)

Social Web Architect
http://bblfish.net/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 29 Apr 2015 15:53:41 +0200

> It has to as per definition of GET .

As far as I can tell, scoping for GET and SEARCH is exactly the same, 
except that in SEARCH, the scope can be defined in the body (not only 
the URI or request header fields).

I gave the example of a GET on google.com; can you explain what the 
material difference is?

> Because since the client is only interested in download speed optimisation with the with GET + Query proposal - the client wants to get at some part of the information quickly - then if he receives…

If you consider a query to be just a speed optimization compared to 
"getting all", sure. I don't how that's helpful though.

> So it looks rather that caching is impossible with SEARCH but built in to GET.

It's a draft. And caching of responses to GET request with bodies would 
need to be both defined and implemented as well. It's a change to the 
standard HTTP caching model (so good luck with that...).

> yes, sorry, it requires one more request.

Which?

> Extra requests are wasteful and should be avoided.

See above.

> We're not proposing to updating GET, just enhancing the existing Partial Content feature.

"A payload within a GET request message has no defined semantics; 
sending a payload body on a GET request might cause some existing 
implementations to reject the request." -- 
<http://greenbytes.de/tech/webdav/rfc7231.html#rfc.section.4.3.1.p.4>

Defining semantics for GET bodies *is* a change to RFC 7231.

Best regards, Julian

---

From: Michael Sweet <[email-masked]>
Date: Wed, 29 Apr 2015 08:15:31 -0700

Eric,

> On Apr 29, 2015, at 6:10 AM, Eric Covener <[email-masked]> wrote:
> 

Regardless of how you want to interpret RFC 2616, RFC 723x is now the "law of the land" WRT HTTP, and it does not make this statement - I simply pointed out that this is a change between the two specs, and that there is a lot of room for interpretation in both specs.

GET is not defined as accepting a message body in the request, in either RFC 2616 or RFC 7231. And in fact RFC 7231 has the following to say:

   A payload within a GET request message has no defined semantics;
   sending a payload body on a GET request might cause some existing
   implementations to reject the request.

which supports what I have said so far: doing a GET with a request body is probably not a good idea without some way for the server to indicate that such semantics are supported by the server's GET implementation.

If you want to extend GET for searches/queries, you need to do it in a way that doesn't break existing servers and the semantics of GET - probably through a new header that allows the client to discover the server supports the extension, with a corresponding Content-Type in the GET request to indicate the type of query present in the message body.

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 17:32:58 +0200

On Wed, Apr 29, 2015 at 08:15:31AM -0700, Michael Sweet wrote:
> GET is not defined as accepting a message body in the request, in either RFC
> 2616 or RFC 7231. And in fact RFC 7231 has the following to say:
> 
> which supports what I have said so far: doing a GET with a request body is
> probably not a good idea without some way for the server to indicate that
> such semantics are supported by the server's GET implementation.

... and all the intermediaries, which is often where most of the problems
will lie. I can tell you for sure that if you try to pass that through an
old haproxy 1.3, only the headers will pass, the body will wait, and the
server will wait for the data until haproxy declares a server timeout.

Fortunately newer versions are much more standards-compliant, but such
products are still found in field and even naive implementations of
non-compliant intermediaries are found at many places.

The purpose of RFC723x was to tell people what's known to be broken and
what is safe. It doesn't mandate what people have to do, just what they
should do to avoid shooting themselves in the foot. If there are people
who prefer to try a few bullets before knowing if their feet are *that*
sensible, we can't do anything more to warn them :-/

Regards,
Willy

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 17:32:33 +0200

> On 29 Apr 2015, at 17:15, Michael Sweet <[email-masked]> wrote:
> 
> Eric,
> 

yes, something like the Accept-Patch header defined in RFC-5987 [1] or the Accept-Post header defined in LDP [2] . One could define an Accept-Get header returned on Options, HEAD and GET. 

Extending behavior always comes by extending parts that are undefined. This can always lead to breakages at first. To avoid the breakages either clients have to know out of band that a or service supoorts it, or an Accept-GET header is required. In the much longer term the fact that this behavior is defined, should make it possible to avoid this extra step. 

Henry

[1] https://tools.ietf.org/html/rfc5789
[2] http://www.w3.org/TR/ldp/#header-accept-post

Social Web Architect
http://bblfish.net/

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 17:38:21 +0200

> On 29 Apr 2015, at 17:32, Willy Tarreau <[email-masked]> wrote:
> 

One way to sidestep broken old proxies, is what we need to do anyway in our
project is to use https, which is gaining momentum and adoption, even with
its flaws. We are trying to build distributed secure Social Web, and in order
to maintain privacy and security we need https. So those types
of proxies are not a problem to us. We are still very interested in proxy behavior
because we want to allow every citizen to have his own proxy on his Freedom Box.

Henry

Social Web Architect
http://bblfish.net/

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 17:43:18 +0200

On Wed, Apr 29, 2015 at 05:38:21PM +0200, [email-masked] wrote:
> of proxies are not a problem to us.

For *your* case maybe but when you want to extend a standard (and I'd say
that there's nothing older in HTTP than the GET request which predates
headers and proxies), you have to think wide, very wide.

> We are still very interested in proxy behavior

Yes and local anti-virus agents deployed on the PC, accessing the traffic
in the browser before it's encrypted and which are well-known for not
following standards and causing false bug reports.

Willy

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 18:08:16 +0200

> On 29 Apr 2015, at 17:43, Willy Tarreau <[email-masked]> wrote:
> 

yes of course. Research is needed here, as I pointed out previously. 
The situation may not be as bad as feared ( My Apache server worked fine 
for example ).  This research will then help work out what the deployment
strategy has to be. 

Is this hypothetical problem the only one? Perhaps we can put this aside
for the time being and see if there are other issues that are problematic. 

Henry

Social Web Architect
http://bblfish.net/

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 18:21:18 +0200

> yes of course. Research is needed here, as I pointed out previously. 

Research *was done* already which led to the warning in the RFC. It also
led to people considering that they don't want to waste their time
implementing HTTP/2 in clear not sending websocket on port 80 because
some broken intermediaries are there. Sure these ones will fade away,
and the upgrade protocols involved there are designed to support a
graceful fallback in case of failure. Despite this there's some breakage
in field. What you're trying to do relies on something deeply burried in
the roots of HTTP which faces even more potential for breakage and
security issues (eg: send a second request in the body and cross fingers
for caches not to take it and inject the first response's contents for
the second request).

> for example ).

Sure, your Apache server worked fine. It's easy to spot standards compliant
products which have no problem parsing a message complying with standards.
It's much harder to spot the bad ones because in general they're much less
visible (which is why we call them interception proxies). Sometimes it will
be your load balancer, sometimes your firewall, sometimes a compression
module in your server, sometimes a URL filter. Just two weeks ago I had
a customer complain that haproxy was preventing his POST payload from
reaching the server and causing timeouts. The network trace revealed that
it indeed never left the client... He was running a broken "bandwidth
limitation" product on the PC (that's a nice way to save bandwidth).
That's just an example of all the crap you can find in field which will
take years to get detected and even more to get fixed. And even those
which used to be standard-compliant for some time would now be broken
by your extension, leading to even more confusion.

> This research will then help work out what the deployment
> strategy has to be. 
> 
> Is this hypothetical problem the only one?

What is hypothetical is not the problem but your belief that you can
inventory all cases of trouble in a finite time by doing some research.

> Perhaps we can put this aside

Well you can use it between your own components, but you'll have a hard
time explaining to all your users that their freshly acquired products
which used to respect standard still 2015 are not standard-compliant
anymore past 2015. At least with Upgrade we had the ability to play on
the compliance side to convince vendors to fix their crap and sell their
fixes as feature updates ("support for websocket and HTTP/2").

Willy

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 18:53:36 +0200

> On 29 Apr 2015, at 18:21, Willy Tarreau <[email-masked]> wrote:
> 

yes, for that and many other reasons we need to use http/2 too.

> by your extension, leading to even more confusion.

I agree, crap is out there.

But in any case this crap is found there are potentially a number
of fallback positions. 

> inventory all cases of trouble in a finite time by doing some research.

I was not thinking about other deployment problems. I was thinking of
architectural problems and logical flaws with this proposal. I like to
build things that are correct at all levels.

I mean if the logical and architectural problems are all solved, then
fine we can look at the deployment issues. But it would be silly to waste
time on the research if even after we gathered all the information there we'd
find that architectural problems remain.

But hey if you and everybody else here thinks that deployment is the only
issue left here, then that would be good to know. :-)

> fixes as feature updates ("support for websocket and HTTP/2").

Again defining something that is undefined does not break a standard.
The latest spec says that GET can have Content-Length header. Presumably
the people allowing this had their reasons. Perhaps this is it.

Social Web Architect
http://bblfish.net/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 29 Apr 2015 19:04:59 +0200

> the people allowing this had their reasons. Perhaps this is it.

Or perhaps it is not. The "people allowing this" are over here, and we 
have an issue tracker related to that change that I already pointed to 
(<http://trac.tools.ietf.org/wg/httpbis/trac/ticket/19>).

Best regards, Julian

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 29 Apr 2015 19:08:40 +0200

> issue left here, then that would be good to know. :-)

The difference between theory and deployment is covered by the difference
in number of pages between RFC2616 and RFC723x. Without that, the standard
would be much much simpler and cleaner, and there would be almost no SHOULD.

I really think you're minimizing the size of the problem here.

> the people allowing this had their reasons. Perhaps this is it.

The first reason is to simplify the implementations : when you see that
you have no reason for doing a special case of GET when you already support
POST, PUT and any other method supporting a body, you ensure that it works
just like the other ones. Conversely, when you don't expect to need a body,
you simply don't implement this at all and whatever follows the headers is
processed in the easiest way for your implementation. I used to see a NAS
segfault when receiving a body after a GET request or even pipelined GET
requests, I've seen products block the body (and wrote one which used to),
others will silently drop it. In fact you're very lucky when you get an
error because you can react quickly. All other cases are painful because
users say "it doesn't respond, even after one minute" and nobody knows
where to look at.

Willy

---

From: Zhong Yu <[email-masked]>
Date: Wed, 29 Apr 2015 12:12:11 -0500

> issue left here, then that would be good to know. :-)

It appears to me that the only problem SEARCH solves(or rather,
avoids) is the limitation of URI length. That *is* a deployment issue.

So GET is fine, it's just constrained by the current deployments for
your use cases. You want a workaround, which, nevertheless, works
through *the current deployment*.

Zhong Yu
bayou.io

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 20:00:58 +0200

> On 29 Apr 2015, at 19:12, Zhong Yu <[email-masked]> wrote:
> 

Ok, we do have a workaround. Andrei Sambra and Sandro Hawke who are closely
with Tim Berners Lee at the Distributed Information Group at MIT, have started
working a couple of weeks ago on  a proposal for a platform called SoLiD 
( Social Linked Data )  with which we can build the disitrubted secure social web.
This is described in a very early form in this document

  https://github.com/linkeddata/SoLiD

There they have been playing with moving the query to the headers presumably
to avoid the problem discussed here. The query looks like this:

GET /data/ HTTP/1.1
Host: example.org
Query: SELECT * WHERE { ?s ?p ?o . }

and the response looks like this:

HTTP/1.1 200 OK

{
  "head": {
    "vars": [ "s", "p", "o" ]
  },
  "results": {
    "ordered" : false,
    "distinct" : false,
    "bindings" : [
      {
        "s" : { "type": "uri", "value": "https://example.org/data/" },
        "p" : { "type": "uri", "value": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" },
        "o" : { "type": "uri", "value": "http://www.w3.org/ns/ldp#BasicContainer" }
      },
      {
        "s" : { "type": "uri", "value": "https://example.org/data/" },
        "p" : { "type": "uri", "value": "http://purl.org/dc/terms/title" },
        "o" : { "type": "literal", "value": "Basic container" }
      }
    ]
  }
}

I imagine we could use this as a fall back position to work around issues
if they appear. One would need to specify the query mime type too in a header
of course.

This would limit the query sizes quite dramatically, and does not feel as good
as the GET with body. So if we were to specify this in an RFC one could have
either a fallback RFC for the intermediate stage, or specify the fallback position
in the GET + query in the body RFC.

Would that work?

Henry Story

Social Web Architect
http://bblfish.net/

---

From: Julian Reschke <[email-masked]>
Date: Wed, 29 Apr 2015 21:48:19 +0200

> and the response looks like this:

There are constraints on header field sizes as well. So this may work a 
*bit* better than URI parameters, but it will hit some limit as well.

The result will need to vary on "Query", so it won't be cached a lot.

Also don't forget to think about non-ASCII characters in the header field.

> in the GET + query in the body RFC.

Or you could use SEARCH :-)

Best regards, Julian

---

From: Michael Sweet <[email-masked]>
Date: Wed, 29 Apr 2015 12:52:15 -0700

If "global" scope on servers (or the web, depending on the server) is OK, would defining a well-known URI path be useful here?  E.g., "GET /.wellknown/search?q=string"?

> On Apr 29, 2015, at 10:12 AM, Zhong Yu <[email-masked]> wrote:
>

---

From: "[email-masked]" <[email-masked]>
Date: Wed, 29 Apr 2015 23:54:03 +0200

> On 29 Apr 2015, at 21:48, Julian Reschke <[email-masked]> wrote:
> 
> There are constraints on header field sizes as well. So this may work a *bit* better than URI parameters, but it will hit some limit as well.
> 

Thanks. 
Can one make the result vary on the body too?

> Or you could use SEARCH :-)

yes, SEARCH is another option. I have a feeling that we've pretty much covered the ground
in the last week here. 

I find GET more elegant for the moment long term but SEARCH can work too. I wonder
what the consensus is like on the WG.

I wonder also what we can do until the IETF meeting in Prague to make progress on this.

Henry

Social Web Architect
http://bblfish.net/

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 30 Apr 2015 00:18:52 +0200

> On 29 Apr 2015, at 19:08, Willy Tarreau <[email-masked]> wrote:
> 
>> 

It would not matter if servers blocked or dropped the body, since the client
would still get the full response and this proposal is that if it understood
the body it would just send partial content. This is no different from a client
asking for Partial Content and receiving instead the full content in one go.
Also the fact that you had to fix software that failed when receiving bodies
in GET means that you were not the only one to do that, so that most software
actually can withstand these now, or would have gone out of buisness from this.

Anyway as mentioned there is a fallback position using headers which would
look something like this

GET /People/Berners-Lee/card HTTP/1.1
Host: www.w3.org
Query: DESCRIBE <http://www.w3.org/People/Berners-Lee/card#i> 
Query-Lang: application/sparql-update
Vary: Query

That is I think the type of query that will be the most used in the
applications I am working on. If the client gets the whole body back,
that is not really a problem. The Query is there just to get the minimal
bounded graph around Tim Berners Lee's WebID, ie a subset of the
original graph.

This is completely resource bound, there is no change of scope,
and so it satisfies Roy Fielding's requirements. It satisfies your
requirement that there is no body sent. It also is efficient since
a client can make the GET request on a newly discovered resource 
and receive either the partial representation that it would prefer 
for efficiency sake or the full representation by a non query aware server. 
All of this without making a extra request to the server, which is important
for very strongly peer to peer usages which we need for the distributed
social web, if we don't all want to live forever in a police state (
we can discuss the implictions of this and how it relates to Snowden's
revelation around a beer in Prague ). The fact that both of these results
is ok for the client is in fact the proof that this is functioning
RESTfully.

For those who wish to live more dangerously, or who need more space for 
larger queries, putting the content in the body would be another option. 
If we formalise it now we can expect that in 5 to 10 years time it may 
indeed work perfectly.

Does this answer your concerns?

Henry

Social Web Architect
http://bblfish.net/

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 30 Apr 2015 00:25:33 +0200

> asking for Partial Content and receiving instead the full content in one go.

I'm not speaking about servers blocking the body, but intermediaries, leading
to servers not receiving the body they're waiting for prior to respond.

> actually can withstand these now, or would have gone out of buisness from this.

The problem is that it's easy to fix software in two places :
  - client side : users complain, they're directed to an update and that's done
  - server side : developers want their application there and request a server
    update to be compatible with the new feature.

And in the middle (ie: from bogus browser plugins and MITM anti-virus to
transparent proxies, CDNs, server-side SSL-offloaders, caches and load
balancers), that's a completely different story because these components
are shared between many users/customers, they work very well for everyone
except your extension so there's no compelling reason for their owners to
request a fix and risk their update on a upgrade.

Willy

---

From: Julian Reschke <[email-masked]>
Date: Thu, 30 Apr 2015 07:20:01 +0200

> Can one make the result vary on the body too?

It essentially requires extending the HTTP caching model. But when you 
do so, it really doesn't matter whether you include just the body, or 
method + body.

> I wonder also what we can do until the IETF meeting in Prague to make progress on this.

We don't need any F2F meetings for this; just write drafts and discuss 
them over here.

Best regards, Julian