Subject: Multi-GET, extreme compression?

From: Phillip Hallam-Baker <[email-masked]>
Date: Sat, 16 Feb 2013 13:01:59 -0500

HTTP 1.1 has a request/response pattern. This covers 90% of needs but means
that if the protocol is followed correctly forces a round trip delay on
each content request. Which of course leads to various browsers pushing the
envelope and pushing multiple requests out before responses have come back.

With content streams this is not necessary of course... In fact that is
pretty much the purpose of having streams.

Which suggests a need for a Multi-GET method to allow a request for a list
of content...

If we had such a method then the format would be something like

MGET <Common Headers> List <URI, Content header>

And the typical communication pattern of a browser would be:

GET /toplevel.html
MGET </image1.jpg /image2.jpg ...>

Given this particular communication pattern which has an implicit delta
encoding, do we really need to worry about a separate delta encoding?

-- 
Website: http://hallambaker.com/

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sat, 16 Feb 2013 10:22:58 -0800

This sounds very similar to Willy's proposal:
http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/0324.html
Willy and I go back and forth about it:
http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/0333.html

On Sat, Feb 16, 2013 at 10:01 AM, Phillip Hallam-Baker <[email-masked]>wrote:

> HTTP 1.1 has a request/response pattern. This covers 90% of needs but
> means that if the protocol is followed correctly forces a round trip delay
> on each content request. Which of course leads to various browsers pushing
> the envelope and pushing multiple requests out before responses have come
> back.
>

---

From: Cyrus Daboo <[email-masked]>
Date: Sun, 17 Feb 2013 14:18:33 -0500

Hi Phillip,

> encoding, do we really need to worry about a separate delta encoding?

We added a multiget REPORT to CalDAV (RFC4791) and CardDAV (RFC6352) which 
is used by clients when sync'ing a lot of resources (e.g., initial account 
setup). The one major criticism has been lack of cacheability of the 
individual resources included in the multiget.

So in your example, if one client did MGET (image1, image2) and another did 
GET (image1) then ideally the second could be returned by the cache. So 
would we expect caches to be able to "split-up" the resources in an MGET so 
it could serve them up individually for a GET, or in some other combination 
for a different MGET?

-- 
Cyrus Daboo

---

From: Helge Heß <[email-masked]>
Date: Sun, 17 Feb 2013 12:15:14 -0800

On Feb 17, 2013, at 11:18 AM, Cyrus Daboo <[email-masked]> wrote:
> We added a multiget REPORT to CalDAV (RFC4791) and CardDAV (RFC6352) which is used by clients when sync'ing a lot of resources (e.g., initial account setup). The one major criticism has been lack of cacheability of the individual resources included in the multiget.

The other major criticisms being:
a) content needs to be XML encoded
b) only allows for GETs, not for other operations

I'd also like to see a generic, HTTP-level BATCH request. Please lets not do 'just' an MGET.

hh

---

From: James M Snell <[email-masked]>
Date: Sun, 17 Feb 2013 15:07:08 -0800

An mget that leverages server push in http/2 and individual cacheable
response streams would be very interesting and could address at least some
of the prioritization issues.
On Feb 17, 2013 12:17 PM, "Helge Heß" <[email-masked]> wrote:

> > We added a multiget REPORT to CalDAV (RFC4791) and CardDAV (RFC6352)
> which is used by clients when sync'ing a lot of resources (e.g., initial
> account setup). The one major criticism has been lack of cacheability of
> the individual resources included in the multiget.
>

---

From: Roberto Peon <[email-masked]>
Date: Sun, 17 Feb 2013 15:19:47 -0800

MGET (or whatever batch request) implies you know all of what you're
requesting when you're requesting it, which is rarely the case.
As a result, my guess is that this won't solve the prioritization issue for
the browser.
-=R

---

From: Dr Robert Mattson <[email-masked]>
Date: Mon, 18 Feb 2013 11:47:32 +1100

Hey Guys,

What do you think of the request compression and response multiplexing scheme advocated by HTTP-MPLEX?

Rob

Sent on a mobile device, typos expected.

On 18/02/2013, at 10:07 AM, James M Snell <[email-masked]> wrote:

> An mget that leverages server push in http/2 and individual cacheable response streams would be very interesting and could address at least some of the prioritization issues.
>

---

From: James M Snell <[email-masked]>
Date: Sun, 17 Feb 2013 16:58:11 -0800

A requester does not necessarily need to know everything they are
getting in advance.

That is, assume that a server defines a set of N resources and assigns
that set a singular url that represents the entire collection. When I
do..

  MGET /resource/set HTTP/2.0

The server responds by opening N server-push response streams back to
the client, each associated with the original MGET. Each would have
it's own Content-Location and Cache-Control mechanisms allowing
intermediate caches to still do the right thing. The client does not
necessarily know what all it is getting from the server in advance but
knows it needs to be prepared to handle multiple items.

Alternatively, the MGET could include multiple individual URLs, in
which case it still actually behaves the same way. The only difference
is that the client has a better understanding of exactly what it's
wanting to retrieve.

example:

MGET /assets/*.js HTTP/2.0   --> Get all the javascript files
MGET /assets/*.png HTTP/2.0  --> Get all the image files

If the cache is able to keep track of exactly which resources were
pushed in response to the MGET, then caches could keep right on doing
the right thing.

Since the resources are sent via server push, the server is
responsible for determining the priority for which resources get sent.

Yes, largely theoretical and easily abused for sure. But ought to at
least be something worth investigating.

- James

On Sun, Feb 17, 2013 at 3:19 PM, Roberto Peon <[email-masked]> wrote:
>>> > account setup). The one major criticism has been lack of cacheability of the

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Mon, 18 Feb 2013 02:12:27 +0100

>envelope and pushing multiple requests out before responses have come back.

(Pushing the envelope, as per RFC 2068, published in January 1997...)
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Dr Robert Mattson <[email-masked]>
Date: Mon, 18 Feb 2013 12:21:38 +1100

Hi James,

A similar idea was proposed at www01, i think it was called 'n for the price of 1: bundling web objects for more ...'  Can't recall the rest. Google is your friend here.

Rob

---

From: Helge Hess <[email-masked]>
Date: Sun, 17 Feb 2013 17:22:53 -0800

On Feb 17, 2013, at 3:19 PM, Roberto Peon <[email-masked]> wrote:
> MGET (or whatever batch request) implies you know all of what you're requesting when you're requesting it, which is rarely the case.

Well, it's essentially always the case for DAV (CalDAV, CardDAV, etc) and other web/REST services. It might not be the case for browsers.

hh

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 17 Feb 2013 17:30:32 -0800

I'm having difficulty grokking this proposal. Can you describe more clearly
how this would work with the web platform? Specifically, what kind of
markup in a HTML document should cause a browser to use a MGET for a
resource set as you describe it.

Also, how does this work for HTTP/1.X? Since we'll be living in a
transitional world for awhile, I'd like to understand how this allows for
HTTP/1.X semantics backwards compatibility.

On Sun, Feb 17, 2013 at 4:58 PM, James M Snell <[email-masked]> wrote:

> >>> I'd also like to see a generic, HTTP-level BATCH request. Please lets
> not
> >>> do 'just' an MGET.
> >>>
> >>> hh
> >>>
> >>>
> >
>
>

---

From: Helge Hess <[email-masked]>
Date: Sun, 17 Feb 2013 17:38:13 -0800

On Feb 17, 2013, at 5:30 PM, William Chan (陈智昌) <[email-masked]> wrote:
> I'm having difficulty grokking this proposal. Can you describe more clearly how this would work with the web platform? Specifically, what kind of markup in a HTML document should cause a browser to use a MGET for a resource set as you describe it.

? e.g. <img>, <script>, CSS links etc.

> Also, how does this work for HTTP/1.X? Since we'll be living in a transitional world for awhile, I'd like to understand how this allows for HTTP/1.X semantics backwards compatibility.

An old server would return a 405 when the BATCH comes in, then the client needs to switch to performing the operations individually.

hh

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 17 Feb 2013 17:52:34 -0800

On Sun, Feb 17, 2013 at 5:38 PM, Helge Hess <[email-masked]>wrote:

> > I'm having difficulty grokking this proposal. Can you describe more
> clearly how this would work with the web platform? Specifically, what kind
> of markup in a HTML document should cause a browser to use a MGET for a

I'm confused. We issue individual GETs for the individual resource URLs.
How do we know to combine those individual resources into this magical
/resource/set path?

Furthermore, as I previously linked to in the very first reply to the
thread, when we discussed MGET previously, I highlighted how the browser
incrementally parses the document and sends GETs for resources as it
discovers them. Since my browser does not have a crystal ball telling it
that more resources are coming and when they are coming, the browser simply
issues GETs as soon as it can (subject to some other constraints like DNS
and available TCP sockets and some weak attempts at resource scheduling).

> An old server would return a 405 when the BATCH comes in, then the client
> needs to switch to performing the operations individually.
>

So, you handwaved over how the client would magically transform URL1 + URL2
+ URL3 into magical example.com/resource/set. Assuming that's possible, how
do you do the reverse transformation, when a HTTP/2=>HTTP/1.X gateway needs
to translate HTTP/2 MGET requests for this /resource/set into the
individual GETs for the original URLs. And even if this is possible, how
reasonable is it to pay this roundtrip on receiving the 405? We've fought
really hard to eliminate roundtrips.

---

From: Helge Hess <[email-masked]>
Date: Sun, 17 Feb 2013 18:10:21 -0800

On Feb 17, 2013, at 5:52 PM, William Chan (陈智昌) <[email-masked]> wrote:
> I'm confused. We issue individual GETs for the individual resource URLs. How do we know to combine those individual resources into this magical /resource/set path?

Well, I personally don't care too much about HTML here but about services, but I do think you can use the facility for this too. The browser would need to do some clever batching and latency management, but thats not really related to HTTP but HTML and an issue in any protocol.
Fixing HTML would be a different thing, but sure, you could introduce resource-set tags which would directly map to batched requests.

Presumably you receive your HTML in a streamed fashion as packets arrive, presumably parsing a packet is way faster than any network traffic. In fact many resources will be in the first few packets aka the head (scripts, CSS). For CSS its even more condensed within one resource, you probably get a few URLs within a very short time.

> Furthermore, as I previously linked to in the very first reply to the thread, when we discussed MGET previously, I highlighted how the browser incrementally parses the document and sends GETs for resources as it discovers them.

Yes, you might want to wait n (3?) milliseconds before sending out additional requests and batch what you get within that timeframe. You don't really send out requests in realtime while parsing, do you? ;-)

> So, you handwaved over how the client would magically transform URL1 + URL2 + URL3 into magical example.com/resource/set. Assuming that's possible, how do you do the reverse transformation, when a HTTP/2=>HTTP/1.X gateway needs to translate HTTP/2 MGET requests for this /resource/set into the individual GETs for the original URLs.

I can't follow you here. A BATCH of 5 GETs would exactly be the same like 5 individual GETs w/ less HTTP overhead and better compression. Its trivial to convert this in both directions.

> And even if this is possible, how reasonable is it to pay this roundtrip on receiving the 405? We've fought really hard to eliminate roundtrips.

Maybe I'm missing something, but I thought the goal is to reduce 10...N requests to 1 in the best case. That 10 requests are 11 in the legacy case seems to be fine to me, plus a browser could remember on which sites it has seen a 405 and avoid the hit in the future.

hh

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 18 Feb 2013 13:23:37 +1100

Helge et al,

If you'd like to pursue this, I'd suggest that the folks who are interested get together and make a more complete proposal, e.g., as an Internet-Draft. Developing an approach like this on-list isn't going to use your time or others' well.

Thanks,

--
Mark Nottingham   http://www.mnot.net/

---

From: Helge Hess <[email-masked]>
Date: Sun, 17 Feb 2013 18:26:34 -0800

> If you'd like to pursue this, I'd suggest that the folks who are interested get together and make a more complete proposal, e.g., as an Internet-Draft. Developing an approach like this on-list isn'…

Fair enough ;-)

hh

---

From: James M Snell <[email-masked]>
Date: Sun, 17 Feb 2013 18:32:31 -0800

If they are in individual img, script, stylesheet elements, you would do a
single mget with each distinct URL listed. I presume that a new element
representing a single resource set would be needed for the other case.

The approach for this would be to list the grouped resources as early in
the page as possible. It is definitely not without it's problems.

>>
>> An old server would return a 405 when the BATCH comes in, then the
client needs to switch to performing the operations individually.

Fwiw that's not what I had in mind at all.

If the mget specifies separate URLs, the transforming proxy simply splits
those out into distinct get requests. Otherwise it translates it onto a
single get and hopes for the best. The server will either do the right
thing or return a 404.

Mget would not necessarily be backwards compatible and a client ought to
only use it if the origin server supports it.

- James

>
> So, you handwaved over how the client would magically transform URL1 +
URL2 + URL3 into magical example.com/resource/set. Assuming that's
possible, how do you do the reverse transformation, when a HTTP/2=>HTTP/1.X
gateway needs to translate HTTP/2 MGET requests for this /resource/set into
the individual GETs for the original URLs. And even if this is possible,
how reasonable is it to pay this roundtrip on receiving the 405? We've
fought really hard to eliminate roundtrips.

---

From: James M Snell <[email-masked]>
Date: Sun, 17 Feb 2013 18:33:44 -0800

I'll write up an initial draft this week.
On Feb 17, 2013 6:26 PM, "Helge Hess" <[email-masked]> wrote:

> > If you'd like to pursue this, I'd suggest that the folks who are
> interested get together and make a more complete proposal, e.g., as an
> Internet-Draft. Developing an approach like this on-list isn't going to use
> your time or others' well.
>
> Fair enough ;-)
>
> hh
>
>

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Sun, 17 Feb 2013 21:41:49 -0500

> ? e.g. <img>, <script>, CSS links etc.

Yes,

> needs to switch to performing the operations individually.

An old server would never understand the HTTP/2 stream format required to
deliver the multi-responses.

So the only case where HTTP/1.1 would be an issue would be in a HTTP/2 to
HTTP/1 gateway and there the gateway would have to either refuse to do an
MGET or break it down into separate requests.

---

From: William Chan (陈智昌) <[email-masked]>
Date: Sun, 17 Feb 2013 19:00:20 -0800

Sorry Mark, one last email to highlight where I believe the confusion has
come from. I look forward to James' I-D.

On Sun, Feb 17, 2013 at 6:10 PM, Helge Hess <[email-masked]>wrote:

> Well, I personally don't care too much about HTML here but about services,
> but I do think you can use the facility for this too. The browser would
> need to do some clever batching and latency management, but thats not
> really related to HTTP but HTML and an issue in any protocol.
> Fixing HTML would be a different thing, but sure, you could introduce
> resource-set tags which would directly map to batched requests.
>
> Presumably you receive your HTML in a streamed fashion as packets arrive,
> presumably parsing a packet is way faster than any network traffic. In fact
> many resources will be in the first few packets aka the head (scripts,
> CSS). For CSS its even more condensed within one resource, you probably get
> a few URLs within a very short time.
>
> discovers them.
> Yes, you might want to wait n (3?) milliseconds before sending out
> additional requests and batch what you get within that timeframe. You don't
> really send out requests in realtime while parsing, do you? ;-)
>

If you had read the previous email thread I linked to at the very
beginning, you would realize that contrary to Willy's expectation, I
demonstrated that we do indeed send out requests ASAP (putting aside some
very low-latency batching). We disable Nagle in order to prevent kernel
level delays in this manner, since we do indeed want to get requests out
ASAP.

> the individual GETs for the original URLs.
>
> I can't follow you here. A BATCH of 5 GETs would exactly be the same like
> 5 individual GETs w/ less HTTP overhead and better compression. Its trivial
> to convert this in both directions.
>

I believe the confusion arises because you seem to think I was confused
about MGET coalescing X GETs. No, I totally understand how that would work.
 I have no questions there, and I already linked to the previous discussion
which you appear not to have read despite my repeated references. If you
look to where I replied directly to James's proposal, I expressed confusion
about these paragraphs:
"""
That is, assume that a server defines a set of N resources and assigns
that set a singular url that represents the entire collection. When I
do..

"""

I do not know what this magic /resource/set is, and I do not see how to
divide it back up into individual GETs. Are your comments really in
relation to James' proposal as stated? If so, your explanation of how we
combine <img>,<script>, etc into "/resource/set" is incomprehensible and I
look forward to seeing a Internet-Draft.

I see that James has hinted that indeed we would need new markup to
indicate a "resource set".

>
> > And even if this is possible, how reasonable is it to pay this roundtrip
> on receiving the 405? We've fought really hard to eliminate roundtrips.
>
> Maybe I'm missing something, but I thought the goal is to reduce 10...N
> requests to 1 in the best case. That 10 requests are 11 in the legacy case
> seems to be fine to me, plus a browser could remember on which sites it has
> seen a 405 and avoid the hit in the future.
>

The goal is to, amongst others, "Substantially and measurably improve
end-user perceived latency in most cases, over HTTP/1.1 using TCP." (as
taken from the httpbis charter for HTTP/2.0 work). Coming up with a more
concise request representation (by compressing or otherwise sharing
redundant information across requests) is merely a means to achieve
improved end-user perceived latency. If you introduce roundtrips, then we
have optimized for a means at the expense of the ultimate goal.

Also, your comment on simply remembering origin servers only works if there
is no interference from intermediaries. Since many implementors here are
exploring HTTP/2.0 over port 80 in the clear, it seems unreasonable early
on to rely on intermediaries all understanding HTTP/2.0, at least early on.

---

From: Willy Tarreau <[email-masked]>
Date: Mon, 18 Feb 2013 08:06:35 +0100

Hi William,

> level delays in this manner, since we do indeed want to get requests out

As you told me it was not possible to guess URIs the way HTML is parsed
nowadays, I realized that for the protocol to succeed, we must make it
easy to implement on all sides (browsers, intermediaries, servers). We
must not make the protocol optimal for situations that do not exist or
which forces any party to do complex or undesired things (such as waiting
a few ms). Anyway, nobody must ever sleep for some time, everything must
always be event-driven because there is no way to recover for lost time.

Having thought about this for a while, I think we must keep in mind that
the enemy is RTT and we want to avoid stacking them. Thus I think that
a reasonable solution would be to set a limit on the number of concurrent
connections or streams to a given server, and to batch requests only when
there are too many unreplied ones.

This means that when DNS+RTT is shorter than the HTML parsing time, requests
leave one at a time. When DNS+RTT is higher than the parsing time, requests
leave in batches.

Also, the behaviour you explained to me means that requests should probably
be merged much like a pipeline method and less like a pack of requests. By
this, I mean that we should keep the opportunity to add requests very late
at low cost (eg: append something to the packet). For this reason, I don't
think the MGET method would be the most suited for the task because as you
said, you don't know that in advance. And if we're going to always prepare
MGET requests, then they will be a de-facto replacement for GET even for
single objects, which means we failed somewhere. One solution to this could
be to have structured requests ordered approximately like this :

     1) length from 1 to 6
     2) Host
     3) METH
     4) URI for req 1
     5) Header fields
     6) Data
     7) *(length + URI)
     8) END

That way requests may be aggregated until the send() is performed. And we
may even go further with better frame encoding if we allow a request to
reuse headers from the previous one, because then even if you did the send()
segments may still be merged in kernel buffers. This is what pipelining does
in 1.1 (except that pipelining offers no provisions for reusing elements from
the previous request).

And after all, that's also how objects are linked in the HTML document :
you discover requests in the middle of the stream until you reach the </HTML>
tag. Above it would be the same for the server : it would process all requests
with the same header fields until it sees the END tag.

For intermediaries and servers, this would be almost stateless. I'm saying
"almost" because reusing data means a state is needed, but this state lasts
only till the end of the requests batch, so in practice it's just like
tomorrow where we have to keep some request information while processing it
(at least for logging).

Regards,
Willy

---

From: Patrick McManus <[email-masked]>
Date: Mon, 18 Feb 2013 09:18:05 -0500

in the spirit of producing an ID that answers the big questions - I'm
a bit confused at what problem] a mget proposal is trying to solve
beyond what we've got in front of us now.. is it improved compression?

We already have (a few) schemes that do quite well on that count so it
isn't an existential problem that needs solving at any cost - and this
has a high cost associated with it. Frankly, its not clear to me that
the compression it gives would even be competitive with the delta
schemes - I'd like to see its proponents prove that out.

But beyond that it has costs derived from
* added latency to determine resource sets. That's pretty much a
non-starter for me because it conflicts with my core goals for the new
protocol
* reduced flexibility.. right now each resource has its own set of
headers which contain a lot of redundancy. The delta schemes preserve
that property while exploiting the redundancy. The mget scheme
requires all the resources in a set have the same headers, but in
truth small variations exist and are quite useful. Accept headers vary
by media type, Cookies vary, etc.. I'd hope an ID would look at page
loads that use those patterns.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Mon, 18 Feb 2013 16:21:22 -0500

On Mon, Feb 18, 2013 at 9:18 AM, Patrick McManus <[email-masked]>wrote:

> schemes - I'd like to see its proponents prove that out
>

I do not understand your argument here. Performance of MGET should be
pretty much identical to delta encoding because it is essentially the same
thing but without the complexity.

If MGET will do 80% of the requirements compression is meant to address
then my vote would be to nuke compression.

MGET is useful regardless of whether or not compression is implemented
while header compression is only an optimization.

Further, the problem with compression is that it is the type of topic that
people can bikeshed endlessly. There will never be a one true header
compression scheme. If the number of schemes allowed is more than zero then
it will inevitably end up as much more than one.

To clarify my original proposal, there are two forms of multiple get
possible

Type1: GET List< ContentID>

The first type of multiple get is simply a list of Content Identifiers
which at minimum consists of an ID but could consist of an ID plus a cache
tag if we are in a situation where this would be useful.

Type2 GET ContentID

The second type of multiple get is a single request that maps onto a set of
resources rather than just one resource. For example, accessing all the
messages in a user's email mailbox.

While this is one type of multiple get, it is not the type of multiple get
that I was considering in the context of rendering compression moot. It is
even more efficient on the request but does not allow a client side proxy
to assist with the response and does not support the case where the client
has some but not all of the resources indicated.

That might not matter of course. Client proxies were useful in 1993 when
the whole of CERN was hanging off a T1 and 99% of content was static.

I agree that the second case raises some interesting design issues. But I
would suggest that those are necessary design considerations that are
intrinsic to multiplexing/multi-streaming that have to be considered in any
case if that feature is to be supported.

---

From: Brian Pane <[email-masked]>
Date: Mon, 18 Feb 2013 17:19:53 -0800

> On Mon, Feb 18, 2013 at 9:18 AM, Patrick McManus <[email-masked]>
[...]
> I do not understand your argument here. Performance of MGET should be pretty
> much identical to delta encoding because it is essentially the same thing
> but without the complexity.

In some ways, MGET actually seems more complex.  Here's my rationale:

To avoid head-of-line blocking, a server or intermediary will need to
be able to return the requested resources out of order.  Consider the
simple case where the client MGETs five small, static resources.  The
request is processed by a CDN node, 10 msec away from the client, that
has all but the first of those resources in memory.  That cache has to
fetch the first resource from an origin server 100 msec away.  If MGET
uses in-order responses, the mean response time will be 3x worse with
MGET than it would have been with 5 separate GETs in parallel.

It's straightforward to do out-of-order responses if MGET works as
"MSYN_STREAM" - treat the MGET as shorthand for creating the right
number of streams and applying the request headers into each of those
streams.  But that only solves the header compression problem for
requests, not for responses.  If you want to reduce response header
transfer size too, you end up having to implement both MGET and header
compression.  And MGET would create some new error modes, especially
in combination with the recently proposed control frame continuation
mechanism.

Brian

---

From: Mark Baker <[email-masked]>
Date: Mon, 18 Feb 2013 22:53:05 -0500

On Sat, Feb 16, 2013 at 1:01 PM, Phillip Hallam-Baker <[email-masked]> wrote:
> that if the protocol is followed correctly forces a round trip delay on each
> content request.

It doesn't force it; you can pipeline the requests.

Mark.

---

From: Nico Williams <[email-masked]>
Date: Mon, 18 Feb 2013 22:24:14 -0600

> It doesn't force it; you can pipeline the requests.

In the context of web pages, the first GET is for the HTML of the
page, and only after the browser begins (or even completes) parsing of
that do the subsequent GETs get issued.

That's a round trip just to get started.

Why couldn't a browser do a GET of the page and also say "also send me
all the stylesheets, scripts, and images you have that this resources
refers to"??  Seems utterly reasonable to me.

All we need is a way to classify which related resources to fetch
immediately without further requests and which not to.  For web pages
the classification is probably something like: stylesheets, scripts,
images, audio, and audio/video -- add resource size thresholds too, or
perhaps a way of saying "send me any of these types of resources that
you think I'll need".  We may need a way to express profiles instead
(text-based browser, sight-impaired user, hearing-impaired user,
scripts disabled, ...).  These MGET profiles might be very
HTML-specific, but we might be able to generalize them enough too.

Nico
--

---

From: Nico Williams <[email-masked]>
Date: Mon, 18 Feb 2013 22:26:06 -0600

On Sat, Feb 16, 2013 at 12:01 PM, Phillip Hallam-Baker <[email-masked]> wrote:
> content request. Which of course leads to various browsers pushing the

The problem here is that the user-agent needs to get the top-level
resource first, then it will know the names of the other resources.
We can probably do better.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 19 Feb 2013 15:37:42 +1100

> We can probably do better.

Nico,

If I understand you, you're talking about making some really fundamental changes to the Web Architecture, which is squarely out of the WG's charter.

I don't mind discussing ideas and understanding how we got here, so long as they don't distract from our work. I get the feeling that this is starting to happen. 

Again, if you have a proposal, please write it up in detail and make it to the WG; endlessly discussing the minutia of a half-formed idea is not a productive use of anyone's time.

Thanks,

---

From: Nico Williams <[email-masked]>
Date: Mon, 18 Feb 2013 22:44:28 -0600

> If I understand you, you're talking about making some really fundamental changes to the Web Architecture, which is squarely out of the WG's charter.

Is Phillip's proposal also out of charter?

> Again, if you have a proposal, please write it up in detail and make it to the WG; endlessly discussing the minutia of a half-formed idea is not a productive use of anyone's time.

I... posted twice on this, within minutes.

Nico
--

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 19 Feb 2013 16:07:57 +1100

> Is Phillip's proposal also out of charter?

If it requires significant modification of HTTP APIs to function, or changes to HTML, yes. 

> I... posted twice on this, within minutes.

Yes, but it's not about a simple post count. It's about continuing a thread where I'd already suggested making a more well-formed proposal before continuing discussion.

To be clear - we have a number of outstanding action items, and a considerable amount of work to do to get to a first implementation draft. So, I get very concerned when I see the list being distracted by things that aren't on the plan for that first implementation draft, and may never be in-scope for us.

So, from time to time I'll remind people to keep the discussion focused. This is not a slight against you - if I have a problem with your participation, I'll contact you privately first.

Thanks,

---

From: Nico Williams <[email-masked]>
Date: Tue, 19 Feb 2013 00:59:20 -0600

> If it requires significant modification of HTTP APIs to function, or changes to HTML, yes.

I think neither.  Think of a header by which I ask for a tarball (or
whatever suitable archive format) with the resource and related
sub-resources?  How would such an extension require changes to HTTP
APIs or HTML?  I think it's neither.  It would require changes to the
server, but if the server doesn't grok this header no problem ensues:
the client can tell what happened because of the content type of the
result (assuming success).  To be fair the obvious way to code this on
the client would be either as a wrapper to HTTP APIs (I assume you
meant APIs to HTTP, as opposed to APIs accessed via HTTP) or as
extensions to HTTP APIs.

(And if you really meant HTTP APIs as in APIs accessed via HTTP then I
think those also would require no changes.)

> Yes, but it's not about a simple post count. It's about continuing a thread where I'd already suggested making a more well-formed proposal before continuing discussion.

Ah, my sin then was not reading the whole thread first.  Sorry.

Nico
--

---

From: Nicolas Mailhot <[email-masked]>
Date: Wed, 27 Feb 2013 10:16:55 +0000 (UTC)

James M Snell <jasnell@...> writes:

> Fair enough 

There is a similar need for mput/mpost, the fact current web apps require 
a separate user sequence for each file a user wants to publish/attach to a 
message is one of the few remaining use-cases where they suck compared to 
local apps.

You can work around it by asking users to sip their set of files, but that 
does not make you popular.

Regards,

-- 
Nicolas Mailhot

---

From: Julian Reschke <[email-masked]>
Date: Wed, 27 Feb 2013 11:33:50 +0100

> message is one of the few remaining use-cases where they suck compared to

But that's UI (HTML/JS), not protocol, right? Also, that's solvable; I 
happen to be in a project where our web app supports bulk upload of 
files by drag & drop to the browser window...

 > ...

Best regards, Julian

---

From: Nicolas Mailhot <[email-masked]>
Date: Wed, 27 Feb 2013 15:40:55 +0000 (UTC)

> files by drag & drop to the browser window...

That's just another workaround, where you paper over the missing feature with
gobs of site-specific javascript and by pretending the average user
drag-and-drops files in apps. I've seen the same demoware years ago it does not
work out in real life.

The average user does not drag and drop he uses the file selector, that maps to
standard html forms, that maps to what the protocol knows to do (one element at
a time). In browsers the file selector is restricted to single file selection to
respect what non-js-extended http/html ecosystem allows.

And, lastly, most web app developpers will not bother with loads of feel-good js
workarounds since what the users want is their default file selector, not some
kind of js emulation.

-- 
Nicolas Mailhot

---

From: James M Snell <[email-masked]>
Date: Wed, 27 Feb 2013 07:46:47 -0800

All,  my apologies, I was not able to get to this before the current
publishing cutoff for Orlando. I have a draft written up and will publish
after Orlando.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 27 Feb 2013 17:20:29 +0100

> workarounds since what the users want is their default file selector, not some

I still don't understand what the constraints of the browsers have to do 
with the protocol. Browsers use multipart uploads, and those allow 
multiple files already. What am I missing?

Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Wed, 27 Feb 2013 17:28:10 +0100

> multiple files already. What am I missing?

Also, see 
<http://www.w3.org/TR/html-markup/input.file.html#input.file.attrs.multiple>. 
No JS needed at all.

Best regards, Julian

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Wed, 27 Feb 2013 17:45:11 +0100

> <http://www.w3.org/TR/html-markup/input.file.html#input.file.attrs.multiple>.

Ah, nice, someone finally noticed this part was broken. Pity it's not
deployed yet.
Though the html5 element would still map naturally to an http/2 mput :)

Thanks for the pointer!

-- 
Nicolas Mailhot