Subject: FYI: Mozilla's Resource Packages

From: Anthony Bryan <[email-masked]>
Date: Tue, 17 Nov 2009 09:49:56 -0500

http://limi.net/articles/resource-packages/

Making browsers faster: Resource Packages

A proposal to make downloading web page resources faster in all browsers.

Introduction & Rationale

What if there was a backwards compatible way to transfer all of the
resources that are used on every single page in your site — CSS, JS,
images, anything else — in a single HTTP request at the start of the
first visit to the page? This is what Resource Package support in
browsers will let you do.

Implementation

While Zip files do not have not the most elegant or efficient packing
format out there, they have the following very desirable traits:

    * Easily available reference implementations.
    * Can be unpacked even in partial state — which means that we can
stream the file, and put CSS and JavaScript first in the archive, and
they will unpacked and made available before the entire file has been
downloaded.
    * Excellent toolchain support, zip/unzip is available on all major
platforms, so it’s easy for web developers to use.

We propose this markup to signal a zipped resource package:

<link rel="resource-package"
      type="application/zip"
      href="site-resources.zip" />

-- 
(( Anthony Bryan ... Metalink [ http://www.metalinker.org ]
  )) Easier, More Reliable, Self Healing Downloads

---

From: Julian Reschke <[email-masked]>
Date: Tue, 17 Nov 2009 16:20:11 +0100

Anthony Bryan wrote:
> ...

Thanks for the pointer. Note that the actual proposal at 
<http://limi.net/articles/resource-packages/> has more details.

Questions that come to mind:

(1) Is this specific to HTML? If yes, shouldn't it be proposed to the 
HTML WG?

(2) The link relation itself should be format-agnostic. It's ok to limit 
deployment and implementation to application/zip for now, though.

(3) Related to that, the "type" parameter on the HTML link element is 
optional in HTML4 as well.

(4) I have trouble understanding...:

"You can specify a charset in the resource package definition. If 
unspecified, it is assumed that any non-binary files inside are UTF-8."

Is this about the manifest? This seems to be problematic, as charset 
handling would be different from local file resources (I do agree that 
encouraging UTF-8 is good, though)

(5) How do non-URL characters in filenames in the ZIP map to URLs in 
content? It appears that a default encoding needs to be defined (such as 
->UTF-8->percent-escaped).

BR, Julian

---

From: Nicolas Alvarez <[email-masked]>
Date: Tue, 17 Nov 2009 13:43:33 -0300

> deployment and implementation to application/zip for now, though.

And it should be either registered or a URL...

-- 
Nicolas

I read mailing lists through Gmane. Please don't Cc me on replies; it makes 
me get one message on my newsreader and another on email.

---

From: Julian Reschke <[email-masked]>
Date: Wed, 18 Nov 2009 05:37:30 +0100

Alexander Limi wrote:
> ...
>     (3) Related to that, the "type" parameter on the HTML link element
>     is optional in HTML4 as well.
> 
> 
> Good to know, thanks. I'll make it optional and let the implementation 
> assume application/zip unless otherwise specified, then.
> ...

The application should use whatever the Content-Type response header 
specifies. No reason to assume anything, unless that information is missing.

BR, Julian

---

From: Srirama ChandraSekhar <[email-masked]>
Date: Wed, 18 Nov 2009 10:43:07 +0530

Hi,
The idea looks very good. But I want to point out one scenario which should
be considered before going forward with this approach. Consider a text only
browser or a browser which doesn't want to display images or doesn't want to
execute javascript or doesn't support java script (may be through settings
change by the user), in this case the browser will download the resource
package file only to find that it has unnecessarily downloaded many images
and javascript file which are not going to be used. To address this may be
we can add some header through which browsers should be able to tell the
servers what resources should the resource package file contain.

Regards,
Sriram

---

From: Robert Siemer <[email-masked]>
Date: Wed, 18 Nov 2009 14:02:12 +0800

All this resource package thing looks like a general preloading
mechanism...

If we use standard URLs, we can catch things like:
-compression
-content-type
-caching questions
because it is already specified and out there.

Using javascript (and/or other tricks), we can even preload right now.
-- So, what is missing?

>From the proposal (http://limi.net/articles/resource-packages/ ) I see
only one goal left over:
-serve everything in a single HTTP request, freeing up the other
parallel requests to fetch resources that are page-specific

Why not use pipelining?? So, my alternatives are:

-use more parallel connections to the same server
-improve pipelining by specifying what to pipeline and in which order
-foster HTTP over SCTP

I definitively do not see the Resource Package approach to be the answer
to the problem.

Robert

On Wed, 2009-11-18 at 10:43 +0530, Srirama ChandraSekhar wrote:
> Hi,
> The idea looks very good. But I want to point out one scenario which
> should be considered before going forward with this approach. Consider
> a text only browser or a browser which doesn't want to display images
> or doesn't want to execute javascript or doesn't support java script
> (may be through settings change by the user), in this case the browser
> will download the resource package file only to find that it has
> unnecessarily downloaded many images and javascript file which are not
> going to be used. To address this may be we can add some header
> through which browsers should be able to tell the servers what
> resources should the resource package file contain.
> 
> Regards,
> Sriram
> 
> On Tue, Nov 17, 2009 at 8:19 PM, Anthony Bryan
> <[email-masked]> wrote:

---

From: Alexander Limi <[email-masked]>
Date: Tue, 17 Nov 2009 22:20:25 +0000

2009/11/17 Julian Reschke <[email-masked]>

> (1) Is this specific to HTML? If yes, shouldn't it be proposed to the HTML
> WG?
>

Possibly? I don't know much about how these things happen, standards-wise.

> deployment and implementation to application/zip for now, though.

Right. If there's ever a newer, widely supported compression format that did
the same thing, that would work too.

> encouraging UTF-8 is good, though)

The manifest probably has to be ASCII (using quoted values like %20 for
spaces etc), sorry about not specifying that.

The UTF-8 default is for any other file in the zip, like JS or CSS, or even
HTML files, should that be useful.

> ->UTF-8->percent-escaped).

Percent-escaping would be my initial suggestion, but I don't know enough
about any potential issues here if we choose to go that route. I agree that
it needs to be defined, though.

-- 
Alexander Limi · Firefox User Experience · http://limi.net

---

From: Julian Reschke <[email-masked]>
Date: Wed, 18 Nov 2009 09:48:44 +0100

Srirama ChandraSekhar wrote:
> should be considered before going forward with this approach. Consider a 
> text only browser or a browser which doesn't want to display images or 
> doesn't want to execute javascript or doesn't support java script (may 
> be through settings change by the user), in this case the browser will 
> download the resource package file only to find that it has 
> going to be used. To address this may be we can add some header through 
> which browsers should be able to tell the servers what resources should 
> the resource package file contain.
> ...

Which would make the package content-negotiated, which will make caching 
harder (in particular considering the many bugs related to "Vary").

BR, Julian

---

From: "Martin J. Dürst" <[email-masked]>
Date: Wed, 18 Nov 2009 18:35:58 +0900

On 2009/11/18 7:20, Alexander Limi wrote:
> 2009/11/17 Julian Reschke<[email-masked]>
>

>> handling would be different from local file resources

But different clients will have different ways to handle local file 
resources, so the server shouldn't try to second-guess and adapt to 
local conventions. Using something that works easily over the network, 
such as UTF-8, seems to be the best choice.

>> (I do agree that

I'm not clear why you would need ASCII. The manifest file is something 
new, and thus saying it's UTF-8 would not have any problems, or would 
it? In any case, you should look at how non-ASCII filenames inside a zip 
file can be handled. The W3C Web Applications WG and the W3C I18N Core 
WG had a look at this issue in the context of 
http://www.w3.org/TR/widgets/. You should be able to find that 
discussion if you search the mailing list archives.

> HTML files, should that be useful.

So this is a replacement for a charset=foo parameter on the media type?
How do the mime types of the various files get determined in the first 
place? It's easy for humans to guess media types in an obvious example 
such as
    javascript/jquery.js
    css/reset.css
    css/grid.css
    css/main.css
    images/save.png
    images/info.png
but we are speaking about computers, and on the average much less 
obvious directory/file names.

> it needs to be defined, though.

You should do whatever makes things work well with the IRI spec (RFC 
3987, now being updated as draft-duerst-iri-bis). This means that 
non-ASCII characters in Web addresses in content should be transcoded to 
UTF-8, and then either matched directly (if the manifest file is UTF-8), 
or via %-encoding. I think that's what Julian is proposing, I'm just 
trying to give a slightly longer explanation.

Regards,   Martin.

-- 
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp   mailto:[email-masked]

---

From: Mark Baker <[email-masked]>
Date: Wed, 18 Nov 2009 09:45:16 -0500

> I definitively do not see the Resource Package approach to be the answer

Agreed.  Are the authors familiar with RFC 2557?

Some of the discussion on hybi, for example client-prioritized
multiplexed streams, are far more appealing as a general solution IMO.

Mark.

---

From: "Sander Steffann" <[email-masked]>
Date: Wed, 18 Nov 2009 16:37:31 +0100 (CET)

> Agreed.  Are the authors familiar with RFC 2557?

I don't think that is usable for this case. It wouldn't be backwards
compatible. Also the server can't know if the client already has the
related resources or not, which could cause unnecessary traffic.

There are probably more efficient or more flexible ways to solve this
problem (using SCTP, other changes in the protocol etc), but I like the
proposal because it is backwards compatible and easy to use.

- Sander

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 18 Nov 2009 16:48:26 +0000

> browsers will let you do.

Why is this better than HTTP pipelining?

If the issue is HTTP pipelining is not widely used by clients, or not
aggressively enough, perhaps the solution is a something sent by the
server, perhaps in a HTML META element (same way as Resource
Packages), which says "it's ok, you can use pipelining aggressively on
*the following resources* <list of URIs>".

It seems to me that HTTP pipelining requires the browser to:

   1. Receive first part of initial page.
   2. Send data block which is a set of requests for the additional resources.
   3. Receive all the resources in order.

Whereas Resource Packages require the browser to:

   1. Receive first part of initial page.
   2. Send data block which is request for the resource package.
   3. Receive all the resources in order.

With the main performance differences being:

   a. The batch of requests for HTTP pipelining will use more bytes,
      but need not be a lot if the browser is sensible.  In modern
      networks, performance is determined by number of packets and timing,
      not just bytes, and multiple requests are batched into single
      packets if the timing is right.

   b. Use of Resource Packages implies unnecessary downloads by
      browsers which have disabled Javascript and/or images, or are
      text-only.  They can simply not download the Resource Package,
      of course, but that seems rather defeatist, because a main reason
      to disable images is to speed up downloads, not to be penalised
      for it.

   c. Use of Resource Packages is wasteful if some but not all of the
      resources are cached, either locally or in proxies.  This is
      most likely when some of the resources are changed.

And for the long term:

   d. Future optimisations to HTTP, such as HTTP over SCTP, Google's
      SPDY, Multi-server HTTP and others, may transfer multiple
      resources more efficiently, for example by prioritising the most
      important requests, and sending differences from previously
      cached values.

      Resource Packages may, to some extent, thwart some of these
      strategies.

      Then again, if Resource Packages become widely used, it's
      possible that protocol, server and proxy design will evolve to
      include generating them on demand and dynamically modifying them
      to optimise transfers.

      To allow this, IMHO it's important that Resource Packages are
      identified as such, so that proxies and caches will be able to
      safely inspect and modify them in future if the need arises.

-- Jamie

---

From: Julian Reschke <[email-masked]>
Date: Wed, 18 Nov 2009 18:13:51 +0100

> *the following resources* <list of URIs>".

That's an interesting suggestion; maybe a single opt-in would be 
sufficient (no URI list).

Do we have data on how frequently pipelining is broken by 
intermediaries, not origin servers?

BR, Julian

---

From: Jamie Lokier <[email-masked]>
Date: Wed, 18 Nov 2009 17:23:52 +0000

>       safely inspect and modify them in future if the need arises.

Unfortunately I see the proposal does not identify Resource Packages
to the transport: They are simply application/zip resources, so it
would be wrong for servers and (especially) proxies to automatically
inspect and modify those.  (Of course they will anyway, if there is a
compelling advantage).

The main benefit I see to using the existing application/zip MIME type
is that you can begin using Resource Packages without any cooperation
from your service provider - just put up a .zip file.

-- Jamie

---

From: Nicolas Alvarez <[email-masked]>
Date: Wed, 18 Nov 2009 14:40:54 -0300

> specifies. No reason to assume anything, unless that information is

And it should send an Accept header with the archive formats it supports for 
this feature (which would be different from what the browser usually sends; 
text/html is *not* an "accepted" format for a resource package!).

-- 
Nicolas