Subject: Re: [Wpack] Factoring out some parts of the Web Bundle format

From: Daniel Ehrenberg <[email-masked]>
Date: Fri, 19 Feb 2021 19:54:15 +0000

On 2020-12-01 15:47, Daniel Ehrenberg wrote:
> Hi WPACK WG,
> 
> As I mentioned in the message "Broadening the WPACK WG charter", I'm
> very interested in Web Bundles for subresource loading, e.g.,
> https://github.com/WICG/webpackage/blob/master/explainers/subresource-loading.md
> . The Web Bundle format has a great architecture, in terms of having an
> extensible set of sections, to permit various different usage profiles.
> This WG could use take advantage of this architecture in order to define
> a cleanly layered format. In particular, a few features are not used for
> subresource loading:
> - primary URL: not meaningful, as the bundle is not being navigated to
> - manifest section: not used, similarly--the manifest should be linked
> from the document which includes the subresources
> - variants: not recommended, as content negotiation should be done when
> deciding which bundle to include, to avoid wasteful downloads
> 
> I think we can factor each of these features out from the core Web
> Bundles Internet-Draft and put them into a separate Internet-Draft which
> indicates how to use bundles to represent whole pages, much like the
> "signatures" section was recently removed from the Web Bundles draft.
> For this, I would suggest:
> - The primary URL should be located in a separate, optional section,
> like the manifest section.
> - The primary URL and manifest sections should be factored out into this
> other draft
> - For variants, the story is a bit trickier, since it is integrated into
> the responses section. Honestly, I'm having trouble understanding the
> motivation for Web Bundle's variants support, e.g., what use cases it is
> meant to satisfy. Some kinds of negotiation (e.g., with Client Hints or
> UA) extends outside of the variants model, if I understand correctly. Is
> there something else we can do to future-proof the binary format for
> this support, rather than including this particular format in the core
> RFC?
> 
> There are other additional sections which may be useful for some
> applications. For example, within build tooling, a section could be
> added to represent the dependency graph among subresources. So, Web
> Bundles with this modified factoring would represent an excellent basis
> for a variety of uses. 
> 
> What do you think?
> 
> Thanks,
> Dan

I've filed a few PRs against Jeffrey's draft proposal to make these
changes [1][2][3]. What do you think? Should we follow up on this topic
in the upcoming IETF 110 WPACK WG meeting?

Dan

[1] https://github.com/WICG/webpackage/pull/617
[2] https://github.com/WICG/webpackage/pull/618
[3] https://github.com/WICG/webpackage/pull/619

---

From: Larry Masinter <[email-masked]>
Date: Fri, 19 Feb 2021 17:27:23 -0800

Why don't you use HTTP range retrieval to access parts of a  bundle? 
It's OK not to have URLs for the parts without an explicit reification.

It seemed to work for PDF.

--
https://LarryMasinter.net https://interlisp.org

> -----Original Message-----
> From: Wpack <[email-masked]> On Behalf Of Daniel Ehrenberg
> Sent: Friday, February 19, 2021 11:54 AM
> To: [email-masked]
> Subject: Re: [Wpack] Factoring out some parts of the Web Bundle format
> 
> https://github.com/WICG/webpackage/blob/master/explainers/subresourc
> e-
> > loading.md . The Web Bundle format has a great architecture, in terms
> > of having an extensible set of sections, to permit various different
> > usage profiles.
> > This WG could use take advantage of this architecture in order to
> > define a cleanly layered format. In particular, a few features are not
> > used for subresource loading:
> > - variants: not recommended, as content negotiation should be done
> > when deciding which bundle to include, to avoid wasteful downloads
> >
> > Bundles Internet-Draft and put them into a separate Internet-Draft
> > which indicates how to use bundles to represent whole pages, much like
> > the "signatures" section was recently removed from the Web Bundles
> draft.
> > - The primary URL and manifest sections should be factored out into
> > this other draft
> > - For variants, the story is a bit trickier, since it is integrated
> > into the responses section. Honestly, I'm having trouble understanding
> > the motivation for Web Bundle's variants support, e.g., what use cases
> > it is meant to satisfy. Some kinds of negotiation (e.g., with Client
> > Hints or
> > UA) extends outside of the variants model, if I understand correctly.
> > Is there something else we can do to future-proof the binary format
> > for this support, rather than including this particular format in the
> > core RFC?
> >
> > Bundles with this modified factoring would represent an excellent
> > basis for a variety of uses.
> >
> > What do you think?
> >
> > Thanks,
> > Dan
> 
changes
> [1][2][3]. What do you think? Should we follow up on this topic in the
> upcoming IETF 110 WPACK WG meeting?
> 
> Dan
> 
> _______________________________________________
> Wpack mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/wpack

---

From: Jeffrey Yasskin <[email-masked]>
Date: Tue, 23 Feb 2021 15:00:53 -0800

I'm not sure which part of Daniel's emails you're replying to, but I know
of 2 reasons not to use HTTP range requests for bundles in general.

1) Ranges interact badly with content-encoded resources. That is, the range
refers to the post-compression byte ranges, which aren't what's available
in the index. We could avoid content-encoding the bundles, and instead
content-encode individual subresources, but that loses cross-resource
compression, which is a big reason to bundle things in the first place.
Transfer-encoding would seem to be a solution, but nowadays that's only
defined for HTTP/1.1:
https://httpwg.org/http-core/draft-ietf-httpbis-messaging-latest.html#field.transfer-encoding
.

2) If the client receives the index of a bundle, compares it to the cache,
and then issues range requests for the pieces it doesn't have, we have to
also figure out what the server is doing while it's waiting for those range
requests to come in. I think it probably should keep sending the bundle
linearly in order to avoid wasting time and in case the client isn't
planning to send ranges, but then it will probably have sent parts of the
bundle that it now also has range requests for. Should it send those as if
the range request wasn't connected to a full-bundle request? Reorder those
parts to the end of the multipart/byteranges response? Something else?
There are probably answers here, but getting it right seems complicated.

Jeffrey