Subject: [Wpack] About content-based origins

From: "Martin Thomson" <[email-masked]>
Date: Tue, 24 Mar 2020 11:34:19 +1100

Ted's note prompted me to send a much-belated announcement (sorry folks, I forgot).

The draft is here:
https://tools.ietf.org/html/draft-thomson-wpack-content-origin-00

A nicer version here:
https://martinthomson.github.io/wpack-content/draft-thomson-wpack-content-origin.html

This approach could a dramatically different approach to addressing the use cases set out in our charter.

In short, this aims to address the core question of how offline content might *ultimately* be attributed to a web origin in a fundamentally different way.  There are two key concepts:

1. Content is given its own origin, using a new system for identification.

2. A target origin can "accept" content and state from one of these new origins.

There are a lot of details here (read the draft), but the major advantage I see is that you don't have to make an offline decision about authority, and that means you can be offline for much longer (lifting the 7 day limit).

What it does have in common with signed exchanges approach is the need for a bundling format, but in its current form it is less dependent on the details of the format.  That might allow that to be simpler, but I'm sure that the need to mint new identifier types will more than make up for any slack there.

The draft is quite rough.  I'm sure that it has the remnants of a few bad ideas still hanging around.  Ask questions if you think something is unclear.

---

From: Ted Hardie <[email-masked]>
Date: Tue, 24 Mar 2020 10:47:34 -0700

Hi Martin,

Thanks for your note.  A question in-line.

On Mon, Mar 23, 2020 at 5:34 PM Martin Thomson <[email-masked]> wrote:

> In short, this aims to address the core question of how offline content
> might *ultimately* be attributed to a web origin in a fundamentally
> different way.  There are two key concepts:
>
I note that you did not include "hashing is a key element of
identification" in the key concepts here, despite it being pretty prominent
in the draft.  Is it something you consider fundamental here, or is it an
approach to demonstrate a method of achieving the two desiderata above?

> There are a lot of details here (read the draft), but the major advantage
> I see is that you don't have to make an offline decision about authority,
> and that means you can be offline for much longer (lifting the 7 day limit).
>
> I would personally describe this as a trade-off, rather than an
advantage.  The inability to make an offline decision about authority has
consequences for how different resources available to the client relate to
each other (e.g some come from a bundle and some from a cache).  To take an
unfortunately pressing example, let's assume a local authority has provided
a set of resources to local residents.  A new item is created and
distributed.  For those getting it via direct connection to the local
authority's server, the new item can be related to the other items.  With a
signed exchange and a signature verification, those receiving it
peer-to-peer can as well.  In a content-origin approach, my understanding
is that the new item could be received and displayed, but the ability to
integrate it with the rest of set is not active until the client can go
online and confirm with the local authority's serve that it verifies that
"ni:hashmumble" is the target.   The problem here for an offline client is
pretty serious, as it cannot determine whether the new resource is from the
local health authority or not.  If it updates information like a set of
symptoms or treatments, the consequences are large.

Have I missed something here?  If I have not, then this approach makes a
trade-off that limits its utility in ways that make peer-to-peer usage of
the approach problematic for use cases I personally care about it.  Others,
of course, may make other assessments of that trade-off.

regards,

Ted

> What it does have in common with signed exchanges approach is the need for
> a bundling format, but in its current form it is less dependent on the
> details of the format.  That might allow that to be simpler, but I'm sure
> that the need to mint new identifier types will more than make up for any
> slack there.
>
> The draft is quite rough.  I'm sure that it has the remnants of a few bad
> ideas still hanging around.  Ask questions if you think something is
> unclear.
>
> _______________________________________________
> Wpack mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/wpack
>

---

From: Ben Schwartz <[email-masked]>
Date: Tue, 24 Mar 2020 16:03:41 -0400

For a lighter example: if I'm distributing updates to a single-player
videogame, does the client have to get online and ping the server before
the game can access my local saved state?

(I can think of ways to avoid this while preserving the origin separation,
by forking the local state and blocking network access, but it seems
complicated.)

On Tue, Mar 24, 2020 at 1:48 PM Ted Hardie <[email-masked]> wrote:

> local health authority or not..  If it updates information like a set of
>> What it does have in common with signed exchanges approach is the need
>> for a bundling format, but in its current form it is less dependent on the

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 25 Mar 2020 16:06:13 +1100

On Wed, Mar 25, 2020, at 04:47, Ted Hardie wrote:
> identification" in the key concepts here, despite it being pretty 
> prominent in the draft. Is it something you consider fundamental here, 
> or is it an approach to demonstrate a method of achieving the two 
> desiderata above?

Exemplar only.  I think that signatures offer some greater flexibility, but I wanted to show how this might work without them first.

> trade-off that limits its utility in ways that make peer-to-peer usage 
> of the approach problematic for use cases I personally care about it. 
> Others, of course, may make other assessments of that trade-off.

No, that's an accurate assessment.  The requirement to defer attribution of information to a particular entity is part of that trade-off.  In preparing slides, I think that the question of how to manage attribution is going to be an important one.

I think that Jeffrey is proposing that perhaps we can attribution of the sort that people care about, without also having attribution of the sort that the web security model cares about.  That might be a cool way to split this.  That is, you can tell people that this is example.com, but maintain segregation and give example.com the right of refusal for the associated content and state at the point you come back online.

---

From: Devin Mullins <[email-masked]>
Date: Tue, 24 Mar 2020 23:35:44 -0700

Hi Martin,

Thanks for the engineering work on this. For others, I've already reviewed
this, had some chats with others on the AMP team, and had a few
back-and-forths with Martin (he's been very gracious in engaging his time,
thank you). I'm trying to estimate the feasibility of this, both for the
AMP use-case, and for the non-AMP use-case [1], which we hope to see
flourish to the extent that publishers and users desire it, and only
impeded by implementation costs where necessary for the common good.

With the caveat that this is very early feedback, the high-level bit is
that, with some modifications, it seems mostly feasible for AMP. However,
there are two main downsides:
  1. worse UX on sites that run A/B experiments
  2. technical constraints make this harder for publishers to adopt on
non-AMP, as compared to SXG

A lot of details follow. I suggest to rename the subject if you reply to a
detail below, in order to reduce cross-talk and make browsing the archive
easier.

Details:

We would need the display URL to be the attested URL even before transfer.
I suspect a flash in the URL bar would be frustrating for both users and
publishers. Jeffrey had proposed using signatures for this. I'd suppose UAs
could choose to render this with a "may be stale" indicator.

In terms of publisher implementation feasibility, I think the mostly likely
implementation would be for a publisher to produce a Sec-Content-Origin
(Sec-CO) response for the current version of the resource iff it matches
the Sec-CO request. I suspect that keeping state of past versions of the
resource would be infeasible, especially as it couldn't be done purely at
the edge; it would require something like a region-wide cache. The value of
the above stateless implementation would be inversely proportional to how
often the resource changes (e.g. "transient content" [2] such as Related
Articles). I think this would be a workable constraint for publishers
wishing to publish such bundles, but felt it's worth noting, nonetheless.

Re: fallback behavior on failed state transfer, distributors would need a
way to monitor failure rates. This serves two purposes:
  1. Automatically detecting errors in the distribution pipeline. This is
the same purpose served by
https://wicg.github.io/webpackage/loading.html#signed-exchange-report.
  2. Verifying that, for some fraction of navigations, bundles are meeting
AMP's stated UX goals (by using the verified content in the bundle and not
potentially arbitrary content after the redirect). This is a dynamic
equivalent of what can be done mostly statically with SXG, because the
distributor can run the same algorithm as the UA, modulo client/server skew
in clocks and root stores. I think this is a trade-off AMP could make. The
alternative is that the fallback behavior keeps the content-based origin
(CBO).

There is a possible vector for user ID transfer from distributor to
publisher; filed as https://github.com/martinthomson/wpack-content/issues/1.

This limits the ability to run session-sticky experiments (in
https://amp.dev/documentation/components/amp-experiment/ and many non-AMP
frameworks). The publisher has the following options:
  1. Delay rendering content under experiment until after state transfer.
This hurts UX for the sake of experimentation, creating a trade-off that
otherwise doesn't exist on origin (modulo ease of implementation).
  2. Generate a client-side UUID before state transfer, and join it with
the pre-existing user ID after transfer. This means the session could not
include CBO pageviews anywhere except in the first pageview.
  3. Generate variants of the bundle for each experimental state. The
distributor chooses which bundle to send to the user (based on what? not
sure). For a page with M independent experiments each with N arms, that's
N^M variants. For large enough M and N, this is likely both infeasible for
publishers and martinthomson/wpack-content#1. (But I need some guidance
from experts on typical ranges for M and N.)

Perhaps this is a need the UA could address. A straw proposal -- a
selectExperimentArm() API that:
  1. only exposes as many bits as the UA deems OK, and
  2. exposes different bits to different attested origins, so they can't be
joined
On the one hand, perhaps this is too much scope for wpack. On the other, it
addresses a problem that is somewhat particular to CBOs.

This impedes content management (e.g. dialogs for GDPR and CCPA), compared
to SXG. The publisher has a few choices:
  1. Delay rendering the dialog under after transfer. I think this would
cause layout shift (https://web.dev/cls/).
  2. Render the dialog, and risk discovering that it's already been
acknowledged. At that point:
    a. Hide it, causing layout shift.
    b. Leave it there. This is way outside my domain expertise, but ISTR
there are some rules or at least best practices wrt when it's okay to
re-show an opt-out dialog.
One of these options may be an okay trade-off; I'm not sure.

Analytics providers (via
https://amp.dev/documentation/components/amp-analytics/ and many non-AMP
libraries) would have to do one of two things:
  1. Delay pingback until after transfer. This risks a lower fraction of
successful pingbacks, as e.g. the user might close the tab before transfer.
  2. Generate a client-side UUID, send pingback before transfer, and send a
follow-up after transfer to join with pre-existing user ID. This captures
those otherwise lost pingbacks pseudonymously (anonymously?), though it
requires server-side work on the pingback endpoint.

For the above client-side mitigations, two feature requests seem necessary:
  1. Store the transfer metadata (e.g. indexedDB renames by time) somewhere
more permanent. For various reasons the event handler may not fire or
finish, and it would be good to later scan for stragglers to merge or
expunge.
  2. DOM API to wait for transfer to complete.

Some subresources are ACLs by Origin header (e.g. paid fonts). Likely
mitigation is to defer loading until transfer, with a timeout whose
duration depends on the publisher's font-display preference. (Martin
suggests an optimization where the encrypted subresource is loaded early,
and only the decryption key after transfer.)

Last but not least, I am concerned that this is a bar that is
disproportionately difficult for non-AMP publishers to meet. The above
client-side mitigations are possible, but quite a bit of work.
  1. For frameworks like AMP that strongly encourage publishers to use a
rolling release, it is possible to upgrade many use-cases in the wild with
minimal publisher effort.
  2. For other frameworks, it may be possible to provide this support, but
the requirement to upgrade versions may limit applicability. Because
upgrades occur less often, they often require more work.
  3. For custom JS that manages state, authors will need to modify it to
move or copy state into indexedDB, and handle transfer events and merge
conflicts.

>From a "minimal publisher effort" perspective, I am especially interested
in making it possible to have as close to a turnkey solution as is
reasonable, for instance at the level of the CDN or CMS. If such a solution
exists, sites would likely contain a hybrid of all three cases above, and
thus want to opt into such support incrementally. How should a CDN
determine which pages to provide Sec-CO responses for?
  1. URL patterns are the simplest answer, but I suspect hard for
publishers to create and maintain with minimal false negatives and
positives.
  2. Would it be helpful to thread an "I support state transfer" annotation
for some portion of the build journey, from individual JS function all the
way to bundle? Would folks use this?

I don't have a good solution to the issue of non-AMP developer cost, and
thus fear this is a solution that would feature a disproportionate amount
of AMP pages compared to the background distribution of the web. But I'm
hopeful that other JS framework developers can chime in with respect to
feasibility of support for CBO and state transfer.

I'm leaving out some of Martin's replies to the above; no offense intended.
It took me long enough just to summarize the above, and I'd better send it
early enough so at least a few people have time to read it before the
meeting.

Thanks,
Devin

[1]
https://blog.amp.dev/2019/05/22/privacy-preserving-instant-loading-for-all-web-content/
[2]
http://www.seobythesea.com/2011/12/how-google-might-identify-transient-content-on-webpages/

---

From: Ted Hardie <[email-masked]>
Date: Wed, 25 Mar 2020 10:28:06 -0700

Hi Devin,

Thanks for the write-up.  A clarifying question, snipped from the middle of
your response:

On Tue, Mar 24, 2020 at 11:36 PM Devin Mullins <twifkak=
[email-masked]> wrote:

>
> In terms of publisher implementation feasibility, I think the mostly
> likely implementation would be for a publisher to produce a
> Sec-Content-Origin (Sec-CO) response for the current version of the
> resource iff it matches the Sec-CO request. I suspect that keeping state of
> past versions of the resource would be infeasible, especially as it
> couldn't be done purely at the edge; it would require something like a
> region-wide cache. The value of the above stateless implementation would be
> inversely proportional to how often the resource changes (e.g. "transient
> content" [2] such as Related Articles). I think this would be a workable
> constraint for publishers wishing to publish such bundles, but felt it's
> worth noting, nonetheless.
>

If the ecosystem supported something like signed bundles and unsigned
bundles which used content identifiers, would you see a site sending a
Sec-Content-Origin response resulting in the user agent essentially
upgrading the bundle's treatment to the same as a signed bundle's, or would
there a third state (unsigned bundled, verified NI, signed bundle)?

regards,

Ted

---

From: Devin Mullins <[email-masked]>
Date: Wed, 25 Mar 2020 11:25:28 -0700

> there a third state (unsigned bundled, verified NI, signed bundle)?

If you're asking about user-agent behavior, I'll defer to folks who work on
user-agents such as Martin and Jeffrey.

In terms of how the framework would behave, I'm not the expert, but from
the hip, I'd guess, if in all major user agents, signed bundles can access
storage on their attested origin, then we'd bundle the same JS in both
signed and unsigned cases. We'd have to move the state of any of our
render-critical components [1] to indexedDB for unsigned, and after that, I
don't see a reason to keep the non-indexedDB version around just for
signed; it'd be a maintenance burden. (But I'm betting there's nuances I
don't know about.) If transfer event handlers end up being a lot of code,
we could optimize them away in signed bundles to save JS binary size, but
I'm guessing they won't be a lot of code, therefore not worth the effort to
factor the code for that ability.

To make the above work, I'd think we'd need the "wait for transfer" DOM API
to resolve immediately for signed bundles and non-bundled pages.

Of course, with all things, this will take time. Since support for signed
exchanges exists today, it would be easiest to migrate support to signed
bundles initially, so I would expect a staggered rollout.

I hope I answered your question!

[1] or portions of the analytics/consent components, possibly

---

From: Jeffrey Yasskin <[email-masked]>
Date: Wed, 25 Mar 2020 12:18:41 -0700

On Wed, Mar 25, 2020 at 10:29 AM Ted Hardie <[email-masked]> wrote:

> Hi Devin,
>
> Thanks for the write-up.  A clarifying question, snipped from the middle
> of your response:
>

I don't think there should be three states.

I've been trying various models and sets of terminology for the possible
treatments of bundle content over the last couple months, and I'm not yet
sure what model makes most sense. This week, my favorite is to use
HTTP's notion of a response being "authoritative",
https://tools.ietf.org/html/draft-ietf-httpbis-semantics-07#section-11.1. A
response in a bundle can either be authoritative for their claimed URL or
not, and they're authoritative if any of several conditions holds:
1. The bundle itself was served by the same origin.
2. The bundle was signed by the origin.
3. A live request to the claimed origin adopts the content.

I think that doesn't quite match Martin's notion of aliasing origins, so I
still have some thinking to do.

Jeffrey

---

From: Devin Mullins <[email-masked]>
Date: Wed, 25 Mar 2020 16:03:42 -0700

In light of my comment about racing the Sec-CO request, a colleague pointed
out a potential hole in the double-hash scheme. There may be flaws in
below; I haven't thought about it too much. <trite tone=apologetic>It may
be worth more clearly defining the threat model.</trite>

The publisher could delay responding to the Sec-CO request. In the interim,
the page could send its own hash to //publisher.example/this-is-my-hash.
Then the publisher would respond with the preimage sent by the bundle.
This requires:

A way for the bundle to get its own hash. Ideas:

   1. document.location -- we could fix the ni scheme to be based on the
   double-hash
   2. a fetch of distributor.example/what-is-the-hash-of-referer
   3. compute client-side based on document.body, in which the publisher
   could also include an encoding of the headers
   4. somebody computes the fixmanifold of a hash function -- is that a
   thing? i can't math [1]

A way for the publisher to associate the this-is-my-hash request with the
Sec-CO request. Ideas:

   1. IP+timestamp
   2. a token as a query param to both requests

This depends on the publisher server to have an ability that most don't
have, but it wouldn't be hard to purpose-build a server that only handled
this-is-my-hash and Sec-CO requests. By segregating its workload from the
other content requests, it would be easier to scale -- its need for
session-stickiness or regional storage wouldn't affect latency of the
edge-serving of normal content.

Overall, this still requires more effort than responding `Sec-CO: yes`, but
possibly less than a distributor/publisher backchannel. Also, possibly
immune to IP privacy depending on above.

I hope first that my above napkin sketch is wrong, and secondly that the
fix for this isn't deferral of all requests until post-transfer. My guess
is that would harm performance of the instant navigation use case.

One possible mitigation is deferral of all on-origin requests. Then there
needs to be a backchannel between the this-is-my-hash origin and the state
transfer origin. This is maybe vaguely isomorphic to the
distributor/publisher backchannel? Not really, since subdomains. The
publisher could easily CNAME sec-co.publisher.example to
sec-co.distributor.example.

Actually, I wonder if CNAME is an issue regardless of everything above.
https://tools.ietf.org/html/draft-thomson-wpack-content-origin-00#section-5.1
states the UA must follow redirects. The publisher could easily encode a
redirect from publisher.example/foo to sec-co.publisher.example/foo, and
then CNAME to the distributor, and let the distributor respond with its
known preimage.

[1] "fixmanifold" being vaguely an extension of a fixpoint. A function f(x)
s.t. for all x, hash(concat(x, f(x)) == f(x). Then the bundle could encode
its own hash.

On Tue, Mar 24, 2020 at 11:35 PM Devin Mullins <[email-masked]> wrote:

> publisher; filed as
> https://github.com/martinthomson/wpack-content/issues/1.
>
>   2. exposes different bits to different attested origins, so they can't
> be joined
> On the one hand, perhaps this is too much scope for wpack. On the other,
> it addresses a problem that is somewhat particular to CBOs.
>
>   2. Generate a client-side UUID, send pingback before transfer, and send
> a follow-up after transfer to join with pre-existing user ID. This captures
>   1. Store the transfer metadata (e.g. indexedDB renames by time)
> somewhere more permanent. For various reasons the event handler may not
> fire or finish, and it would be good to later scan for stragglers to merge
> or expunge.
>   2. Would it be helpful to thread an "I support state transfer"
> annotation for some portion of the build journey, from individual JS
> function all the way to bundle? Would folks use this?
>
> I'm leaving out some of Martin's replies to the above; no offense
> intended. It took me long enough just to summarize the above, and I'd
> better send it early enough so at least a few people have time to read it
> before the meeting.
>
> Thanks,
> Devin
>
> [1]

---

From: Devin Mullins <[email-masked]>
Date: Wed, 25 Mar 2020 17:16:02 -0700

Augh, I forgot that 30x responses change the attested URL. Ignore my
comments about CNAME, I suppose.

On Wed, Mar 25, 2020 at 4:03 PM Devin Mullins <[email-masked]> wrote:

> In light of my comment about racing the Sec-CO request, a colleague
> pointed out a potential hole in the double-hash scheme. There may be flaws
> in below; I haven't thought about it too much. <trite tone=apologetic>It
> may be worth more clearly defining the threat model.</trite>
>
> The publisher could delay responding to the Sec-CO request. In the
> interim, the page could send its own hash to
> //publisher.example/this-is-my-hash. Then the publisher would respond with
> the preimage sent by the bundle. This requires:
>
> Overall, this still requires more effort than responding `Sec-CO: yes`,
> but possibly less than a distributor/publisher backchannel. Also, possibly
> [1] "fixmanifold" being vaguely an extension of a fixpoint. A function
> f(x) s.t. for all x, hash(concat(x, f(x)) == f(x). Then the bundle could
> encode its own hash.
>
>> Thanks for the engineering work on this. For others, I've already
>> reviewed this, had some chats with others on the AMP team, and had a few
>> A lot of details follow. I suggest to rename the subject if you reply to
>> a detail below, in order to reduce cross-talk and make browsing the archive
>> easier.
>>
>> Details:
>>
>> We would need the display URL to be the attested URL even before
>> transfer. I suspect a flash in the URL bar would be frustrating for both
>> users and publishers. Jeffrey had proposed using signatures for this. I'd
>> suppose UAs could choose to render this with a "may be stale" indicator.
>>
>>   2. Verifying that, for some fraction of navigations, bundles are
>> meeting AMP's stated UX goals (by using the verified content in the bundle
>> and not potentially arbitrary content after the redirect). This is a
>> dynamic equivalent of what can be done mostly statically with SXG, because
>> the distributor can run the same algorithm as the UA, modulo client/server
>> skew in clocks and root stores. I think this is a trade-off AMP could make.
>> The alternative is that the fallback behavior keeps the content-based
>> origin (CBO).
>>
>> https://amp.dev/documentation/components/amp-experiment/ and many
>> non-AMP frameworks). The publisher has the following options:
>> This impedes content management (e.g. dialogs for GDPR and CCPA),
>> compared to SXG. The publisher has a few choices:
>>   2. For other frameworks, it may be possible to provide this support,
>> but the requirement to upgrade versions may limit applicability. Because
>>> There are a lot of details here (read the draft), but the major
>>> advantage I see is that you don't have to make an offline decision about
>>> authority, and that means you can be offline for much longer (lifting the 7
>>> day limit).
>>>
>>> The draft is quite rough.  I'm sure that it has the remnants of a few
>>> bad ideas still hanging around.  Ask questions if you think something is
>>> unclear.
>>>

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 27 Mar 2020 15:32:19 +1100

Slowly crawling out from a giant pile of email...

I will try to get to your longer email later.

On Thu, Mar 26, 2020, at 10:03, Devin Mullins wrote:
> pointed out a potential hole in the double-hash scheme. There may be 
> flaws in below; I haven't thought about it too much. <trite 
> tone=apologetic>It may be worth more clearly defining the threat 
> model.</trite>

I didn't have time to write this down, but I don't think that we need to worry so much about your attack.

My threat model is similar to that of CORS.  That is, the origin that is accepting - in this case content and state, in CORS a request - something should not be ignorant of what it is accepting.

The reason I ended up at double-hashing was that a hash of the content is all that is really available.  Having the server demonstrate that it is willing to accept a bundle without it being susceptible to making mistakes.  After all, the bundle gets to write into the origin (and potentially exfiltrate from it after the transfer occurs).  For something like a bundle, which encompasses a lot of things, having the server indicate that it knows the exact bundle is likely the best way to achieve some indication that it knows what it is getting into and consents to that.  With a hash, you can split it, share part of it, and ask for the remainder.  Or double-hash like a one-time password.

If a bundle were signed, proof of ownership of the signing key might also work.

In the end, if the bundle and origin want to collude I am of the opinion that that is fine.  An origin that is willing to cooperate with the bundle is intentionally exposing itself to attacks on the integrity of its own state.  Going to the extremes you describe is taking great care to aim your footgun. So it's not really worth stopping in my view.

(I don't think that for a secure hash you can do anything like what you describe with fixpoints.  Such a thing would seem to lead to trivial collision attacks.  What seems more likely to me is that the bundle is constructed using some very precise canonicalization scheme such that it can be reconstructed from within and then hashed.  That might be quite restrictive, but I don't see why it wouldn't work.)

---

From: Devin Mullins <[email-masked]>
Date: Fri, 27 Mar 2020 13:06:29 -0700

> Slowly crawling out from a giant pile of email...

No worries, understood; my life is a giant pile of email.

> without it being susceptible to making mistakes ... if the bundle and
> origin want to collude I am of the opinion that that is fine

Ah, good to know. To clarify: Am I right in assuming my above proposed
attack is somewhere on the spectrum of
https://github.com/WICG/webpackage/pull/424 ? That is, it requires
publisher server-side work, but is a potential end-run around mitigations
to backchannel sharing such as per-origin IP obfuscation and fingerprint
entropy reduction. If so, then what motivates
https://github.com/martinthomson/wpack-content/issues/1 ?

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 30 Mar 2020 15:28:11 +1100

On Sat, Mar 28, 2020, at 07:06, Devin Mullins wrote:
> publisher server-side work, but is a potential end-run around 
> mitigations to backchannel sharing such as per-origin IP obfuscation 
> and fingerprint entropy reduction. If so, then what motivates 

This particular threat model is a really difficult one for me to get my head around properly, I have to admit.

I think that the key mechanism that this assumes we need to look at is navigation.  This appears to aim to protect is the transfer of information from the linking site to the target of a link.  In this case, from a distributor to a publisher.

Today, navigation carries information in essentially two obvious places: the URL and the Referer header (I think that's all).  Web packaging potentially adds another path whereby information can travel.  Namely the content of the bundle.

Assuming that a distributor is given license to generate bundles by the target origin, the linking party can pack any state it chooses into a bundle[1].  That information is then available to the target origin via requests to the resources in the bundle, or via requests to state established by the bundle.  It seems fairly obvious that the amount of state that is transferred could be considerable.

How the target origin receives this information follows similar lines to your email.  But in practice, it doesn't need to be that fancy.  If we are talking about hashes, the total amount of data involved isn't that much.  I don't think that a site would be particularly put off by the cost of transferring hashes to the sites that they link to.  32 bytes per user per target isn't that much data to save or transfer.  You can then rely on post-facto linkage. I'm sure that RTB systems transfer orders of magnitudes more than this as part of their normal operation.

(As an aside, this isn't exactly equivalent to giving the distributor control over your private key unless you exercise the options in footnote [1], but it comes close.  In any case, I don't believe that there is any material difference between this option and having the distributor ask the publisher to sign bundles containing the data on a per-request basis, aside from perhaps having better performance characteristics at bundle creation time.)

Let's say that we had a system that was able to detect and maybe block information transfer in URL or Referer[2].  Is the contention that this system would be unable to detect this sort of information transfer when it was general purpose content?

Separately, I'm about your stated desire to provide diversification of content for the purposes of experimentation here.  It seems to be in direct tension with this privacy requirement.  I'm interested in knowing how you might choose to trade the two off; I don't have any reference here as we haven't spent a whole lot of time looking at this specific problem.

Cheers,
Martin

[1] There is a variant of this that requires only a tiny bit more information to travel between distributor and publisher, but allows the publisher to more closely audit what is being shared.

[2] I don't know if this is possible in general, but I remain open to being convinced otherwise.  The best defense I can think of for this sort of system requires no such detection.  Instead, you try to prevent individual targeting by confirming that others have the same content. However, those designs are trivially vulnerable to sybil attacks.

... The webex calls we were on this week had a URL that was ~422 characters, most of it opaque.  Maybe that was critical information, I don't know.  But I am not sure it would be possible to tell if this contained information that was designed to de-anonymize someone.  Similarly, with only a fifth of the characters, the (up to) 244 bits of random gunk in a Google docs URL might all be used finding the document, but I could imagine that far fewer than this would be needed if someone wanted some bits spare.

---

From: Devin Mullins <[email-masked]>
Date: Mon, 30 Mar 2020 16:34:41 -0700

On Sun, Mar 29, 2020 at 9:28 PM Martin Thomson <[email-masked]> wrote:

> Separately, I'm about your stated desire to provide diversification of
> content for the purposes of experimentation here.  It seems to be in direct
> tension with this privacy requirement.  I'm interested in knowing how you
> might choose to trade the two off; I don't have any reference here as we
> haven't spent a whole lot of time looking at this specific problem.
>

 If you haven't seen it yet, my longer email has one proposal, but I
haven't thought about it too much; it may have holes. This is an area I
need to learn a lot more about.

> head around properly, I have to admit.

Only my guess, but it seems to be at least as much about re-balancing the
costs/benefits of bad actions as it is about blocking them (i.e. making the
costs infinite).

I don't mean to imply stake in any particular threat model; I think UA
vendors are in a better position to do so. (I will do my best to help
inform that decision with feasibility estimates for one use case.) Mostly
I'm trying to ensure there is rough consensus on this part. As a consumer
of the signed exchanges spec, I'm obviously interested in a successor spec
that has properties that most or all are satisfied with.

Let's say that we had a system that was able to detect and maybe block
> information transfer in URL or Referer[2].  Is the contention that this
> system would be unable to detect this sort of information transfer when it
> was general purpose content?
>

I don't know. It seems like there'd be a lot of ways to evade detection and
hide things in the URL that don't even require server changes. (For
example, many servers don't validate the URL slug, so it could be
steganographically altered and then inspected client-side.)

I don't think that a site would be particularly put off by the cost of
> transferring hashes to the sites that they link to.  32 bytes per user per
> target isn't that much data to save or transfer.  You can then rely on
> post-facto linkage.
>

Good point. This is mitigated by martinthomson/wpack-content#1, yes?

But AFAICT, this + //publisher.example/this-is-my-hash is not mitigated by
#1. That said, I guess the solution is that, for anti-fingerprinting, the
state transfer request and any previous subresource requests should be
considered in different buckets. The state transfer itself doesn't affect
the feasibility of the attack. The unverified bundle can make a fetch
to //publisher.example/distributor-id?foo, and even if the Sec-CO request
fails, the publisher can join foo with a subsequent credentialled fetch, in
order to inform future responses. The UA could consider this ~isomorphic to
the linking page making that same fetch to /distributor-id?foo.

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 31 Mar 2020 14:43:37 +1100

On Tue, Mar 31, 2020, at 10:34, Devin Mullins wrote:
> > I don't think that a site would be particularly put off by the cost of transferring hashes to the sites that they link to. 32 bytes per user per target isn't that much data to save or transfer. You can then rely on post-facto linkage.
> 

Not really.  The push can include the H(Content) and the site can calculate H(H(C)).  So the truncated hash only really limits the information the client has to include, trading it for more from the server.  Since opening that issue, I'm not convinced that it is worth pursuing for anything other than that trade-off (which is only a maybe-optimization at best).

> But AFAICT, this + //publisher.example/this-is-my-hash is not mitigated by #1. 

Right.  What *might* do that is the fact that a publisher willing to accept *any* bundle ends up being exposed to a broad spectrum of cross site forgery attacks.  It allows an attacker to supplant any script that the site might reference on its own origin.  You might be able to avoid that with a really strong CSP, but I'm not sure that would be a strong guarantee.

But as I mentioned before, there are ways to partition resources such that a site can validate a bundle without taking on that risk.  And signing doesn't help significantly, except to add a signature to the 64+ bytes per user per target cost.

> That said, I guess the solution is that, for 
> anti-fingerprinting, the state transfer request and any previous 
> subresource requests should be considered in different buckets.

One thing that we are seeing in some contexts is not just completely different connections for credentialed vs. uncredentialed requests, but proxying of uncredentialed requests, so that the amount of fingerprinting information is limited.

> The state transfer itself doesn't affect the feasibility of the attack. The 
> unverified bundle can make a fetch to 
> //publisher.example/distributor-id?foo, and even if the Sec-CO request 
> fails, the publisher can join foo with a subsequent credentialled 
> fetch, in order to inform future responses. The UA could consider this 
> ~isomorphic to the linking page making that same fetch to 
> /distributor-id?foo.

Right.  That's where this gets really hairy.  But this all goes to challenging the feasibility of the goal of preventing information transfer across origins.  The fact that this is a content-based origin doesn't change the dynamics at all.  If origins are willing to cooperate in the exchange of information, then I don't know how to prevent that.  It might be possible to make things sufficiently unlinkable, but even for something with protections as restrictive as Tor Browser I doubt that you can prevent transfer.

(That's a challenge, BTW, because I would be very happy to be shown to be wrong.)

---

From: "Martin Thomson" <[email-masked]>
Date: Tue, 31 Mar 2020 15:41:00 +1100

Wow, that's a long list.

I think that a lot of this relates to the inherent difficulty of providing customization under a regime like was described.

This includes all the experimentation things, CCPA/GDPR consent notices [1], analytics, loading resources with referring origin-based ACLs [2], personalization of content, etc...

This is something that a signature-based system helps with because you can front-load all of this into custom bundles that are all signed with the same key, but you treat all of these as equivalent.

I think that Ekr and I both separately mentioned in on the call last week, but it's worth writing this down somewhere (because I failed to do so thus far).  There is a variant of this design that uses signature public keys as the primary identifier and considers all content signed with the corresponding private key as the one co-extant origin.  That avoids most of this mess.  There are a few challenges, but as the liveness test (Sec-Signer-Origin maybe) remains, concerns about revocation are approximately the same as it pertains to protecting the target origin.

The reputation issue remains, as the granularity of key use becomes imperative, and I don't think we should accept any signer-based origins as valid if the the same key is used for a different target origin or a certificate.  In other words, don't cross the streams: if you want to attest as example.com that this content is yours, then you will need to use a different key for that purpose.  (I suggest we continue to discuss that question on the other thread.)

The other question you raise is the difficult of implementation for this.  I don't know to what extent state is used in frameworks and sites, so I don't have a good handle on this.  The suggestion to keep a record of pre-transfer state is a good one that seems immediately useful.  However, for the general problem, I posited offline that there are a few cases in which merge strategies could be trivial:

1. Exclusive access means no merges: If a content-based origin has exclusive ownership over a particular store, then merges are trivial.  This is true for a single content-based origin that transitions to a target origin: you can likely assume that the target origin has no pre-existing state before the transfer.  You only have to worry about multiple different content-based origins that write to the same logical store, or a content-based origin that doesn't transfer to the target origin for some reason.  It is quite likely that many transfers will take information for which the target origin has no pre-existing state.

Signer-based origins are more likely to enjoy exclusive access, though we do have to admit the possibility of a single target origin having multiple signing keys and so have multiple sets of state to reconcile, but it might also be possible to have good alignment between access to stores and signing keys to avoid the need for merging.

2. Simple merge strategies: If state is trivial, then very simple merge strategies can be effective with very little effort.  Say that you had a control that set a preference.  That might be a boolean (dark mode on/off), or something more complex.  Last change wins is a perfectly workable strategy in case of conflict.  A less simple strategy is to record discrete changes with timestamps and then taking changes in time order.

Finally, for non-trivial state, my hypothesis is that servers want to record that state server-side, so the question becomes whether any sort of offline modification of that state is desirable anyway.  At the point that a site wants that capability, then yes, this is engineering.  But that is engineering they have to undertake anyway at that point.

[1] The informed consent doctrine is something I think is despicable, but I can't deny the force of law.
[2] Not secure in any real sense, but this method seems to be pretty effective in practice.

On Wed, Mar 25, 2020, at 17:35, Devin Mullins wrote:
> reviewed this, had some chats with others on the AMP team, and had a 
> few back-and-forths with Martin (he's been very gracious in engaging 
> his time, thank you). I'm trying to estimate the feasibility of this, 
> both for the AMP use-case, and for the non-AMP use-case [1], which we 
> hope to see flourish to the extent that publishers and users desire it, 
> and only impeded by implementation costs where necessary for the common 
> good.
> 
> that, with some modifications, it seems mostly feasible for AMP. 
> However, there are two main downsides:
> A lot of details follow. I suggest to rename the subject if you reply 
> to a detail below, in order to reduce cross-talk and make browsing the 
> archive easier.
> 
> Details:
> 
> transfer. I suspect a flash in the URL bar would be frustrating for 
> both users and publishers. Jeffrey had proposed using signatures for 
> this. I'd suppose UAs could choose to render this with a "may be stale" 
> indicator.
> 
> resource iff it matches the Sec-CO request. I suspect that keeping 
> state of past versions of the resource would be infeasible, especially 
> as it couldn't be done purely at the edge; it would require something 
> like a region-wide cache. The value of the above stateless 
> implementation would be inversely proportional to how often the 
> resource changes (e.g. "transient content" [2] such as Related 
> Re: fallback behavior on failed state transfer, distributors would need 
> a way to monitor failure rates. This serves two purposes:
>  1. Automatically detecting errors in the distribution pipeline. This 
> is the same purpose served by 
> meeting AMP's stated UX goals (by using the verified content in the 
> bundle and not potentially arbitrary content after the redirect). This 
> is a dynamic equivalent of what can be done mostly statically with SXG, 
> because the distributor can run the same algorithm as the UA, modulo 
> client/server skew in clocks and root stores. I think this is a 
> trade-off AMP could make. The alternative is that the fallback behavior 
> keeps the content-based origin (CBO).
> 
>  1. Delay rendering content under experiment until after state 
> transfer. This hurts UX for the sake of experimentation, creating a 
> trade-off that otherwise doesn't exist on origin (modulo ease of 
> implementation).
> the pre-existing user ID after transfer. This means the session could 
> not include CBO pageviews anywhere except in the first pageview.
> distributor chooses which bundle to send to the user (based on what? 
> not sure). For a page with M independent experiments each with N arms, 
> that's N^M variants. For large enough M and N, this is likely both 
> infeasible for publishers and martinthomson/wpack-content#1. (But I 
> need some guidance from experts on typical ranges for M and N.)
> 
> On the one hand, perhaps this is too much scope for wpack. On the 
> other, it addresses a problem that is somewhat particular to CBOs.
> 
> https://amp.dev/documentation/components/amp-analytics/ and many 
> non-AMP libraries) would have to do one of two things:
>  2. Generate a client-side UUID, send pingback before transfer, and 
> send a follow-up after transfer to join with pre-existing user ID. This 
> captures those otherwise lost pingbacks pseudonymously (anonymously?), 
> though it requires server-side work on the pingback endpoint.
> 
> fire or finish, and it would be good to later scan for stragglers to 
> merge or expunge.
> suggests an optimization where the encrypted subresource is loaded 
> early, and only the decryption key after transfer.)
> 
> rolling release, it is possible to upgrade many use-cases in the wild 
> with minimal publisher effort.
> but the requirement to upgrade versions may limit applicability. 
> Because upgrades occur less often, they often require more work.
> From a "minimal publisher effort" perspective, I am especially 
> interested in making it possible to have as close to a turnkey solution 
> as is reasonable, for instance at the level of the CDN or CMS. If such 
> a solution exists, sites would likely contain a hybrid of all three 
> cases above, and thus want to opt into such support incrementally. How 
> should a CDN determine which pages to provide Sec-CO responses for?
> I don't have a good solution to the issue of non-AMP developer cost, 
> and thus fear this is a solution that would feature a disproportionate 
> amount of AMP pages compared to the background distribution of the web. 
> But I'm hopeful that other JS framework developers can chime in with 
> respect to feasibility of support for CBO and state transfer.
> 
> better send it early enough so at least a few people have time to read 
> it before the meeting.
> 
> Thanks,
> Devin
> 
> [1]