Subject: [451] #80: Distinguishing intermediaries from origins

From: Mark Nottingham <[email-masked]>
Date: Mon, 24 Aug 2015 16:02:12 +1000

<https://github.com/httpwg/http-extensions/issues/80>

This is the only non-editorial issue open on this draft.

We discussed it in Prague, but didn't make much headway. I think there's agreement that it's important to make the distinction, but other than that there wasn't much direction from the room.

To move things along, I'll put forth three straw-man proposals:

1. Specify that 451 is for origin server use only; if the network censors something and is allowed to state that this is happening, it'll need to use a different (possibly defined in the future) status code.

2. Specify a second status code (452?) to indicate that the network is doing the censoring.

3. Specify a header (Status-Source?) to disambiguate whether the code is being generated by the network or the origin.

Thoughts?

If we don't come to any conclusion, #1 seems like the fallback, since #2 or #3 can always be added later.

Cheers,

--
Mark Nottingham   https://www.mnot.net/

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 24 Aug 2015 06:27:45 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>[...]
>1. Specify that 451 is for origin server use only; if the network 
>censors something and is allowed to state that this is happening, it'll 
>need to use a different (possibly defined in the future) status code.
>

Defining "the network" in a context like this is nailing gruel to a wall.

Is a CDN an origin server or "the network" ?

What if some of the CDN nodes must conform to different laws than
the rest due to legal reasons relating to $whatever ?

Is a web-hosting provider with an Acceptable Use Policy an origin
server or "the network" in this context ?

More fundamentally, I fail to see where the semantic difference
between 451 and 452 would ever matter enough to justify a distinction.

And worse:  I can see a lot of perverse incentives to use the wrong
one to deflect blame, making the distinction useless in practice.

I'd prefer we stick with just 451.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
[email-masked]         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 24 Aug 2015 16:31:20 +1000

> On 24 Aug 2015, at 4:27 pm, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

It's not that hard. if you're the origin server as per HTTP (i.e., your hostname shows up in the URL) — whether it's a CDN or not — you're the origin. Otherwise, you're not.

One way to do #3 would be to require / request that the host generating the status code put its name in a header; that would help disambiguate (as long as the origin name was used, and not a deployment name).

Cheers,

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 24 Aug 2015 06:53:34 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>It's not that hard. if you're the origin server as per HTTP (i.e., your 
>hostname shows up in the URL) - whether it's a CDN or not -
>you're the origin. Otherwise, you're not.

I think the (...) says the exact opposite of what you want:  Most
servers have a single hostname which never appear in URLs, but
serves many URL domains.

But inverting it "(i.e., the domainname in the URL points to your
server)", to match what I think was your intent, isn't much better.

For instance captive gateways where wildcard DNS responses point
everything to a proxy would become 451 rather than 452.

It also leaves the webhosting company to send 451 for AUP reasons,
even though they will disclaim any legal authority over the content
they serve.

As I said:  Nailing gruel to the wall.

>One way to do #3 would be to require / request that the host generating 
>the status code put its name in a header; that would help disambiguate 
>(as long as the origin name was used, and not a deployment name).

I doubt that would happen in reality, there are big incentives to lie.

Leave it at 451 and recommend that the body contains useful details.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 24 Aug 2015 17:02:26 +1000

> On 24 Aug 2015, at 4:53 pm, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

Yes, but they know what origin each request has, and whether or not they're acting as the origin.

> everything to a proxy would become 451 rather than 452.

A captive portal knows it's a captive portal. Also, we already have a status code for what they do.

> even though they will disclaim any legal authority over the content

That's probably worth more discussion; DMCA comes to mind as well in this situation.

> Leave it at 451 and recommend that the body contains useful details.

It can (and hopefully will) regardless. The discussion here was to see whether it'd be useful to make the distinction clear for automated clients (e.g., Chilling Effects, robots); the response has been "yes, that would be useful."

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 24 Aug 2015 07:07:24 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>It can (and hopefully will) regardless. The discussion here was to see 
>whether it'd be useful to make the distinction clear for automated 
>clients (e.g., Chilling Effects, robots); the response has been "yes, 
>that would be useful."

I'm sure they would find it useful.

But only if the censors return 452 to begin with.

Did anybody ask them if they're going to ?

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 24 Aug 2015 17:12:02 +1000

> On 24 Aug 2015, at 5:07 pm, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

Yeah, that question has come up before, and it's a good one. 

I suspect there will be some jurisdictions where ISPs and other network providers would use such a status code, because prohibiting them from doing so would be seen as overreaching. I also suspect there would be many jurisdictions where this wouldn't be the case, and it wouldn't be used at all.

A good part of the motivation here AIUI is to keep the 451 signal "clean" so that ISP censorship doesn't get confused with origin censorship. That only requires that network providers *don't* use 451, not that they proactively use 452 (or any other technical means). 

Cheers,

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 24 Aug 2015 07:25:07 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>I suspect there will be some jurisdictions where ISPs and other network 
>providers would use such a status code, because prohibiting them from 
>doing so would be seen as overreaching.

But that only answers "will they be allowed to?", it doesn't answer
"would they be willing to?" which I think is the much more important
question.

>A good part of the motivation here AIUI is to keep the 451 signal 
>"clean" so that ISP censorship doesn't get confused with origin 
>censorship.

For the ISP there would be considerable benefits to making it look
like origin censorship:  It would reduce help-desk workload, it
would deflect blame for a controversial practice away from the ISP
etc.  etc.

Of course, we can have perverse incentives too:  If we add 452 and
give 451 a strong smell of origin censorship, we will probably
see more 451 responses, than we would without 452 :-)

But I still don't think adding 452 will be an improvement.

---

From: [email-masked]
Date: Mon, 24 Aug 2015 10:11:38 +0200 (CEST)

> would deflect blame for a controversial practice away from the ISP

That only works if the user considers the signal to be reliable. If one lies about it he won't.

So to actually reduce helpdesk workload an ISP needs to be very clear and clean on who blocks what, because otherwise people will just call the helpdesk by default.

-- 
Nicolas Mailhot

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 24 Aug 2015 08:19:31 +0000

--------
In message <[email-masked]>, nicol
[email-masked] writes:
>So to actually reduce helpdesk workload an ISP needs to be very
>clear and clean on who blocks what, because otherwise people will
>just call the helpdesk by default.

And sending "451 Not Allowed by Origin" would certainly do that.

---

From: Tim Bray <[email-masked]>
Date: Tue, 25 Aug 2015 08:49:11 -0700

Interesting.  I’ve been thinking of this in the context of my current day
job at AWS. [OBLIGATORY DISCLOSURE: Not speaking for. Not asking anyone.
They probably think I’m crazy, etc]

Someone cold have a web site pasted together with a combination of s3
static hosting, Lambda API gateway, various database services, CloudFront,
and route53 DNS. Except for, lots of people don't, they get those services
from lots of different vendors; I offer the list to outline how many pieces
make up a web app.  Now, a legal threat might be made against pretty well
any link in this chain, and figuring out whether to use 451 or 452 could be
hard.  BUT, it would be useful to know where the legal blockage is
happening.

451's technical goal is to allow crawlers and other automated agents to
detect and report on legal blockages.  For such tracking, there are two
things it would be useful to know:
1. The resource being blocked
2. Who actually is interposing the blockage

The combination of the URI and the 451 code take care of #1.

As for #2, this feels like a job for an HTTP header, say
“Blocker-for-legal-reasons”.  It should take a list value in the case that
there are multiple entities blocking a request, something that wouldn't be
terribly surprising.  That leaves the question of what sort of values are
appropriate to identify the entities doing the blocking.  URIs are the most
obvious candidates.

Anyone got a pointer to an I-D or RFC that they think does a good job of
specifying a new header?  Rather than argue about abstractions, I’ll draft
up some language to see what such a header would look like.

-- 
- Tim Bray (If you’d like to send me a private message, see
https://keybase.io/timbray)

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Tue, 25 Aug 2015 15:57:22 +0000

--------
In message <CAHBU6is4=_WZG=[email-masked]>
, Tim Bray writes:

>BUT, it would be useful to know where the legal blockage is happening.

No argument there, but adding 452 doesn't aid that goal IMO.

>'Blocker-for-legal-reasons'.  It should take a list value in the case that
>there are multiple entities blocking a request, [...]

How would that ever happen ?  You get 451 back from the first censor
who doesn't explore the full chain to see if anybody else are
censoring ?

Again:  Have you asked any actual *censors* if they would be willing
to fill out such a header ?

---

From: Tim Bray <[email-masked]>
Date: Wed, 26 Aug 2015 08:07:51 -0700

On Tue, Aug 25, 2015 at 8:57 AM, Poul-Henning Kamp <[email-masked]>
wrote:

> ​​
> ​​
> ​​

​I agree. I don't think 452 really adds any useful information, because I
think “origin host” vs “other” distinction is not very interesting. I think
it's useful to know who is censoring.​

​​

> who doesn't explore the full chain to see if anybody else are

​Hm... possibly right. Other opinions?​
​​

> ​​
> ​​

​I'm pretty sure they would.  Lots of ISPs hate it when they are prevented
from serving their customers properly.  Also, see the transparency reports
published by all the big internet companies.​

​​

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Wed, 26 Aug 2015 16:15:10 +0000

--------
In message <[email-masked]>, Tim Bray writes:

>I'm pretty sure they would.  Lots of ISPs hate it when they are prevented
>published by all the big internet companies.

Well, we were pretty sure people would adopt IPv6 in 1998 too :-)

I'd like to see at least one censor saying "Yes, we will do this"
before we put more fat on HTTP

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 27 Aug 2015 11:41:22 +1000

> On 27 Aug 2015, at 1:07 am, Tim Bray <[email-masked]> wrote:
> 
> On Tue, Aug 25, 2015 at 8:57 AM, Poul-Henning Kamp <[email-masked]> wrote:
> ​​>BUT, it would be useful to know where the legal blockage is happening.
> ​​
> ​​No argument there, but adding 452 doesn't aid that goal IMO.
> 
> ​I agree. I don't think 452 really adds any useful information, because I think “origin host” vs “other” distinction is not very interesting.

Well, we do have responses from people who intend to consume these status codes that such a distinction would be interesting to them. 

I do see your point though — if I'm being censored because I'm in Fooistan, and the content is available elsewhere, the place of application of the censorship (network vs. origin) doesn't seem like a primary concern (because in both cases, changing my network path may result in the content becoming available).

> I think it's useful to know who is censoring.​

If we got ambitious, we could define a Censorship Reporting Format to describe who's doing the censorship, what it applies to, etc. and then Link: to that from the response. I don't think we're that ambitious here (and it's starting to sound out of scope for this WG).

So, I think what remains is "is there any dead-simple metadata we can put into the 451 response that will help people collecting information about censorship? If not, we should probably just ship it.

CC:ing in Wendy, Niels, Joe and DKG to see if they have any thoughts.

> ​​Again:  Have you asked any actual *censors* if they would be willing
> ​​to fill out such a header ?
> 
> ​I'm pretty sure they would.  Lots of ISPs hate it when they are prevented from serving their customers properly.  Also, see the transparency reports published by all the big internet companies.​

FWIW, I've been in contact with a few (quite sizeable) sites that say they'd use such a status code if it were standard.

I haven't yet had contact with ISPs that say that, but it's mostly because I haven't asked. Based on the current political climate in .au and .uk, I think at least ISPs in those places would use a status code if it were defined. Happy to try to get some feedback gathered if people think it'd help.

Cheers,

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 27 Aug 2015 16:48:19 +1200

On 27/08/2015 1:41 p.m., Mark Nottingham wrote:
> 
>> On 27 Aug 2015, at 1:07 am, Tim Bray wrote:
>>
>> On Tue, Aug 25, 2015 at 8:57 AM, Poul-Henning Kamp wrote:

We have a lot of interest from us proxy people here in the WG for
consuming Transfer-Encoding:gzip from browsers a few years back. I'm
still not seeing it happen and the emitters are supposed to have a 6-12
week update cycle.

So I'm with PHK on this point. If the emitters are not clearly
signalling (pun intended, all meanings seem to apply) there is no point
in adding more status codes for them to use. The sub-header or payload
clarfication should be enough if they want/need it.

On Tims point about transparency reports:

> On 27/08/2015 3:07 a.m., Tim Bray wrote:

I think if they are the main (only?) ones willing to emit a status code
then 451 already serves it purpose by providing that. The other parties
are keeping silent and far more likey to lie when they do signal at all.

> If we got ambitious, we could define a Censorship Reporting Format to describe who's doing the censorship, what it applies to, etc. and then Link: to that from the response. I don't think we're tha…

This is also what I have been having in mind. But is also something
*they* (the censors, emitters and consumers of 451) can define between
themselves without us HTTP people needing to be directly involved with
wrangling political/legal definitions.

> So, I think what remains is "is there any dead-simple metadata we can put into the 451 response that will help people collecting information about censorship? If not, we should probably just ship i…

I think the answer to that is clearly "No". Ship it.

Amos

---

From: Tim Bray <[email-masked]>
Date: Wed, 26 Aug 2015 22:38:32 -0700

On Wed, Aug 26, 2015 at 6:41 PM, Mark Nottingham <[email-masked]> wrote:

> ​​
> I do see your point though — if I'm being censored because I'm in
> Fooistan, and the content is available elsewhere, the place of application
> of the censorship (network vs. origin) doesn't
> ​​
> seem like a primary concern (because in both cases, changing my network
> path may result in the content becoming available).
>

​I’m actually more interested in automated agents doing tracking, who would
like to track
(a) which resources are blocked, and
(b) who's blocking

For humans, the most useful thing is the response-body that can be sent
along with 451.​
​​

> If we got ambitious, we could define a Censorship Reporting Format to
> describe who's doing the censorship, what it applies to, etc. and then
> Link: to that from the response. I don't think we're that ambitious here
> (and it's starting to sound out of scope for this WG).
>

​Bah.  Use a URI.  You’re not going to improve on that any time soon.​  I
owe the WG draft language.

---

From: Tim Bray <[email-masked]>
Date: Sun, 30 Aug 2015 10:07:28 -0700

I made a quickie cut of draft-ietf-httpbis-legally-restricted-status-02 at
https://www.tbray.org/tmp/451-02.html

The only difference is the new section 4 “Identifying Blocking Entities”,
https://www.tbray.org/tmp/451-02.html#rfc.section.4 and a header
registration in IANA considerations.

I think this is an effective way not only to distinguish intermediaries
from origins but to answer the more useful question “who is actually
blocking access?”

On Wed, Aug 26, 2015 at 10:38 PM, Tim Bray <[email-masked]> wrote:

> ​I’m actually more interested in automated agents doing tracking, who
> would like to track

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Sun, 30 Aug 2015 17:50:26 +0000

--------
In message <CAHBU6ita_a5OeMX8QmLw+Vb=O767VmwjHddLamLyv_QL=+=[email-masked]>
, Tim Bray writes:

>from origins but to answer the more useful question 'who is actually
>blocking access?'

I think this is a much better solution that 452.

---

From: Julian Reschke <[email-masked]>
Date: Mon, 31 Aug 2015 08:41:13 +0200

On 2015-08-30 19:07, Tim Bray wrote:
> I made a quickie cut of draft-ietf-httpbis-legally-restricted-status-02
> at https://www.tbray.org/tmp/451-02.html
>
> The only difference is the new section 4 “Identifying Blocking
> Entities”, https://www.tbray.org/tmp/451-02.html#rfc.section.4 and a
> header registration in IANA considerations.
> ...

Which is:

> 4. Identifying Blocking Entities
>
> As noted above, when an attempt to access a resource fails with status 451, the entity blocking acces may or may not be the origin server. There are a variety of entities in the resource-access path which could choose to deny access, for example ISPs, cache providers, and DNS servers.
>
> It is useful, when legal blockages occur, to be able to identify the entities actually implementing the blocking.
>
> For this purpose, this specification registers the "Blocked-by" HTTP header, whose value identifies the blocking entity. When an entity blocks access to a resource and returns status 451, it SHOULD include a "Blocked-by" header identifying itself.
>
> The value of the "Blocked-by" header is a single URI-reference. When it has the form of a relative reference [RFC3986] the final value is computed by resolving it against the effective request URI.

I think this is the right direction; the alternative of course would be 
a link relation...

Editorial nitpicking:

- s/acces/access/

- avoid lowercase RFC2219 keywords (use "might"?)

- s/HTTP header/HTTP header field/

- s/URI-reference/URI-reference (Section 4.1 of [RFC3986])/

- s/relative reference [3986]/relative reference (Section 4.2 of [RFC3986])/

- s/effective request URI/effective request URI (Section 5.5 of [RFC7230])/

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 31 Aug 2015 16:40:34 +1000

Hi Tim,

If we're going to specify a new header field, it'll need ABNF defined, etc.; see <http://httpwg.github.io/specs/rfc7231.html#considerations.for.new.header.fields>.

Before we get to that, though — is there a compelling reason not to use the Link header for this?

Cheers,

> On 31 Aug 2015, at 3:07 am, Tim Bray <[email-masked]> wrote:
> 
> I made a quickie cut of draft-ietf-httpbis-legally-restricted-status-02 at https://www.tbray.org/tmp/451-02.html
> 
> The only difference is the new section 4 “Identifying Blocking Entities”, https://www.tbray.org/tmp/451-02.html#rfc.section.4 and a header registration in IANA considerations.
> 
> I think this is an effective way not only to distinguish intermediaries from origins but to answer the more useful question “who is actually blocking access?”
> 
> ​​I do see your point though — if I'm being censored because I'm in Fooistan, and the content is available elsewhere, the place of application of the censorship (network vs. origin) doesn't ​​seem like a primary concern (because in both cases, changing my network path may result in the content becoming available).
> 
> ​I’m actually more interested in automated agents doing tracking, who would like to track
> For humans, the most useful thing is the response-body that can be sent along with 451.​
> ​​
> ​Bah.  Use a URI.  You’re not going to improve on that any time soon.​  I owe the WG draft language.
> 
> 
> 
> 
> -- 
> - Tim Bray (If you’d like to send me a private message, see https://keybase.io/timbray)

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 31 Aug 2015 06:49:49 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>Before we get to that, though =E2=80=94 is there a compelling reason not 
>to use the Link header for this?

The Link header could theoretically be used to point to non-blocked versions
of the content, but I don't think that schenario is likely.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 31 Aug 2015 16:53:14 +1000

> On 31 Aug 2015, at 4:49 pm, Poul-Henning Kamp <[email-masked]> wrote:
> 
> --------

That's not what I meant. The Link header is a way to put a URL into HTTP headers, taking care of all of the syntax issues, HTTP corner cases, URL issues like what the base URL is, and giving us an opportunity to more easily optimise the syntax if we decide to do that in HTTP/3. 

In this case, instead of doing:

  Blocked-By: http://example.com/

we'd do:

  Link: <http://example.com>; rel="blocked-by"

There are some reasons why you wouldn't use a Link header for some things, especially if a recipient needed to be able to parse it out very efficiently, and there might be many other Link headers present. I don't think that this is one of those cases, though.

Cheers,

---

From: "Poul-Henning Kamp" <[email-masked]>
Date: Mon, 31 Aug 2015 08:56:53 +0000

--------
In message <[email-masked]>, Mark Nottingham wri
tes:

>That's not what I meant.

I know.  You asked for compelling reasons *not* to use Link and I
gave you the most (but not very) compelling reason not to do so. :-)

I fully agree, we should just use Link for the 451 reason/authority URI

---

From: Tim Bray <[email-masked]>
Date: Mon, 31 Aug 2015 19:11:01 -0700

Sounds like a winner to me.  I’ll do another draft.