Subject: Status code for censorship?

From: Tim Bray <[email-masked]>
Date: Sat, 9 Jun 2012 22:05:39 -0700

Check out
http://yro.slashdot.org/story/12/06/09/1927246/an-http-status-code-for-censorship

The thinking about returning 403 when you’re forbidden to follow a link
seems sound to me.  This idea is superficially appealing; is it deeply
broken in some way that’s not obvious?  -Tim

---

From: Martin Thomson <[email-masked]>
Date: Sat, 9 Jun 2012 22:39:03 -0700

> broken in some way that’s not obvious?  -Tim

The temptation to suggest 418 is strong, but 403 is essentially
correct.  The entity making the authorization decision might not be
the usual or expected one, but that is the decision they are making.

--Martin

p.s. It is less about following the link than it is about interacting
with the resource identified by that link.

---

From: Yoav Nir <[email-masked]>
Date: Sun, 10 Jun 2012 09:53:16 +0300

The value of status codes is in aiding trouble-shooting. So using a different code for when a MitM has made the decision, as opposed to when the server has made the decision seems appropriate.

If you get a 403, you clear cookies, try again, and send an email to thepiratebay.
If you got a number that shows a policy decision by the government (probably enforced by the ISP), you send an email to your congressman/MP/vote for someone else.

Maybe we need several codes, for government mandate, decision by ISP, decision by your employer (implemented on a perimeter firewall), and for a policy you've asked for (like netnanny)

-----Original Message-----
From: Martin Thomson [mailto:[email-masked]] 
Sent: 10 June 2012 08:39
To: Tim Bray
Cc: [email-masked]
Subject: Re: Status code for censorship?

On 9 June 2012 22:05, Tim Bray <[email-masked]> wrote:
> The thinking about returning 403 when you're forbidden to follow a 
> link seems sound to me.  This idea is superficially appealing; is it 
> deeply broken in some way that's not obvious?  -Tim

The temptation to suggest 418 is strong, but 403 is essentially correct.  The entity making the authorization decision might not be the usual or expected one, but that is the decision they are making.

--Martin

p.s. It is less about following the link than it is about interacting with the resource identified by that link.

---

From: J Ross Nicoll <[email-masked]>
Date: Sun, 10 Jun 2012 10:30:26 +0100

Technically, the request is not intended to be repeated if you see a 403 
( "Authorization will not help and the request SHOULD NOT be repeated." )

My interpretation has always been that 403 is a fairly broad-scoped 
error code, and this is not accidental. If there's a need to provide 
machine-readable detail, I would suggest a new header is the correct answer.

Although surely the correct answer here is that the connections should 
be rejected at the TCP level, anyway.

---

From: David Morris <[email-masked]>
Date: Sun, 10 Jun 2012 08:54:08 -0700 (PDT)

On Sun, 10 Jun 2012, J Ross Nicoll wrote:

> Although surely the correct answer here is that the connections should be
> rejected at the TCP level, anyway.

It is kind of hard to report/enforce a content based decision at the TCP
level with any kind of meaningful diagnosic information. Many forms of
censorship are based on the bytes on the wire (content, URL, etc.) so
aborting the connection is about the only choice. Highly likely that
the request will be repeated.

---

From: James M Snell <[email-masked]>
Date: Sun, 10 Jun 2012 11:51:21 -0700

Quite honestly, while 403 would probably work just fine, I think a
dedicated status code in the 5xx range would make for a better approach, if
only from an informational point of view.

HTTP/1.1 512 Service Blocked

Sends a very clear message and makes the fact that the service is being
censored, as opposed to merely being technically unavailable, quite clear.

---

From: Yoav Nir <[email-masked]>
Date: Sun, 10 Jun 2012 22:55:29 +0300

Thinking it over, the message is supposed to be sent to humans, not machines. So the correct thing would be to redirect to a service-provider hosted page, that says what was blocked and why.

"Access to thepiratebay.se<http://thepiratebay.se> is blocked at the request of xxxxx, according to section yyyyyyy of zzzzzzz.  Sorry for the inconvenience"

On Jun 10, 2012, at 9:51 PM, James M Snell wrote:

Quite honestly, while 403 would probably work just fine, I think a dedicated status code in the 5xx range would make for a better approach, if only from an informational point of view.

Sends a very clear message and makes the fact that the service is being censored, as opposed to merely being technically unavailable, quite clear.

On Jun 9, 2012 10:09 PM, "Tim Bray" <[email-masked]<mailto:[email-masked]>> wrote:
Check out http://yro.slashdot.org/story/12/06/09/1927246/an-http-status-code-for-censorship

The thinking about returning 403 when you’re forbidden to follow a link seems sound to me.  This idea is superficially appealing; is it deeply broken in some way that’s not obvious?  -Tim

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Sun, 10 Jun 2012 22:34:49 +0200

On Sun, Jun 10, 2012 at 11:51:21AM -0700,
 James M Snell <[email-masked]> wrote 
 a message of 49 lines which said:

> dedicated status code in the 5xx range would make for a better
> approach,

But in the 4xx range, as discussed in the Slashdot article, we have a
much better candidate:

HTTP/1.2 451 This site has been burnt.

---

From: Tim Bray <[email-masked]>
Date: Sun, 10 Jun 2012 14:02:33 -0700

It’d have to be in the 4xx not 5xx range; it’s not a server error; similar
to 401/403, but distinct.  I actually think this is a good idea; will write
a formal proposal.

-T

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 11 Jun 2012 09:20:13 +1000

4xx status codes indicate that the problem is at the client; 5xx indicates that the error is at the server.

Since HTTP is a layer 7 protocol, not layer 8 (or 9, or 10, depending on how you're counting; I go by the t-shirt), there are going to be arguments for both sides.

If we had to choose, I'd go with a 5xx, because there's clear precedent for intermediary-related errors occurring there (most recently, 511), and this is essentially an intermediary problem. Something like "512 Network Policy Violation". 

However, it'd need to get deployed. Since the primary objective of such a status code seems to be to raise the visibility of censorship, thereby (presumably) causing some reaction against it, it seems unlikely that it'll happen in places where there's strong oversight of whoever is deploying it.

Also, there's the question of whether the IETF should do this; by allocating a status code for censorship, we'd be explicitly supporting its use. Given the IESG's stance on similar issues in the past (e.g., <http://www.ietf.org/rfc/rfc2804.txt>), I suspect adding a status code for censorship would at least require some serious discussion at that level, if not a plenary session, etc.

Cheers,

P.S. The interesting part, for me, is that this is a perfectly valid use of 403, according to our specs:

"The server understood the request, but refuses to authorize it."

Note "server", not "origin server" -- which in both 2616 and bis means *any* server in the chain can send it. 

We also talk about transforming proxies, but the most that's required in this instance is to add a 214 Transformation Applied warn-code. Even then, it's not clearly required, because it's defined in terms of changing the representation, not giving what's effectively a status message:

"""
3.6.6 214 Transformation Applied

must be added by a proxy if it applies any transformation to the representation, such as changing the content-coding, media-type, or modifying the representation data, unless this Warning code already appears in the response.
"""

--
Mark Nottingham   http://www.mnot.net/

---

From: Nicolas Mailhot <[email-masked]>
Date: Mon, 11 Jun 2012 10:19:48 +0000 (UTC)

Yoav Nir <ynir@...> writes:

> Thinking it over, the message is supposed to be sent to humans, not machines.
So the correct thing would be to redirect to a service-provider hosted page,
that says what was blocked and why.

1. browsers will silently block the redirect if the original request was for an
https URL, so that does not work

2. some dumb clients will loop over redirects they don't understand

3. dumb web clients can not parse a complex html page and need the response in a
header or something simple they can grab and relay elsewhere (for example a fat
client that uses curl or some other http lib: curl needs to grab the message so
the fat client can display it in a popup)

4. the message is not always a simple no (at a conference/school guests will
have limited accesses but staff no so the message could be 'go authentify
yourself there to prove you're staff before I let you access what you requested')

5. the 'authentify there' can vary depending on what you're trying to access
(not all network traffic will be routed through the same gateways)

6. ideally the message needs to be signed by the requesting authority so it can
not be spoofed by truants

---

From: Karl Dubost <[email-masked]>
Date: Mon, 11 Jun 2012 06:54:47 -0400

For me the main argument against its implementability is that:

> However, it'd need to get deployed. Since the primary objective of such a status code seems to be to raise the visibility of censorship, thereby (presumably) causing some reaction against it, it se…

That said, I guess, and without malice, that Mark had in mind a state like China, Iran, etc. Censorship happens sometimes in other states through decision of laws and/or regulations. For example, a network in a corporate environment blocking certain sites through proxy (such as social networks). A library blocking some sites through proxy to other users. In these cases, the organization in charge of it might want to advertise it for reasons which seem perfectly legible to them.

-- 
Karl Dubost - http://dev.opera.com/
Developer Relations, Opera Software

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 11 Jun 2012 21:09:27 +1000

On 11/06/2012, at 8:54 PM, Karl Dubost wrote:

> That said, I guess, and without malice, that Mark had in mind a state like China, Iran, etc.

On the contrary; those in charge have rewired my head so that I'm unable to form any such thoughts about specific state actors, companies, implementers or other parties that may take slight.

It's somewhat liberating, but makes reading the Saturday paper a bit problematic.

> Censorship happens sometimes in other states through decision of laws and/or regulations. For example, a network in a corporate environment blocking certain sites through proxy (such as social networks). A library blocking some sites through proxy to other users. In these cases, the organization in charge of it might want to advertise it for reasons which seem perfectly legible to them.

Yes. However, as discussed, current status codes can be used for this, and the HTML will explain what's going on. The one remaining motivation that I can see would be a similar situation that got us 511; non-browser devices that get confused by what's going on. However, since this isn't a redirect, but just a refusal, it's less of an issue, practically.

Cheers,

---

From: "Yi, EungJun" <[email-masked]>
Date: Mon, 11 Jun 2012 21:18:59 +0900

> Note "server", not "origin server" -- which in both 2616 and bis means *any* server in the chain can send it.

+1.

And 403 is also helpful for users because it allows to describe in the
entity why the request is refused.

In 10.4.4. 403 Forbidden, RFC 2616 says,
"If the request method was not HEAD and the server wishes to make
public why the request has not been fulfilled, it SHOULD describe the
reason for the refusal in the entity".

---

From: James M Snell <[email-masked]>
Date: Mon, 11 Jun 2012 08:27:04 -0700

On Mon, Jun 11, 2012 at 4:09 AM, Mark Nottingham <[email-masked]> wrote:
>[snip]

True, however, with 511, there is a distinct practical action that the
user-agent can take in response to the specific code... namely,
authenticating with the network prior to retrying the request. There
is no such clear response action with this.

Further, one additional consideration is the case where a particular
request is blocked through similar policy-driven action. A 4xx
response would be perfectly reasonable in such cases. My initial
inclination was to use a new 5xx code but the more I go through the
cases, the more I think 403 is just fine.

- James

---

From: Tim Bray <[email-masked]>
Date: Mon, 11 Jun 2012 08:39:44 -0700

Well, the practical argument is programmer behavior, in the way that client
devs handle the codes.  I regularly see code along the lines of:

else if (status == 401)
 // do authent stuff
else if (status == 403)
 // apologize and explain
else if (status/100 == 5)
 // apologize for server misbehavior without many details (which would
probably not be helpful to end-user anyhow)

So we want a result like 403, not 5XX.  So 403 would not be flatly
incorrect, but I still think a new 4XX would be desirable. Would it be
used?  Well, in most web frameworks, you can already provide any old
integer status code, so I don’t think any infrastructure development would
be required.  I suspect the browser people would be happy to provide a
boilerplate message if the status code became official.

And finally, I am sure that some websites under government pressure would
be happy to lie and deny censorship exists, but I am quite certain that
lots of others would welcome the chance to make clear the responsibility
for the blockage. I would welcome being so informed in those situations.

So where’s the downside?

-T

On Mon, Jun 11, 2012 at 8:27 AM, James M Snell <[email-masked]> wrote:
>>> Censorship happens sometimes in other states through decision of laws
and/or regulations. For example, a network in a corporate environment
blocking certain sites through proxy (such as social networks). A library
blocking some sites through proxy to other users. In these cases, the
organization in charge of it might want to advertise it for reasons which
seem perfectly legible to them.
>>
>> Yes. However, as discussed, current status codes can be used for this,
and the HTML will explain what's going on. The one remaining motivation
that I can see would be a similar situation that got us 511; non-browser
devices that get confused by what's going on. However, since this isn't a
redirect, but just a refusal, it's less of an issue, practically.

---

From: James M Snell <[email-masked]>
Date: Mon, 11 Jun 2012 08:54:24 -0700

I can definitely live with that.. anything that increases the
visibility of censorship is not a bad thing.  Looks like status code
427 is open currently.

- James

On Mon, Jun 11, 2012 at 8:39 AM, Tim Bray <[email-masked]> wrote:
> incorrect, but I still think a new 4XX would be desirable. Would it be used?
>  Well, in most web frameworks, you can already provide any old integer
> status code, so I don’t think any infrastructure development would be
> required.  I suspect the browser people would be happy to provide a
> And finally, I am sure that some websites under government pressure would be
> happy to lie and deny censorship exists, but I am quite certain that lots of
> others would welcome the chance to make clear the responsibility for the
> blockage. I would welcome being so informed in those situations.
>
> So where’s the downside?
>
> -T
>
>
>
>>> and the HTML will explain what's going on. The one remaining motivation that
>>> I can see would be a similar situation that got us 511; non-browser devices
>>> that get confused by what's going on. However, since this isn't a redirect,
>>> but just a refusal, it's less of an issue, practically.
>>>
>>

---

From: Julian Reschke <[email-masked]>
Date: Mon, 11 Jun 2012 18:03:15 +0200

On 2012-06-11 17:54, James M Snell wrote:
> ...

So is 418. In any case, if we go there it should be 451.

Best regards, Julian

PS: and I do agree with Mark that it's unlikely that it'll be tricky to 
get something standardized that might give the impression that 
censorship is ok.

---

From: "Musatov, Martin - CW" <[email-masked]>
Date: Mon, 11 Jun 2012 16:10:04 +0000

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]] 
Sent: Monday, June 11, 2012 11:03 AM
To: James M Snell
Cc: Tim Bray; Mark Nottingham; Karl Dubost; [email-masked]
Subject: Re: Status code for censorship?

On 2012-06-11 17:54, James M Snell wrote:
> visibility of censorship is not a bad thing.  

Yes, but what incentives are there for censors to comply with its use?
Martin

Looks like status code

Best regards, Julian

PS: and I do agree with Mark that it's unlikely that it'll be tricky to get something standardized that might give the impression that censorship is ok.

---

From: Tim Bray <[email-masked]>
Date: Mon, 11 Jun 2012 09:12:29 -0700

None. But when some ISP or search engine gets a court order saying “You are
forbidden to link to {The Pirate Bay|Anonymous|whatever}, they have a
strong incentive to be transparent about it. Now in some countries, the
censors will also pass a law saying it’s forbidden to disclose the
censorship; but thankfully, not all. -T

On Mon, Jun 11, 2012 at 9:10 AM, Musatov, Martin - CW <
[email-masked]> wrote:

> get something standardized that might give the impression that censorship
> is ok.
>
>
>

---

From: David Morris <[email-masked]>
Date: Mon, 11 Jun 2012 09:19:48 -0700 (PDT)

FWIW .. a couple years ago I had a customer support issue related to
company censorship. In that case, the country enforcement was to
to provide a replacement page which essentially said 'forbidden' and
return it with 200 status and a long expiration. Essentially poluting
the end user's local caching and reducing the load on their interception
process.

On Mon, 11 Jun 2012, Tim Bray wrote:

> None. But when some ISP or search engine gets a court order saying ?You are
> censors will also pass a law saying it?s forbidden to disclose the

---

From: "Martin Nilsson" <[email-masked]>
Date: Mon, 11 Jun 2012 18:30:45 +0200

> So is 418. In any case, if we go there it should be 451.

450 is used by Microsoft for parental control blocking, so it makes sense  
for more than one reason.

/Martin Nilsson

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

---

From: Julian Reschke <[email-masked]>
Date: Mon, 11 Jun 2012 18:45:28 +0200

On 2012-06-11 18:30, Martin Nilsson wrote:
> On Mon, 11 Jun 2012 18:03:15 +0200, Julian Reschke
> <[email-masked]> wrote:
>
> 450 is used by Microsoft for parental control blocking, so it makes
> sense for more than one reason.

Good to know.

Friends @MSFT: how about writing a spec?

Best regards, Julian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 11 Jun 2012 18:57:37 +0200

On Jun 11, 2012, at 18:03, Julian Reschke wrote:

> if we go there it should be 451

Obvious to anyone skilled in the art.
(But please never document, or write to the list, why :-)

Grüße, Carsten

---

From: John Sullivan <[email-masked]>
Date: Mon, 11 Jun 2012 18:23:41 +0100

Mark Nottingham wrote:
> 4xx status codes indicate that the problem is at the client; 5xx
> indicates that the error is at the server.

While that appears to be true, I think this is a case where
ambiguity of the English language is colouring the technical side
of things. (I saw the /. discussion and saw many people making the
same argument for a 5xx code, and that's before you get to the ones
arguing for [6-9]xx codes or 4-digit codes(?!) or worse...)

Take the case of an origin server returning a 403 error: the
client has very likely made a request that is indistinguishable
from a valid request, yet the server alone has decided not to
satisfy it. I don't think this means that 403 should in fact have
been specified as a 5xx error!

To me the crucial difference between the two classes is what
resolution action a user-agent can take. For a 4xx code, the
client must change some aspect of the request if it is to expect
to proceed past the error. Some 4xx codes are suggestive of
specific resolutions (such as 401, or even 406); ones where the
client has no specific intelligence are not going to be
automatically resolvable but the basic premise stands.

For 5xx codes the client can have a reasonable expectation that
at some future time (which might be days or weeks if a collapsed
server needs rebooting!) the exact same request might not receive
an error. (Err. Without changing the underlying resource. Of
course "normal" 403s can be resolved with chmod, and 404 by
uploading the necessary file. That's different.)

(I think there's a stronger argument that 501 and 505 should have
been made 4xx errors, but it's Too Late Now.)

I don't think the above is new, I think it's generally understood
in practice, but I've seen this misconception often enough that
it might be worth saying so in the two heading sections in httpbis.

> [...]
> However, it'd need to get deployed. Since the primary objective
> of such a status code seems to be to raise the visibility of
> censorship, thereby (presumably) causing some reaction against it,
> it seems unlikely that it'll happen in places where there's strong
> oversight of whoever is deploying it.

It doesn't *have* to be deployed. It doesn't even *have* to be
standardised, really. Today, any unknown 4xx code is going to
invoke pretty much the same (and probably correct) client behaviour,
and as you noted, 403 is perfectly sensible here.

Using a non-403 code is a choice made by intermediaries that wish
to be transparent about what they are doing. And in that case,
reserving a code *might* be useful to avoid clashing with any future
standard code that suggests specific resolution actions that are
clearly not appropriate in this instance. People are already using
various different custom non-standardised codes for this and related
things though.

John
--

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 12 Jun 2012 06:11:29 +1000

Sure, but as discussed, how does some appropriate HTML not fit the job?

On 12/06/2012, at 2:12 AM, Tim Bray wrote:

> None. But when some ISP or search engine gets a court order saying “You are forbidden to link to {The Pirate Bay|Anonymous|whatever}, they have a strong incentive to be transparent about it. Now in some countries, the censors will also pass a law saying it’s forbidden to disclose the censorship; but thankfully, not all. -T
> 
> On Mon, Jun 11, 2012 at 9:10 AM, Musatov, Martin - CW <[email-masked]> wrote:
> 
>

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 12 Jun 2012 06:10:18 +1000

> is no such clear response action with this.

That's pretty much what I was saying. Refusal doesn't have a follow-on action.

---

From: "Musatov, Martin - CW" <[email-masked]>
Date: Mon, 11 Jun 2012 20:25:21 +0000

So in the below scenario, would it be something like a visitor to some search engine, say Google, searches for a torrent, sees an indexed result, clicks the link and gets a 4xx (meaning censored link)? --MM

From: Tim Bray [mailto:[email-masked]]
Sent: Monday, June 11, 2012 11:12 AM
To: Musatov, Martin - CW
Cc: Julian Reschke; James M Snell; Mark Nottingham; Karl Dubost; [email-masked]
Subject: Re: Status code for censorship?

None. But when some ISP or search engine gets a court order saying "You are forbidden to link to {The Pirate Bay|Anonymous|whatever}, they have a strong incentive to be transparent about it. Now in some countries, the censors will also pass a law saying it's forbidden to disclose the censorship; but thankfully, not all. -T
On Mon, Jun 11, 2012 at 9:10 AM, Musatov, Martin - CW <[email-masked]<mailto:[email-masked]>> wrote:

-----Original Message-----
From: Julian Reschke [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Monday, June 11, 2012 11:03 AM
To: James M Snell
Cc: Tim Bray; Mark Nottingham; Karl Dubost; [email-masked]<mailto:[email-masked]>
Subject: Re: Status code for censorship?

Looks like status code

Best regards, Julian