Subject: Fwd: New Version Notification for draft-tbray-http-legally-restricted-status-00.txt

From: Tim Bray <[email-masked]>
Date: Mon, 11 Jun 2012 18:36:33 -0700

With apologies to MNot for ignoring his opinion (which I generally
agree with) about not picking particular
status values; but in this case I yielded to temptation.

rfc2xml output at
http://www.tbray.org/tmp/draft-tbray-http-legally-restricted-status.html

 -T

---------- Forwarded message ----------
From:  <[email-masked]>
Date: Mon, Jun 11, 2012 at 6:33 PM
Subject: New Version Notification for
draft-tbray-http-legally-restricted-status-00.txt
To: [email-masked]

A new version of I-D, draft-tbray-http-legally-restricted-status-00.txt
has been successfully submitted by Tim Bray and posted to the
IETF repository.

Filename:        draft-tbray-http-legally-restricted-status
Revision:        00
Title:           A New HTTP Status Code for Legally-restricted Resources
Creation date:   2012-06-11
WG ID:           Individual Submission
Number of pages: 5
URL:
http://www.ietf.org/internet-drafts/draft-tbray-http-legally-restricted-status-00.txt
Status:
http://datatracker.ietf.org/doc/draft-tbray-http-legally-restricted-status
Htmlized:        http://tools.ietf.org/html/submission.filename }}-00

Abstract:
  This document specifies an additional Hypertext Transfer Protocol
  (HTTP) status code for use when resource access is denied for legal
  reasons.

The IETF Secretariat

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 12 Jun 2012 11:49:52 +1000

> status values; but in this case I yielded to temptation.

Since we'll be in Vancouver soon, this may cost you beer. :)

Let's plan to have some discussion there in the WG meeting; Tim, would you be willing to present?

Regards,

--
Mark Nottingham   http://www.mnot.net/

---

From: Tim Bray <[email-masked]>
Date: Mon, 11 Jun 2012 18:57:36 -0700

Happy to. It’ll be a short presentation. -T

---

From: John Sullivan <[email-masked]>
Date: Tue, 12 Jun 2012 14:52:26 +0100

> (HTTP) status code for use when resource access is denied for
> legal reasons.

I think the description is too specific. While many of the instances
under recent discussion will be due to legal constraints, this code
may be applicable for other reasons: local policy, elective
filtering, etc.

To me, the defining aspects of the usefulness of this code are:

1) The code is generated by an intermediary outside of the control
of the origin server.

2) The content is being blocked due to the nature of the content
itself, whether that nature be assumed, suspected, reported,
spidered, analysed on attempted access or something else.

(I wonder if "3) users not downstream of the intermediary(s)
can expect to be able to access the content" is significant.)

Remember that non-legally mandated content filters are already
using various non-standardised status codes for this already:
they *might* want to switch to a standardised code should it
become available.

Consider the following cases:

1) A government agency provides technical information, but for
national security reasons does not want to allow access to IP
addresses within embargoed foreign states. It is neither an
intermediary, nor though it may appear so is the block directly
related to the nature of the content (which it would normally be
happy to serve to anyone else), but to the nature of the
requester. 403 would be more appropriate here.

2) A media company has negotiated broadcast rights for a piece
of content to certain territories, but not to others. Much as
above.

3) An internet cafe provides unrestricted normal browsing, but
blocks access to video/* responses unless "premium" access has
been paid for. Again this is not actually "content related":
they don't care whether it's kittens or anti-government
propaganda, their main motivation is financial and so 403 is
still appropriate.

4) A company implements a filtering web proxy. The CEO's
computer is allowed unfiltered access. Unlike (1),(2), it *is*
the nature of the content that causes block here, and although
the block is not evenly enforced, it is the CEO's access which
is exceptional, it is not generally the nature of the requester
that causes the block: the block is the normal case. This new
status code *is* appropriate here.

5) An ISP provides content-based filtering as a value-add feature.
It may be either opt-in or opt-out. The user is essentially
imposing the block on themselves in a sense, but the new code
is appropriate as the two defining aspects are met. Obviously
the response body should contain enough information to request
to opt out again.

6) A CDN with a direct relationship with the origin server blocks
content the origin attempts to serve due to contractual restrictions
chosen by the CDN itself on the types of content the CDN is willing
to serve. This could be argued either way perhaps, but I would
say the CDN and origin are closely enough related that 403 is
more appropriate.

> the server in question may or may not be an origin server

While I wouldn't say it is never appropriate for an origin server
to generate this code, I would expect that to be a highly unusual
case. In most cases an origin that does not want to serve the
content would simply return 403 or 404 directly. (An origin that
would normally serve the content, and wants to do so, but finds
itself under a temporary court injunction, for example, might
be one case.)

> Responses using this status code SHOULD include an explanation,
> in the response body, of the details of the legal restriction;
> which legal authority is imposing it, and what class of resources
> it applies to. For example:

I would say:

* details of the <del>legal</del> restriction and class of resources
  it applies to are likely to be very closely related, and may
  perhaps be treated as the same piece of information
* the party implementing the block
* the party imposing the block (if different)
* (the party requesting the block maybe: say the plaintiff if the
  block is due to a court injunction. The "party imposing" would
  be the court itself. Or maybe "party imposing" just has to be
  very specific about how the block came about.)
* sufficient information to contact/find contact details of the
  appropriate agent to complain about/request removal of the block

Any and all of these items are optional (as is the code itself) at
the discretion of the block implementing party based on what they
want to disclose, or what they are allowed to disclose.

> 4.  Security Considerations

For entities imposing a block locally, and inspecting status codes
from access logs to identify block hits, it should be remembered that
the given status code may also be generated externally so mere
presence of the status code does not necessarily imply an attempt
to violate local policy.

For all entities, blocks of any type have been known to generate
frequent false positives, so generation of the given status code
does not necessarily imply an attempt to access blocked content.

John
--

---

From: Tim Bray <[email-masked]>
Date: Tue, 12 Jun 2012 09:34:01 -0700

Aaaaaaaand, it turns out MNot was right; I checked with an expert, and 451
is heavily used for “redirect” in the Msft ecosystem, notably including
HotMail’s hundreds of millions of users.  Consider it “4xx” (which I would
still argue for as opposed to 5xx).  -T

---

From: Julian Reschke <[email-masked]>
Date: Tue, 12 Jun 2012 18:43:09 +0200

On 2012-06-12 18:34, Tim Bray wrote:
> Aaaaaaaand, it turns out MNot was right; I checked with an expert, and
> 451 is heavily used for “redirect” in the Msft ecosystem, notably
> including HotMail’s hundreds of millions of users.  Consider it “4xx”
> (which I would still argue for as opposed to 5xx).  -T
> ...

Use it anyway. People who mint new status codes and do not register them 
do not deserve anything else :-)

Best regards, Julian

---

From: James M Snell <[email-masked]>
Date: Tue, 12 Jun 2012 11:01:59 -0700

I'd have to agree with Julian on this... there is a clear, open
registration process in place for http status codes, a single vendor
should not be allowed to hijack status codes from the shared (and
quite limited) available range. If their stuff gets broken because
someone follows the legitimate process later, then the fault is on
them. They should have known better.

On Tue, Jun 12, 2012 at 9:43 AM, Julian Reschke <[email-masked]> wrote:
> Use it anyway. People who mint new status codes and do not register them do
> not deserve anything else :-)
>
> Best regards, Julian
>

---

From: Tim Bray <[email-masked]>
Date: Tue, 12 Jun 2012 20:28:57 -0700

OK, Lauren Weinstein also agrees with you:
https://plus.google.com/u/0/114753028665775786510/posts/RKMHCtYpaeV

So I went back and pushed back on my mail expert (guy who does
mail/Exchange dev on Android), saying “problem in practice or just in
theory?” and he wrote

“I agree that in practice it wouldn't likely be an issue.  I guess I'm
using an abundance of caution criterion.”

So maybe 451 is still plausible.  -T

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 13 Jun 2012 13:40:41 +1000

I wouldn't worry about it too much; AFAICT, the Exchange use case is one where the client and server are tightly coupled anyway, so the worst thing that's going to happen is that an Exchange client is going to interpret some censorship (again, *if* it gets deployed) as a different problem.

Cheers,

On 13/06/2012, at 1:28 PM, Tim Bray wrote:

> OK, Lauren Weinstein also agrees with you:  https://plus.google.com/u/0/114753028665775786510/posts/RKMHCtYpaeV
> 
> So I went back and pushed back on my mail expert (guy who does mail/Exchange dev on Android), saying “problem in practice or just in theory?” and he wrote 
> 
> “I agree that in practice it wouldn't likely be an issue.  I guess I'm using an abundance of caution criterion.”
>

---

From: "Roy T. Fielding" <[email-masked]>
Date: Tue, 12 Jun 2012 22:35:12 -0700

On Jun 12, 2012, at 9:34 AM, Tim Bray wrote:

> Aaaaaaaand, it turns out MNot was right; I checked with an expert, and 451 is heavily used for “redirect” in the Msft ecosystem, notably including HotMail’s hundreds of millions of users.  Consider it “4xx” (which I would still argue for as opposed to 5xx).  -T

4xx indicates an error by the user or user agent.  I don't see
any reason (aside from literary) that would justify using a 4xx
code for this.  5xx is typically used for non-authoritative
responses or server-imposed limitations -- a status that might
be different if the user agent chose a different intermediary
or tried again later.  Hence, 5xx makes more sense here.

....Roy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 13 Jun 2012 07:51:24 +0200

Hi Roy,

On Tue, Jun 12, 2012 at 10:35:12PM -0700, Roy T. Fielding wrote:
> > Aaaaaaaand, it turns out MNot was right; I checked with an expert, and 451 is heavily used for ?redirect? in the Msft ecosystem, notably including HotMail?s hundreds of millions of users.  Consider it ?4xx? (which I would still argue for as opposed to 5xx).  -T
> 

I don't completely agree here : for me, 5xx means that the error is not
the client's fault and that it might randomly work if the client tries
again, which is why network errors fall into this category, as opposed
to the 4xx error by the user/user agent as you explained (and which I
agree with). If a client requests a resource that is forbidden for legal
reasons, we're typically in the situation where the user caused the error
to happen by requesting this resource, and where if he tries again he will
get the same error again. Much like 403 or 404. 5xx would be appropriate
if the server was not able to verify in a database or referential whether
the resource is legally permitted or not.

Best regards,
Willy

---

From: J Ross Nicoll <[email-masked]>
Date: Wed, 13 Jun 2012 08:10:08 +0100

Should this only be for legal restrictions? Do we want to perhaps expand 
the scope to cover, for example, parents filtering a home Internet 
connection, or an employer filtering a workplace connection?

---

From: Nicolas Mailhot <[email-masked]>
Date: Wed, 13 Jun 2012 09:44:38 +0000 (UTC)

Willy Tarreau <w <at> 1wt.eu> writes:

> agree with).

I'd say there is not a sliver of technical difference between the blocking done
by an hotel wifi portal, by a school gateway on lab internet accesses, by a home
proxy on pornography accesses by children gadgets, by a corporate proxy on
facebook accesses, by a conference gateway for visitors, by an isp or country
filtering infra on stuff the country decided is not legitimate (and in all those
cases the equipments used are pretty much the same and marketed by the same
companies).

At every point the web client requested a web resource and it was denied by some
intermediary for violating the terms of use.

What varies is only the ultimate human justification behind the blocking
(financial, organisational, legal…): what the terms of use are based on, how to
get the level of access to bypass it (I'm pretty sure that even in
politically-challenged countries spooks get to see what is blocked), etc But
this justification is not something software should act on, it's something for
humans to act on (and I'm not saying I agree with all the possible kinds of
blocking, just that technically they are all the same).

It is completely wrong to try to enshrine political assessments in technical
protocols, the protocols must provide the technical means to communicate there
is some form of blocking, where the (human) justification can be found
(law/terms of uses), and where the user can provide credentials to lift the
blocking in the case it has this possibility (prove he paid the bill in case of
hotel accesses, prove he is staff for school labs/conferences, prove he is
marketing for facebook access in the Enterprise, that he is of legal age to see
nudity etc)

So the semantics should be the same as error 511 and I'd go as far as to say
error 511 should be generalised to handle those cases.

What error 511 is missing is:

1. a clear definition of 'terms of use' (what humans needs to read to understand
why access is blocked) and 'authentication means' (where the user or software
can provide credentials to get elevated access). This is especially important in
russian doll scenarii where the blocking can occur at different levels and the
different levels are not the same at all for users (a conference in a repressive
country for example: blocking can occur at the conference gateway level, at the
isp used by the conference, or at the national firewall level)

2. how this information is conveyed to users (a free-form html page does not
work for dumb clients or fat clients that use an http library such as curl).
'terms of use' and 'authentication means' location need to be specified in http
headers so the http library can pass it up the stack and the developer can
choose to display it (or not) in a browser window, an app popup or whatever he
deems appropriate

3. guidance to web client implementers on the security precautions to apply when
conveying this info to the user, and how to convey it even when the blocking
occurs mid-browsing in an https session

And it is not helping the user to refuse to acknowledge reality and hide this
information (when it is available). If the user does not even know it is blocked
and why he can not act at all (politically or otherwise)

Regards

-- 
Nicolas Mailhot

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 13 Jun 2012 10:59:23 +0100

> connection, or an employer filtering a workplace connection?

Hmmm. My guess would be that the the broader the scope, the
more likely this is to be problematic. And I suspect that'd
not be a linear relationship, but more like an exponential
increase in difficulty getting done with each small increase
in scope. Just my guess though.

S

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 13 Jun 2012 20:21:16 +1000

> to the 4xx error by the user/user agent as you explained (and which I

This is completely off base.

If the client retries the request, it might indeed work again -- depending on what network path they're using, etc. That's why all of the intermediation-focused errors are in 5xx.

> If a client requests a resource that is forbidden for legal

Sorry, what?

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 13 Jun 2012 20:25:03 +1000

And putting the word "legal" in a protocol element isn't problematic??

I suggested "5xx Network Policy Violation" earlier; my .02 is that's more straightforward, more neutral, more useful, and more likely to actually be used.

Cheers,

---

From: John Sullivan <[email-masked]>
Date: Wed, 13 Jun 2012 14:05:33 +0100

Mark Nottingham wrote:
> If the client retries the request, it might indeed work again --
> depending on what network path they're using, etc. That's why all
> of the intermediation-focused errors are in 5xx.

If the same user agent, making the same request, from the same location,
over the same access link, can randomly switch between a path implementing
the block to one not from request to request, then I think all that shows
is that the block is *very* poorly implemented.

In practice the block will be implemented evenly for a given end point,
and the client will never succeed in that (exact) request no matter how
hard they try. Their only hope of success is to get the status of that
resource changed, which makes it a 4xx error.

>> the resource is legally permitted or not.

You have made the case that, following specific language from the standard,
"4xx status codes indicate that the problem is at the client; 5xx indicates
that the error is at the server."

Taking that at face value, with no wider context, why is 404 a 4xx error?
The client made a request that to any outside observer looks entirely valid,
and usually apart from the precise URL requested is indistinguishable from
another request to the same server which returned a 200 response. The client
could even be following a link within a document it previously retrieved from
that same server. (With appropriate response headers the 404 might even be
cached by an intermediary past the point at which the origin server has been
"fixed" to return 200 responses for the same request.) The resulting 4xx error
could hardly be said to be the client's "fault".

I'm not trying to be facetious here, nor do I think I'm being dense. I really
see this interpretation as an (apparently quite common) mistake based on taking
a very narrow definition of the terms "client error" and "server error" which
is not quite justified by the heading paragraphs (p3 S7.4 and S7.5 in httpbis,
little changed from the 2616 versions), is contradicted by the descriptions of
specific status codes, does not match established usage, and in practical
terms fails to divide the domain of possible error types neatly(ish) into two
classes with broadly distinct:

   * causes
      4xx related to the request, or the state of a specific resource or
          resources, server does not want to satisfy request
      5xx network/server wide, server does want to satisfy request but
          "can't"
   * expected lifetimes
      4xx for a valid request, will not go away until the status of the
          underlying resource is modified
      5xx might "spontaneously" resolve, or resolution doesn't directly involve
          the specific resource
   * useful immediate response strategies
      4xx authenticate, change representation, change some other aspect of
          the request, ask for something different, give up; might make
          (semi-)persistent updates to local state
      5xx retry, or pause, retry; avoid polluting cache/history
   * useful final resolution strategies
      4xx fix/change the status of the resource(s)
      5xx forget after a successful retry, reboot server, reconfigure network

Things can get muddy round the edges, and this status code is I think muddiest
around "causes" (though I think 4xx is still justified), but under any other
metric I can't see why it isn't clearly in 4xx territory.

Conversely I think 5xx (511) for captive portals *is* justified under the
above classification. ("reconfigure network" could mean "walk down to the hotel
lobby and hand over a fiver for network access".)

(As I said previously, I'm deeply unsure about 501, and unconvinced about 505.
My skepticism here is bolstered by the choice of specific status codes to act
on in RFC 5861. *If* the underlying code usage were consistent, surely it could
have been applied based on code class alone.)

John
--

---

From: Tim Bray <[email-masked]>
Date: Wed, 13 Jun 2012 08:16:00 -0700

Yeah, if you look at RFC2616 sections 10.4 (4xx) and 10.5 (5xx), you can
make a good case that it doesn’t belong in either.   I just don’t believe
you can make a bullet-proof argument for either range.

The reasons I currently favor 4xx are:

- Lots of client software simply tries to sweep 5xx errors under the carpet
(“Something went terribly wrong upstream, don’t bother your pretty little
head”)
- I liked the value 451

-T

On Tue, Jun 12, 2012 at 10:35 PM, Roy T. Fielding <[email-masked]> wrote:

> 451 is heavily used for “redirect” in the Msft ecosystem, notably including

---

From: Tim Bray <[email-masked]>
Date: Wed, 13 Jun 2012 08:20:12 -0700

On Wed, Jun 13, 2012 at 3:21 AM, Mark Nottingham <[email-masked]> wrote:

If the client retries the request, it might indeed work again -- depending
> on what network path they're using, etc. That's why all of the
> intermediation-focused errors are in 5xx.
>

Hm?  Surely the expectation would be the opposite; legislative/legal
changes typically have timeframes measured in years or at the quickest
months.   -T

---

From: Ross Nicoll <[email-masked]>
Date: Wed, 13 Jun 2012 16:20:26 +0100

In an ideal world, I'd start a new 600-level status series, for "3rd 
party", but strongly suspect that's infeasible due to impact on existing 
implementations. A 400-level code (451 or otherwise) is probably the 
most practical way of doing this.

Ross

On 13/06/2012 16:16, Tim Bray wrote:
> Yeah, if you look at RFC2616 sections 10.4 (4xx) and 10.5 (5xx), you 
> can make a good case that it doesn’t belong in either.   I just don’t 
> believe you can make a bullet-proof argument for either range.
>
> - Lots of client software simply tries to sweep 5xx errors under the 
> carpet (“Something went terribly wrong upstream, don’t bother your 
> pretty little head”)
> - I liked the value 451
>
> -T
>
> On Tue, Jun 12, 2012 at 10:35 PM, Roy T. Fielding <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     > Aaaaaaaand, it turns out MNot was right; I checked with an
>     expert, and 451 is heavily used for “redirect” in the Msft
>     ecosystem, notably including HotMail’s hundreds of millions of
>     users.  Consider it “4xx” (which I would still argue for as
>     opposed to 5xx).  -T
>

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 13 Jun 2012 18:33:30 +0200

> changes typically have timeframes measured in years or at the quickest

I agree Tim. In my opinion it's just like a "403 forbidden" with a specific
reason for this being administratively forbidden.

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 14 Jun 2012 07:59:40 +1000

On 14/06/2012, at 1:20 AM, Tim Bray wrote:

> Hm?  Surely the expectation would be the opposite; legislative/legal changes typically have timeframes measured in years or at the quickest months.   -T

VPNs. Proxies. Airplanes. Tor.

Living in the non-US* makes it blindingly obvious that geolocation schemes, network border controls, etc. are extremely porous; the Internet works around them.

Cheers,

* Canada doesn't count; /me ducks

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 14 Jun 2012 11:23:47 +1200

On 14.06.2012 04:33, Willy Tarreau wrote:
>> On Wed, Jun 13, 2012 at 3:21 AM, Mark Nottingham wrote:
>>
>> depending

Would these types of differentiation between reasons for rejection be a 
good case for Warning: codes on a 403 response?

ie
  Warning: ... Legal Restriction
  Warning: ... Local administrative policy
  Warning: ... Authentication failed too many times. Your account is now 
closed
  ...

The body of 403 can as easily contain the legal disclaimer text as any 
other 4xx code.

AYJ

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 14 Jun 2012 10:25:58 +1000

On 14/06/2012, at 9:23 AM, Amos Jeffries wrote:

> Would these types of differentiation between reasons for rejection be a good case for Warning: codes on a 403 response?
> 
> ie
> Warning: ... Authentication failed too many times. Your account is now closed
> ...
> 
> The body of 403 can as easily contain the legal disclaimer text as any other 4xx code.

So, again -- what's the use case for a machine consuming these? I haven't seen one yet, unless I've missed something.

Cheers,

---

From: Bjoern Hoehrmann <[email-masked]>
Date: Thu, 14 Jun 2012 05:02:38 +0200

>or tried again later.  Hence, 5xx makes more sense here.

In the interest of possibly improving the status code definitions
and avoiding future debates like this, could you say how 403 For-
bidden fits into this? It's a server-imposed limitation and not an
error by the user or user agent. Link suffices if I missed useful
arguments or admissions on the matter.
-- 
Björn Höhrmann · mailto:[email-masked] · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 14 Jun 2012 07:44:07 +0200

Hi Mark,

On Thu, Jun 14, 2012 at 10:25:58AM +1000, Mark Nottingham wrote:
> So, again -- what's the use case for a machine consuming these? I haven't
> seen one yet, unless I've missed something.

The only one I can think of is logging/accounting. Reporting to the user
"Accessing this content is illegal, this access has been logged" is for
the body, but having the user-facing proxy being able to log to a separate
file when such an event happens can probably make sense.

It reminds me an event which happened to a customer around 10 years ago,
which was a public administration. Some people noticed that one of their
offices was easy to open from the outside. And we discovered in their
proxy's logs that during the week-end, some people were coming there to
download and watch x-movies, probably because the internet access was
well sized for this, despite some filtering on the way. With automated
logging into a separate file, the local admin could have noticed the
event much earlier, because surely a number of attempts had failed.

I'm not saying this is the solution to improper log analysis, but it
shows one use case of machine-readable code.

BTW I like Amos' proposal much more than just a separate code. I was
wondering how many new codes we'd have if we created a new code for
this, and having a large set of possible warnings with the usual 403
seems a lot better to me.

Cheers,
Willy

---

From: Nicolas Mailhot <[email-masked]>
Date: Thu, 14 Jun 2012 09:04:23 +0000 (UTC)

> > Would these types of differentiation between reasons for rejection be a good
case for Warning: codes on a
> 403 response?

1. the block cause must be reflected in headers to be available in (legal or
debugging) logs. With today's spaghetti web sites that shard their content over
multiple servers and delegate more and more bits to nebulous cloud platforms
with fuzzy limits having to read all the html pages exchanged to debug a block
is increasingly un-practical

2. the block cause must be reflected in headers so all the web clients that use
http as transport via some other middleware can be notified of the block cause
(ie the not-html-capable http middleware needs something it can easily parse and
pass up the stack so the web client presentation layer gets a chance to do
something with it and inform the user)

3. the info must absolutely be relayed to the human user so he gets a chance to
do something about it.

4. The info must be complete enough the human user can act on it. Going to the
hotel lobby to buy some more creds is not the same thing as being invited to the
local police station to explain oneself

5. There must be a way to inform the user ways to lift the block when the
possibility exists instead of having him hang in the dark (hotel payment gateway
'credit-exhausted pay-here-to-continue', corporate authentication portal
'are-you-an-employee fill-in-your-corp-login-there',
'content-was-blocked-because-we-believe-it-is-objectionnable,
here-is-the-html-form-to-contest-the-evaluation-if-you-believe-it-is-wrong',
etc, etc)

None of those work of you rely on an html soup error page that may or may not be
parsed by the web client, or that you let the web client ignore at will even
when it is capable of parsing (current browsers refuse to display such pages
when a block occurs over https)

-- 
Nicolas Mailhot

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 14 Jun 2012 20:14:25 +1000

> is increasingly un-practical

The device doing the blocking can log its own actions without making it machine-readable on the wire. 

> something with it and inform the user)

Yes; this is the most compelling case I can see for a separate status code, similar to the one we had for 511. However, I don't yet see where the client will actually *do* something with the response that might require making this distinction. It was necessary in 511 because sites were interpreting the redirect as coming from the origin server, which has all sorts of nasty side effects for non-browser agents; what are the corresponding side effects here?

> 3. the info must absolutely be relayed to the human user so he gets a chance to

That really sounds like HTML; most browsers don't expose the raw status code.

> local police station to explain oneself

Same as #3.

> here-is-the-html-form-to-contest-the-evaluation-if-you-believe-it-is-wrong',

How is that accomplished by a status code, and NOT by HTML?

> when a block occurs over https)

None of these? Really?

Understand -- I'm not against minting a status code here per se, but if we do it, I want it to be for technically valid reasons, not just to scratch an itch to make developers feel more politically active, with no real-world deployment.

Cheers,

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Thu, 14 Jun 2012 14:45:47 +0200

> machine-readable on the wire.

That's pretty much useless for the web app/web client developer the users will
complain at since he won't have access to the device logs and often won't even
know where to ask for them.

Also, as soon as you have a multi-tiered network architecture, or one
organisation making some web resources available to another through two
separately-managed networks, that's the road to log correlation and
multi-organisation coordination hell.

> Yes; this is the most compelling case I can see for a separate status code,
> similar to the one we had for 511. However, I don't yet see where the client
> will actually *do* something with the response that might require making this
> distinction. It was necessary in 511 because sites were interpreting the
> redirect as coming from the origin server, which has all sorts of nasty side
> effects for non-browser agents; what are the corresponding side effects here?

To be honest I don't think the problem-space is any different from error 511.

In my opinion the current discussion just shows that error 511 didn't go far
enough to handle gracefully all real-world blocking scenarii and assumed
(wrongly) that a blocking error with human content that would not necessarily
be displayed to the user was sufficient, because no one cared why the blocking
occurred. When in fact the only reason Tim Bray launched this discussion was
because he wanted to convey the blocking rationale.

After spending quite a lot of time thinking about the problems error 511 does
not solve in our context (which admittedly is a lot more complex than the home
office case), and getting varied feedback from the developers of some of the
web clients I asked implementing error 511, and having to solve a few more
problems caused by current web client inability to interact gracefully with
blocking gateways, I'd really love to see a design such as the following:

%<-----

1. a generic 'blocked by intermediary' error code web clients are forbidden to
retry on (same rationale as error 511)

2. a header containing a free-form short sentence describing the blocking
rationale ('Blocked due to law 666', 'Upgrade your access', 'Too young for
this', whatever). If you want to get fancy this header should be localizable
by suffixing/prefixing it with an iso 639 code (we don't need this but I'm
pretty sure it will be a requirement in countries with multiple official
languages where it is forbidden to discriminate between them, or on
transnational networks; that makes it a candidate for UTF-8 encoding by
default BTW)

The only purpose of this header is to give dumb web clients something to
display or log in case of blocking, and to leave no room for javascript or
complex html those clients can not understand (though for logging you'll also
want the two next headers)

I don't think there is any point in trying to define a set of standard values
for this header as humans are very good at inventing new
political/social/organizational/religious reasons to block some accesses.

3. an optional header containing the location of the full explanation of the
blockage (can be a pdf with terms of service, a web site with lots of fancy
chrome, a video statement of the local leader, some cartoons to explain kids
why they don't have access to adult content, the local three-strikes law
excerpt, an helpdesk wiki page, whatever)

IMHO it is a huge mistake to try to put the pretty explanation in the error
page itself as networking equipments are not all able to host the rich content
needed to explain a situation to many users, and current computers access
stuff right and left so any user on a network subject to blocking is likely to
receive the error multiple times in a row. Better to offload it somewhere the
user will only consult once or twice in a full browser, and that is able to
withstand the load if the blocking is too extensive. The gateway always has
the option to host this page itself and transmit its own IP as location

4. an optional header containing the location of a web portal the user can use
to elevate his access (when it is possible). Can be a full auth portal, a
payment portal, a form where a visitor can enter the token printed on the card
given to him by the hotel personnel, a web page with a button stating 'I agree
that pirating hollywood films is the ultimate sin', whatever

Some organisations will set a cookie (meaning only the web client used to
access the elevating access portal will work), others will authorize all
accesses from the ip that was used to elevate the access (for a time), there
are many possible variations and this scheme is not restrictive at all.

This way any web client can display at least 1. without requiring an html
engine, it is possible to provide users a more complex rationale
representation, and the web client is informed on how to elevate its access
when it is possible.

And it also works fine when you stack multiple levels of blocking gateways
that obey to different rationales and have different ways to trigger
un-blocking

Finally, that permits the writing of password managers and other browser
chrome extensions that auto-supply credentials when they recognize a trusted
portal location in 4. (and 4. also permits the use of https auth portals the
password manager can check the certificate of before injecting credentials).

5. To be complete you also need to define a status code that the portal in 4.
will emit when it is satisfied by the token presented by the user, so browsers
can implement the obvious logic of:
“after receiving a blocking code on any access, if the portal location was
loaded and returned the ok code, retry the access that was previously
blocked̆”

%<-----

The main side effect is the same as for error 511: web client writers will
have to acknowledge intermediaries exist, and not posit that errors are
necessarily emitted from the endpoint. And they'll have to sandbox the error
appropriately so there is no leakage between the endpoint dialog and the
intermediary dialog, especially if it occurs during https browsing.

Like with error 511 that's not a reality web client writers will like and like
with error 511 the intermediary always had the option of blocking the traffic
with no explanation anyway. So there's no point in making it hard for it to
supply information that is likely to help the user.

> That really sounds like HTML; most browsers don't expose the raw status code.

That's why I suggest dual reporting: short rationale in http header, full
rational at a location specified in another header.

>> local police station to explain oneself

ditto

> How is that accomplished by a status code, and NOT by HTML?

It is possible if you complete a generic status code with the kinds of headers
I described before.

>> when a block occurs over https)

None well enough for users to be happy and helpdesk be let alone. The current
situation is painful for everyone, there are too many ways blocking can occur
without the user been given the means to understand the blocking, nor support
given the means to diagnose easily what's wrong.

In fact, it is so painful right now that if being painful was sufficient to
stop humans from implementing blocking, no one would do it today.

Thank you for your attention!

-- 
Nicolas Mailhot

---

From: Julian Reschke <[email-masked]>
Date: Thu, 14 Jun 2012 15:02:16 +0200

> loaded and returned the ok code, retry the access that was previously

This really, really smells like over-engineering to me. Also, HTTP 
header fields are a very bad solution to transporting human-readable 
text; that's what we have markup languages for.

It probably would be useful if you could walk us through a concrete use 
case where a status code + response body is not sufficient. (And if you 
did before, sorry, then please provide a pointer).

Best regards, Julian

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Thu, 14 Jun 2012 17:24:50 +0200

> did before, sorry, then please provide a pointer).

Some use cases and problems (not exhaustive):

A. employee is visiting a foreign location. For safety reasons his laptop is
configured to only access the internet through corporate proxies that will
perform a set of anti-malware checks. To reach those proxies he needs to
launch a vpn on log in. The hotel has a captive portal.

Any web access will only succeed if:
1. the captive portal lets it pass
2. the vpn is set up
3. the corporate proxy okays it

So in this very simple case you have three levels of gateways to pass, one web
client (the vpn client) that is unable to process any complex html code, and a
non-technical user very far from any help.

For this setup to work the hotel captive portal needs to send the vpn client
something simple enough the vpn client recognizes it's a gateway portal
request, and lets the browser connect to it directly instead of passing
through the (not yet available) proxy

The vpn client can not read html.

If browsing is blocked at any point later the user needs to know if it's
because he needs to pay more at the hotel or because his access is blocked by
corporate policy. Hopefully in a non-spoofable way (so need to point to an
auth/explanation location secured by its own known certificate, not a naked
415 page)

B. A company has redundant filtering internet gateways. They do not share
authentication realms (because that is hard to do without making the gateways
dependant on one another). Web clients know both gateways through a .pac file
and decide to switch from one to another for their own reasons (typically,
"web site is slow to answer, let's try the other gateway in case it works
better"). When user auth expires the user need to be notified by the browser
even if he's mid https session, the notification must point to the correct
auth portal (the user has no idea which gateway his browser decided to try at
this moment, and besides its all technical mumbo-jumbo for him), and it must
be explicit enough one can hope to plug a password/certificate manager over it
someday

C. A company authorises direct access to an internal web app by another
company. If blocking occurs at any point the users of this other company must
be informed if the blocking is done by first company network, second company
network or if the app itself has a bug (that needs to be reported to its
application maintenance team)

D. Any time blocking occurs when a user POSTs some data, if the browser is not
aware that the user or some password manager resolved the blockage by
successfully supplying credentials to the location specified in the blocking
error, and if the browser does not then auto-retry the access that failed
before, the post content will be lost (because teaching non-technical users to
refresh the correct page at the correct time manually is a lost cause)

Browser can not know a specific location unlocks the gateway if it is not
specified at the http level and if there is no standard code that conveys
successful unlocking

(you could hide this complexity via various URL redirect tricks, but browsers
block them on https for security reasons nowadays so now the browser itself
must initiate the reloading of the blocked post when it is possible to do so
again)

E. Access is blocked. Users do not understand simple explanations and need
lots of pictures and text. Those explanations are provided in the error page
itself instead of a separate location. Frustrated users do not read the
explanations and just retry in a loop. Gateway melts under the load. No access
is possible for anyone anymore (even accesses for users that were not blocked
previously)

D. Same scenario but this time the network equipment can only provide a single
page with no external dependencies at all. Loads of not-fun with embedded
data: references to try to push the info anyway

E. Any fat client situation (eclipse, git, wget scripts, various rcp or dotnet
clients developed using traditional client/server mindset and that use http
transport because of-the-shelf libraries are available and this way they don't
have to ask to open new ports on the corporate firewalls). None of those will
be able to render a complex html page. All of those can easily be upgraded to
display a short error text on a specific error code and an url the user can
then launch in his browser

D. Any loging/debuging scenario where web pages bodies will be typically
ignored (to limit logging cost) but http envelope won't

E. Any scenario where you're worried about spoofing and would like your users
to check the reason given in the short error before opening a complete
(possiby hostile) site (just like you'd want them to check mail titles before
acting on their content)

F. If you don't add some free-form text to the error code you'll have to
invent new error codes for each blocking situation instead of having a single
one everyone can implement support for

You could perhaps get by with only the two location headers (one for the fine
manual no one will read but everyone will complain at if it's missing, the
other for the un-block location that will be heavily used) but it's a lot
easier to have a short text that says 'blocked due to rule 22' or 'blocked due
to rule 5' and in both cases point to the same terms of use web page/pdf, than
to try to have separate stable entry points for all the possible block reasons
as soon as you have non-trivial terms of use.

And fat client presentation layer will really want to have some short text to
print before naked urls. If you don't give it to them they'll invent something
(that will likely not represent accurately all blocking scenarii)

-- 
Nicolas Mailhot

---

From: Albert Lunde <[email-masked]>
Date: Thu, 14 Jun 2012 12:11:38 -0500

Rather than having details of the policy/legal etc. restrictions in 
additional headers, would there be any merit in having a structured XML 
mime multi-part, containing information in a parseable form?

I suppose if one put text or HTML body parts first, a simple-minded 
client would display that, but an advanced client could get more details.

---

From: Tim Bray <[email-masked]>
Date: Thu, 14 Jun 2012 17:36:42 -0700

You folks who want an enriched semantic payload describing the
finer-grained aspects of different modes & causes of policy-driven blockage
need to come up with some detailed proposals, maybe a controlled vocabulary
too.

Also, it’s clear that this is entirely separate issue from the original
proposal, for a single HTTP status code saying “legal restrictions” with a
SHOULD saying “please explain in the response body”.  Neither need depend
on the other.  -T

On Thu, Jun 14, 2012 at 10:11 AM, Albert Lunde <[email-masked]> wrote:

> I suppose if one put text or HTML body parts first, a simple-minded client
> would display that, but an advanced client could get more details.
>
>
>
>
>

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Fri, 15 Jun 2012 09:33:59 +0200

(Apologies for replying to myself)

To put the point shorter, I think the core problem with "assign a generic
code, put human explanations in html or xml body" is that's a layering
violation for all applications that use http as transport.

Those applications (reasonably) expect being able to set up the transport at
the http level without ever having to parse metalanguage bodies. When you put
relevant transport information in metalanguage bodies, you break this
expectation.

And in case of blocking or censorship, transport set up does not stop at
punting a metalanguage error to the human. In all but the most extreme cases
it is possible to lift all or part of the blocking by providing some
credentials or tokens somewhere, and to avoid breaking web apps that were
mid-transaction when the block occurred, the web client must then retry the
access that was blocked.

There are several possible designs. The full one in proposed yesterday. The
others are (if I've followed the discussion correctly) :

A.

1. Generic 'blocked by intermediary' code
2. All other information somewhere in a metalanguage body

That's error 511 design. The cons are that it is useless for dumb clients,
there is no clear way to unlock except by parsing not-standardised
metalanguage soup, the blocking cause is not explicit at the http level in
logs (you need to infer it from what emitted this error), it's a transport
layering violation, and in cases when your network equipment can not host a
full web site you have to stuff all the rationale in the error body, even if
most users don't need to read it twice, and it needlessly overloads the wire.

B. Same thing but with separate error code for each human reason for blocking

That's basically what Tim Bray is proposing with his separate censorship code.
It has the same drawbacks as the previous proposal, except that the blocking
cause is explicit at the http level. The other side of the coin is that you'll
have to define new codes for all human blocking rationales, which will lead to
code explosion and codes which are only supported by a limited set of web
clients

C.
1. Generic 'blocked by intermediary' code
2. Optional unlock location in header
3. Standard 'ok' code unlock location returns when it is safe to retry the
access that triggered 1.

Same thing as A. but solves the unlock problem and lets the unlock location
use https portal preventing spoofing. Blocking rationale is still nowhere and
you still have logging/debuging limitations

D.
Same thing as C. but with optional blocking rationale location

Lets you inform humans why they are blocked with rich content without
overloading the error body. Simplifies logging since you can correlate
blocking rationale location with blocking causes. Requires a separate
rationale location for each blocking reason (not nice in case of complex use
terms)

E.
Same thing as D. but with short blocking rationale in a separate header

Pro: solves all functional problems I see
Cons: heavy design, puts human text in headers

F.
Make all intermediaries explicit, proxy traffic hop from hop, communicate
blocking at the proxy protocol level)

(What Willy favoured for corporate proxies, I don't think it scales up to the
national censorship level, nor that it would be a good idea to go there)

I freely admit we could probably get by with C. only even though without D.
we'd probably continue to have frictions and E. would be heaven. I don't see
the point of B. except as a political point which has no place in a technical
standard.

Am I missing something obvious?
Is there one of those designs people would like me to try to formalize more or
are there good reasons to reject them all directly without bothering?

Best regards,

-- 
Nicolas Mailhot

---

From: Julian Reschke <[email-masked]>
Date: Fri, 15 Jun 2012 09:42:56 +0200

> logs (you need to infer it from what emitted this error), it's a transport

That's not helpful. I hate tag soup as much as you, but in the end it's 
simply HTML, *is* standardized, and has tons of implementations.

> most users don't need to read it twice, and it needlessly overloads the wire.

For 511, the reason for the scope of the spec is that for the case it 
covers, there is no protocol around that can be used. The network 
authentication dialogue *is* HTML, and those who design and deploy these 
devices see no need to make it more complicated. After all, it "works" 
(mostly).

The spec for 511 just addresses one specific problem (this content being 
sent with a success status), nothing more. The scope is *intentionally* 
constrained. That's not an error.

Best regards, Julian

---

From: John Sullivan <[email-masked]>
Date: Fri, 15 Jun 2012 15:11:09 +0100

> mid-transaction when the block occurred, the web client must then retry the

No, the expectation is that the block is permanent, the entity
implementing the block *wants* to (semi)permanently pollute that
URL, retries are unlikely to be useful.

The user *might* be able to get the block lifted by complaining to
the right place, but that's a longer process. In some cases the
local sysadmin might agree a particular URL is blocked in error
with an email round-trip, in some cases it might take societal
upheaval to get the blocking policy changed.

> there is no clear way to unlock except by parsing not-standardised

For 511/captive portal the situation is that the user probably can
rapidly resolve the block themselves, either by completing the attached
form, or by phoning/walking to reception etc, and then trying again.
The blocker doesn't want to break that access permanently, they just
want to get paid for it first, but I don't see how except in vanishingly
rare cases there could be fully automatic recovery, it's going to require
some form of user action.

John
--

---

From: Cullen Jennings <[email-masked]>
Date: Fri, 15 Jun 2012 08:43:09 -0600

When trying to decide if a new error code is needed in a protocol, I often ask myself "what should the receiver of this new error code do differently when it gets the new error code". Could you say more about what is the semantic meaning of this error code to the client and how that client should behave when receiving it? I'm particularly interested in how that differs from existing error codes. 

Thanks, Cullen

---

From: "Nicolas Mailhot" <[email-masked]>
Date: Fri, 15 Jun 2012 16:48:46 +0200

Le Ven 15 juin 2012 16:11, John Sullivan a écrit :
> but I don't see how except in vanishingly

It doesn't have to be vanishingly rare.
It is vanishingly rare right now because part of the action loop is not
visible to the web client because it was pushed to page body instead of being
reflected in the http dialog

-- 
Nicolas Mailhot

---

From: Ross Nicoll <[email-masked]>
Date: Fri, 15 Jun 2012 15:50:51 +0100

I believe the intention is that the client knows where to attribute 
blame; a 403 from a site would mean that the site is rejecting the 
request (and therefore I should talk to the admins if I feel this is in 
error), a 451 (or otherwise) would indicate a third party is rejecting 
the request (and therefore I need to talk to that third party instead).

(I did actually originally argue 403 was sufficient, but feel a good 
case has been made for there needing to be an extra code)

Ross

On 15/06/2012 15:43, Cullen Jennings wrote:
>> From:<[email-masked]>

---

From: Tim Bray <[email-masked]>
Date: Fri, 15 Jun 2012 08:18:41 -0700

I think the intent is increased transparency of operation, and the specific
action to achieve this would be for the user-agent to pass along to the
user at least the "Unavailable for Legal Reasons" message and also the
content of the response body, if provided.  Very similar to the way 404 is
handled now.

-T

On Fri, Jun 15, 2012 at 7:43 AM, Cullen Jennings <[email-masked]> wrote:

>
> When trying to decide if a new error code is needed in a protocol, I often
> ask myself "what should the receiver of this new error code do differently
> when it gets the new error code". Could you say more about what is the
> semantic meaning of this error code to the client and how that client
> should behave when receiving it? I'm particularly interested in how that
> differs from existing error codes.
>
> Thanks, Cullen
>
>
>