Subject: Re: Working Group Last Call: draft-ietf-httpbis-bcp56bis-08

From: Julian Reschke <[email-masked]>
Date: Sun, 6 Jan 2019 19:28:58 +0100

On 2018-11-30 20:46, Patrick McManus wrote:
> Hi All - I believe that BCP56bis is ready for Working Group Last Call.
> ...

Here's my mainly editorial feedback:

- Although HTTP URIs are by definition URLs, the document currently 
seems to avoid saying "URI", even when it should. For instance, the 
correct term really is "URI scheme", not "URL scheme". I believe we 
should check all uses of "URL" and make sure it's correct. Yes, I can 
work on a PR if desired.

- The document talks about discovering URIs of related resources, but 
doesn't even mention URI templates. I understand that using URI 
templates in link header fields still needs work, but minimally we 
should mention this instead of leaving readers wondering...

- Some lists use comma/semicolon as separators but then continue upper-case

- "GET is one of the most common and useful HTTP methods" - indeed. 
Actually it *is* *the* most common and useful method, no?

- "Applications SHOULD NOT define GET requests to have side effects, 
since implementations can and do retry HTTP GET requests that fail." - 
that should be a "MUST NOT", I think.

- Several occurrences where the document says "header" when it should 
say "header field"

- "This means that status codes are not a reliable way to carry 
application-specific signals. Specifying that a particular status code 
has a specific meaning in the context of an application can have 
unintended side effects; if that status code is generated by a generic 
HTTP component can lead clients to believe that the application is in a 
state that wasn’t intended." - this makes it sound as if it's to be 
expected and normal that components rewrite status codes; I don't think 
that is true. The text goes on saying that "404" can be relied on, but 
this keeps me wondering on what this is based. I believe this subject 
needs more discussion.

- "Because the set of registered HTTP status codes can expand, 
applications using HTTP should explicitly point out that clients ought 
to be able to handle all applicable status codes gracefully (i.e., 
falling back to the generic n00 semantics of a given status code; e.g., 
499 can be safely handled as 400 by clients that don’t recognise it). 
This is preferable to creating a “laundry list” of potential status 
codes, since such a list is never complete." - Nit: the list *could* be 
complete if it contains all syntactically valid error codes.

- "Applications using HTTP SHOULD specify if any request headers need to 
be modified or removed upon a redirect; however, this behaviour cannot 
be relied upon, since a generic client (like a browser) will be unaware 
of such requirements." - I would feel better about this if the base 
standard actually defined this the existing header fields.

- "Applications are advised avoid allowing the use of mobile code where 
possible; when it cannot be avoided, the resulting system’s security 
properties need be carefully scrutinised." - maybe "are advised *to* 
avoid"? or even "are advised to disallow"?

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jan 2019 10:30:23 +1100

Thanks, Julian. Responses below.

> On 7 Jan 2019, at 5:28 am, Julian Reschke <[email-masked]> wrote:
> 
> - Although HTTP URIs are by definition URLs, the document currently seems to avoid saying "URI", even when it should. For instance, the correct term really is "URI scheme", not "URL scheme". I believe we should check all uses of "URL" and make sure it's correct. Yes, I can work on a PR if desired.

HTTP *is* a URL scheme, though -- and it's most commonly known for that. Given the audience, to me it doesn't seem prudent to confuse people with technically-correct-but-obtuse terminology. 

How do others feel about this?

> - The document talks about discovering URIs of related resources, but doesn't even mention URI templates. I understand that using URI templates in link header fields still needs work, but minimally we should mention this instead of leaving readers wondering..

Yes. I had reservations about mentioning it for the reasons you highlight, but added:
  https://github.com/httpwg/http-extensions/commit/1ddb98efc2d
Please say if there's somewhere else that could benefit from a mention.

> - Some lists use comma/semicolon as separators but then continue upper-case

Fixed.

> - "GET is one of the most common and useful HTTP methods" - indeed. Actually it *is* *the* most common and useful method, no?

Fixed.

> - "Applications SHOULD NOT define GET requests to have side effects, since implementations can and do retry HTTP GET requests that fail." - that should be a "MUST NOT", I think.

We don't have a corresponding requirement for safety in HTTP; I tend to think it'd better to make this prose, rather than a requirement. See:
  https://github.com/httpwg/http-extensions/commit/0d8d25888

> - Several occurrences where the document says "header" when it should say "header field"

See <https://github.com/httpwg/http-core/issues/111>. Since this document is blocked by the core docs, we can revise that at the RFC Editor if need be.

> - "This means that status codes are not a reliable way to carry application-specific signals. Specifying that a particular status code has a specific meaning in the context of an application can have unintended side effects; if that status code is generated by a generic HTTP component can lead clients to believe that the application is in a state that wasn’t intended." - this makes it sound as if it's to be expected and normal that components rewrite status codes; I don't think that is true.

It is normal; e.g., gateways generate 502-504 all the time; caches generate 304. Servers often generate 4xx range responses without consulting the resource that they're serving. 

> The text goes on saying that "404" can be relied on, but this keeps me wondering on what this is based. I believe this subject needs more discussion.

I think the text is as good as we're going to get it in a reasonable amount of time, but if you have suggestions for improvement, please convey them.

> - "Because the set of registered HTTP status codes can expand, applications using HTTP should explicitly point out that clients ought to be able to handle all applicable status codes gracefully (i.e., falling back to the generic n00 semantics of a given status code; e.g., 499 can be safely handled as 400 by clients that don’t recognise it). This is preferable to creating a “laundry list” of potential status codes, since such a list is never complete." - Nit: the list *could* be complete if it contains all syntactically valid error codes.

What about yet-to-be-defined codes? I think the state you refer to can only happen when the status code space is completely exhausted.

> - "Applications using HTTP SHOULD specify if any request headers need to be modified or removed upon a redirect; however, this behaviour cannot be relied upon, since a generic client (like a browser) will be unaware of such requirements." - I would feel better about this if the base standard actually defined this the existing header fields.

Yes; see <https://github.com/httpwg/http-core/issues/38>.

> - "Applications are advised avoid allowing the use of mobile code where possible; when it cannot be avoided, the resulting system’s security properties need be carefully scrutinised." - maybe "are advised *to* avoid"? or even "are advised to disallow"?

Fixed.

Thanks!

--
Mark Nottingham   https://www.mnot.net/

---

From: Julian Reschke <[email-masked]>
Date: Tue, 8 Jan 2019 06:20:40 +0100

> HTTP *is* a URL scheme, though -- and it's most commonly known for that. Given the audience, to me it doesn't seem prudent to confuse people with technically-correct-but-obtuse terminology.

It is. But:

- the document is not always talking about HTTP URLs when it says "URL", and

- some of the referenced material has "URI" in the title, so just 
avoiding the term here IMHO doesn't make things clearer for readers who 
don't have our experience with these terms.

>    https://github.com/httpwg/http-extensions/commit/1ddb98efc2d

+1

> Please say if there's somewhere else that could benefit from a mention.

As a reader of this document, I'd like to find out what the status of 
URI templates is in relation to Link header fields. I understand the 
answer is "we're not there yet", and maybe it would be good to just 
state that.

>    https://github.com/httpwg/http-extensions/commit/0d8d25888

Ok.

> See <https://github.com/httpwg/http-core/issues/111>. Since this document is blocked by the core docs, we can revise that at the RFC Editor if need be.

-1. Revising docs after approval is problematic. (a) We may forget it, 
(b) there is little quality control. I don't think we can send this to 
the RFC Editor before the base docs are done.

> It is normal; e.g., gateways generate 502-504 all the time; caches generate 304. Servers often generate 4xx range responses without consulting the resource that they're serving.

Yes. But, from a protocol point of view, the distinction between 
"server" and "resource" doesn't matter. It's an implementation detail.

The current text sort of implies that defining new status codes is 
inherently wrong because they might not travel correctly; this is IMHO 
very misleading.

> What about yet-to-be-defined codes? I think the state you refer to can only happen when the status code space is completely exhausted.

Yes, it's a nit :-).

> Yes; see <https://github.com/httpwg/http-core/issues/38>.

I'm not happy to have this requirement here until we have understood to 
fulfil it ourselves in the base spec.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jan 2019 16:47:16 +1100

> On 8 Jan 2019, at 4:20 pm, Julian Reschke <[email-masked]> wrote:
> 
> - some of the referenced material has "URI" in the title, so just avoiding the term here IMHO doesn't make things clearer for readers who don't have our experience with these terms.

Specifically, two documents; RFC7320 and RFC5785bis. I've updated "well-known URL" to "well-known URI" for the latter; in my judgement, the former's references aren't directly related enough to use of "URL" to cause such confusion.

> As a reader of this document, I'd like to find out what the status of URI templates is in relation to Link header fields. I understand the answer is "we're not there yet", and maybe it would be good to just state that.

Anyone can mint a header that carries a uri template; we just don't have a generic way to carry one around. Also, I'm wary of saying something like the above, since it would trigger an unnecessary document update as soon as one were defined.

> -1. Revising docs after approval is problematic. (a) We may forget it, (b) there is little quality control. I don't think we can send this to the RFC Editor before the base docs are done.

I'll leave that for the chair to decide. In the meantime: <https://github.com/httpwg/http-extensions/issues/748>

> Yes. But, from a protocol point of view, the distinction between "server" and "resource" doesn't matter. It's an implementation detail.
> 
> The current text sort of implies that defining new status codes is inherently wrong because they might not travel correctly; this is IMHO very misleading.

I really don't see how you get that. The text is talking about using existing status codes, not minting new ones.

> I'm not happy to have this requirement here until we have understood to fulfil it ourselves in the base spec.

<https://httpwg.org/specs/rfc7231.html#considerations.for.new.header.fields> already says "Whether the header field ought to be preserved across redirects." The text you question in BCP56bis is about *new* headers; Core issue #38 is about defining that behaviour for headers that core defines. Why does this block?

---

From: Julian Reschke <[email-masked]>
Date: Tue, 8 Jan 2019 07:08:45 +0100

> Specifically, two documents; RFC7320 and RFC5785bis. I've updated "well-known URL" to "well-known URI" for the latter; in my judgement, the former's references aren't directly related enough to use…

OK, I'll check the other cases whether there's more that I think needs 
to change.

> Anyone can mint a header that carries a uri template; we just don't have a generic way to carry one around. Also, I'm wary of saying something like the above, since it would trigger an unnecessary…

As long as it says "at the time of this writing", not necessarily.

I'm raising this because I feel this actually is something that readers 
of this spec will be looking for.

> ... >>>> - Several occurrences where the document says "header" when it 
should say "header field"
> I'll leave that for the chair to decide. In the meantime: <https://github.com/httpwg/http-extensions/issues/748>

Procedural feedback: this IMHO is something the WG needs to form 
consensus on, not a chair decision.

> I really don't see how you get that. The text is talking about using existing status codes, not minting new ones.

It talks about status codes in general; that includes new ones, no?

> <https://httpwg.org/specs/rfc7231.html#considerations.for.new.header.fields> already says "Whether the header field ought to be preserved across redirects." The text you question in BCP56bis is abo…

Unless I'm missing something, the text is about all header fields: 
"Applications using HTTP SHOULD specify if any request headers need to 
be modified or removed upon a redirect;..."

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jan 2019 17:21:02 +1100

> On 8 Jan 2019, at 5:08 pm, Julian Reschke <[email-masked]> wrote:

> I'm raising this because I feel this actually is something that readers of this spec will be looking for.

I feel like the same criticism could have been made of the URI Templates spec itself, and yet it's silent on the matter.

P.S. <https://mnot.github.io/I-D/link-template/>

>> ... >>>> - Several occurrences where the document says "header" when it should say "header field"
> Procedural feedback: this IMHO is something the WG needs to form consensus on, not a chair decision.

Your point was about how the working group's decision gets incorporated into the document, pre- vs post- LC. That's a matter for the chair(s), the IESG and the RFC Editor (it's not uncommon for the document that gets published as an RFC to be different from that which exists WGLC, based upon input). *Making* the decision requires WG consensus.

Having said that, if it's clearer to hold this document for core, I don't have a problem with that.

> It talks about status codes in general; that includes new ones, no?

Potentially, but new status codes are required to be generic, not application-specific, so it ends up in the same place. Status codes are *not* a guaranteed end-to-end signal; they can (and often are) superseded by HTTP components in the middle.

> Unless I'm missing something, the text is about all header fields: "Applications using HTTP SHOULD specify if any request headers need to be modified or removed upon a redirect;..."

Would "...request headers that it defines..." work?

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Tue, 8 Jan 2019 09:51:57 +0100

> I feel like the same criticism could have been made of the URI Templates spec itself, and yet it's silent on the matter.

Right.

> P.S. <https://mnot.github.io/I-D/link-template/>

Yes. I think it would be good to point people to this, even when it's 
not ready yet.

> Your point was about how the working group's decision gets incorporated into the document, pre- vs post- LC. That's a matter for the chair(s), the IESG and the RFC Editor (it's not uncommon for the…

Right.

> Having said that, if it's clearer to hold this document for core, I don't have a problem with that.

I believe this document is to dependant on the base specs to go to the 
RFC Editor before we're done with those. Consider the cross-linkking. 
We've been burn with this approach before. See the errata for RFC 7240 
(<https://www.rfc-editor.org/errata_search.php?rfc=7240>).

> Potentially, but new status codes are required to be generic, not application-specific, so it ends up in the same place. Status codes are *not* a guaranteed end-to-end signal; they can (and often a…

Under certain well-understood circumstances, by design, or when there is 
a bug. My point is that they usually are reliable when the actual origin 
servers gets to respond.

I'm concerned that the current text will cause people to stuff all 
information into header fields or the payload, and just send 400.

> Would "...request headers that it defines..." work?

Yes.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 8 Jan 2019 20:11:49 +1100

> On 8 Jan 2019, at 7:51 pm, Julian Reschke <[email-masked]> wrote:
> 
> Yes. I think it would be good to point people to this, even when it's not ready yet.

I don't agree; it's a dead I-D.

> I believe this document is to dependant on the base specs to go to the RFC Editor before we're done with those. Consider the cross-linkking. We've been burn with this approach before. See the errata for RFC 7240 (<https://www.rfc-editor.org/errata_search.php?rfc=7240>).
> 
> Under certain well-understood circumstances, by design, or when there is a bug. My point is that they usually are reliable when the actual origin servers gets to respond.

Well-understood by implementers, perhaps, but often not application designers. 

> I'm concerned that the current text will cause people to stuff all information into header fields or the payload, and just send 400.

Yes. Unless you want to intentionally trigger generic HTTP processing based upon a chosen status code, that's best practice.

Fixed.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 8 Jan 2019 10:31:27 +0100

> Yes. Unless you want to intentionally trigger generic HTTP processing based upon a chosen status code, that's best practice.

This is really news to me. This probably requires a new top-level thread.

Best regards, Julian

---

From: Roberto Polli <[email-masked]>
Date: Tue, 8 Jan 2019 14:29:37 +0100

Hi @all,

> This is really news to me. This probably requires a new top-level thread.
FWIW I agree with Julian.

Have a nice day,
R.

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 9 Jan 2019 16:34:19 +1100

Promoting to a new thread, as suggested.

I explored the thinking behind this in a blog post a while back:
  https://www.mnot.net/blog/2017/05/11/status_codes

Julian, do you disagree with that, or just how it's expressed here?

> On 8 Jan 2019, at 8:31 pm, Julian Reschke <[email-masked]> wrote:
>

---

From: Patrick McManus <[email-masked]>
Date: Wed, 9 Jan 2019 13:39:26 -0500

Can you reference the specific text in 56bis that is being discussed.. it
would help focus the discussion. thanks!

On Wed, Jan 9, 2019 at 12:37 AM Mark Nottingham <[email-masked]> wrote:

> >>>> Potentially, but new status codes are required to be generic, not
> application-specific, so it ends up in the same place. Status codes are
> *not* a guaranteed end-to-end signal; they can (and often are) superseded
> by HTTP components in the middle.
> >>>
> >>> Under certain well-understood circumstances, by design, or when there
> is a bug. My point is that they usually are reliable when the actual origin
> servers gets to respond.
> >> Yes. Unless you want to intentionally trigger generic HTTP processing
> based upon a chosen status code, that's best practice.
> >> ...
> >

---

From: Tommy Pauly <[email-masked]>
Date: Wed, 09 Jan 2019 10:47:25 -0800

I believe the text that started this discussion comes from section 4.6 (HTTP Status Codes): https://tools.ietf.org/html/draft-ietf-httpbis-bcp56bis-08#section-4.6

The specific text of interest is:

   Instead, applications using HTTP should specify the implications of
   general classes of responses (e.g., "successful response" for 2xx;
   "client error" for 4xx and "server error" for 5xx), conveying any
   application-specific information in the message body and/or HTTP
   header fields, not the status code.  [RFC7807 <https://tools.ietf.org/html/rfc7807>] provides one way for
   applications using HTTP to do so for error conditions.

   There are limited exceptions to this; for example, applications might
   use 201 (Created) or 404 (Not Found) to convey application semantics
   that are compatible with the generic HTTP semantics of those status
   codes.  In general, though, applications should resist the temptation
   to map their semantics into fine-grained status codes.

Thanks,
Tommy

> On Jan 9, 2019, at 10:39 AM, Patrick McManus <[email-masked]> wrote:
> 
> Can you reference the specific text in 56bis that is being discussed.. it would help focus the discussion. thanks!
> 
> On Wed, Jan 9, 2019 at 12:37 AM Mark Nottingham <[email-masked] <mailto:[email-masked]>> wrote:
>   https://www.mnot.net/blog/2017/05/11/status_codes <https://www.mnot.net/blog/2017/05/11/status_codes>
> 
> > On 8 Jan 2019, at 8:31 pm, Julian Reschke <[email-masked] <mailto:[email-masked]>> wrote:
> > 
> Mark Nottingham   https://www.mnot.net/ <https://www.mnot.net/>
> 
>

---

From: Michael Sweet <[email-masked]>
Date: Wed, 09 Jan 2019 14:09:31 -0500

FWIW I am 100% OK with the text as-is, and this is exactly the approach IPP uses - 200 (really any 2xx status) for any IPP POST, unless we need something "generic" at the HTTP level:

- Authentication (401/403)
- Bad request where the client sends something other than a proper application/ipp message body (400)
- Bad/wrong URI (404)
- Expect: 100-continue (100)
- Redirection (3xx)
- Upgrade to TLS (426 and 101)
- Something bad on the server/printer (5xx)

In the case where the client needs to interact with resources (icons, ICC profiles, localization files, camera video feeds, etc.), the client just does regular HTTP GET (or PUT, for upload) requests with the standard HTTP semantics.

We occasionally add new IPP status codes, but have never needed new HTTP status codes.

This approach has worked well for us for over 20 years now, with (no shit) tens of billions of printers and clients, both locally and over the Internet, spanning almost 50 specifications/RFCs that have extended IPP to support many different architectures and technologies.

> On Jan 9, 2019, at 1:47 PM, Tommy Pauly <[email-masked]> wrote:
> 
> I believe the text that started this discussion comes from section 4.6 (HTTP Status Codes): https://tools.ietf.org/html/draft-ietf-httpbis-bcp56bis-08#section-4.6 <https://tools.ietf.org/html/draft-ietf-httpbis-bcp56bis-08#section-4.6>
> 
>> On Jan 9, 2019, at 10:39 AM, Patrick McManus <[email-masked] <mailto:[email-masked]>> wrote:
>> 

_________________________________________________________
Michael Sweet, Senior Printing System Engineer

---

From: Patrick McManus <[email-masked]>
Date: Wed, 9 Jan 2019 14:54:04 -0500

When particular instances of this have come up in the past, the discussion
seems to hinge on whether a generic HTTP implementation would evolve to do
something meaningful with the new status code other than what it would do
with the x00 one. Generic implementation in this case being something that
wasn't designed for the purpose of implementing the new application-on-http
which can certainly encode the signal in headers or body. I think the text
is consistent with that pattern - perhaps it should call it out a little
more explicitly (i.e. what would an off the shelf library do differently in
the face of 275 vs 200 and a header in this case?)

On Wed, Jan 9, 2019 at 2:12 PM Michael Sweet <[email-masked]> wrote:

> FWIW I am 100% OK with the text as-is, and this is exactly the approach
> IPP uses - 200 (really any 2xx status) for any IPP POST, unless we need
> something "generic" at the HTTP level:
>
> In the case where the client needs to interact with resources (icons, ICC
> profiles, localization files, camera video feeds, etc.), the client just
> does regular HTTP GET (or PUT, for upload) requests with the standard HTTP
> semantics.
>
> This approach has worked well for us for over 20 years now, with (no shit)
> tens of billions of printers and clients, both locally and over the
> Internet, spanning almost 50 specifications/RFCs that have extended IPP to
> support many different architectures and technologies.
>
>
> I believe the text that started this discussion comes from section 4.6
> (HTTP Status Codes):
> https://tools.ietf.org/html/draft-ietf-httpbis-bcp56bis-08#section-4.6
>

---

From: Mark Baker <[email-masked]>
Date: Thu, 10 Jan 2019 10:22:28 -0500

I'm having trouble understanding these paragraphs even after reading
them in context.

(disclaimer; this is my first time reading this draft since probably 01)

On Wed, 9 Jan 2019 at 13:51, Tommy Pauly <[email-masked]> wrote:
>    "client error" for 4xx and "server error" for 5xx),

Since that's just an example, are there other valid (or invalid)
specifications of "implications"?

>conveying any
>    header fields, not the status code.  [RFC7807] provides one way for

What criteria was used to identify 201 and 404 as exceptions? It seems
to me that any generic code could be included.

---

From: Julian Reschke <[email-masked]>
Date: Tue, 15 Jan 2019 15:58:41 +0100

On 2019-01-08 07:08, Julian Reschke wrote:
>>> avoiding the term here IMHO doesn't make things clearer for readers 
>>> who don't have our experience with these terms.
>>
>> Specifically, two documents; RFC7320 and RFC5785bis. I've updated 
>> "well-known URL" to "well-known URI" for the latter; in my judgement, 
>> the former's references aren't directly related enough to use of "URL" 
>> to cause such confusion.
> 

Isssue: <https://github.com/httpwg/http-extensions/issues/757>

PR: <https://github.com/httpwg/http-extensions/pull/758>

This changes URL to URI in the context of "URI scheme" and "URI syntax".

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 15 Jan 2019 12:37:00 -0700

> On 10 Jan 2019, at 8:22 am, Mark Baker <[email-masked]> wrote:
> 

I think it's reasonable to have application-specific implications of classes of responses defined for a specific resource.

> to me that any generic code could be included.

They don't have effect on generic HTTP implementations, beyond their Nxx semantics.

---

From: Julian Reschke <[email-masked]>
Date: Sun, 20 Jan 2019 17:51:20 +0100

> Julian, do you disagree with that, or just how it's expressed here?

Ok, let's look at that blog post first. I just re-read it twice, and it 
isn't as far-reaching as the text in bcp6bis. For instance:

> Beyond that, it’s a mistake to try to map each part of your application “deeply” into HTTP status codes; in most cases the level of granularity you want to be aiming for is much coarser. When in doubt, it’s OK to use the generic status codes 200 OK, 400 Bad Request and 500 Internal Service Error when there isn’t a better fit.

Note the "when in doubt".

Later:

> So You Want To Create a HTTP Status Code
> 
> If after reading all of that, you still think you need a new status code – it’s something that’s potentially applicable to all resources, it has specific effect on HTTP implementations, there isn’t an existing status code that has that effect, and you’re willing to go through the process, take a look at these guidelines.

So that is specific to new codes, not using the ones we have.

So, no, I don't really disagree with that blog post. But that's what 
bcp56bis says. In particular:

> 4.6. HTTP Status Codes
> 
> The primary function of a HTTP status code is to convey semantics for the benefit of generic HTTP software, not to convey application-specific semantics.

But then RFC 7231 just says "The status-code element is a three-digit 
integer code giving the result of the attempt to understand and satisfy 
the request." - much broader.

I disagree with this narrow definition. HTTP after all is an 
application-level protocol.

> Status codes are often generated or overwritten by intermediaries, as well as server and client implementations. This can happen, for example, when network errors are encountered, a captive portal is present, when an implementation is overloaded, or it thinks it is under attack. As a result, the status code that a server-side application generates and the one that the client software receives often differ.

I think the more likely case is that the request actually did not reach 
the origin server, so one is really dealing with a non-authoritative answer.

> This means that status codes are not a reliable way to carry application-specific signals. Specifying that a particular status code has a specific meaning in the context of an application can have unintended side effects; if that status code is generated by a generic HTTP component can lead clients to believe that the application is in a state that wasn’t intended.

But how likely is that for the more specific codes such as 409 or 422 
(defined for ages, just recently added to the base spec)?

> Instead, applications using HTTP should specify the implications of general classes of responses (e.g., “successful response” for 2xx; “client error” for 4xx and “server error” for 5xx), conveying any application-specific information in the message body and/or HTTP header fields, not the status code. [RFC7807] provides one way for applications using HTTP to do so for error conditions.

This sounds like intermediaries can break the status code, but for some 
reason can't break the payload.

(I like adding details to the payload, as per RFC 4918 which IMHO did 
that first, and later RFC 7807, but that does not necessarily imply that 
more specific status codes do not make sense as well)

> There are limited exceptions to this; for example, applications might use 201 (Created) or 404 (Not Found) to convey application semantics that are compatible with the generic HTTP semantics of those status codes. In general, though, applications should resist the temptation to map their semantics into fine-grained status codes.

I think using fine-grained status codes actually does no harm, except 
when developers get carried away with that. Thus the "in doubt" in the 
blog post makes sense.

> Because the set of registered HTTP status codes can expand, applications using HTTP should explicitly point out that clients ought to be able to handle all applicable status codes gracefully (i.e., falling back to the generic n00 semantics of a given status code; e.g., 499 can be safely handled as 400 by clients that don’t recognise it). This is preferable to creating a “laundry list” of potential status codes, since such a list is never complete.

+1 (except for the nit about completeness)

> Applications using HTTP MUST NOT re-specify the semantics of HTTP status codes, even if it is only by copying their definition. They MUST NOT require specific reason phrases to be used; the reason phrase has no function in HTTP, is not guaranteed to be preserved by implementations, and the reason phrase is not carried at all in the HTTP/2 [RFC7540] message format.

+1

> Applications MUST only use registered HTTP status codes. As with methods, new HTTP status codes are rare, and required (by [I-D.ietf-httpbis-semantics]) to be registered with IETF Review. Similarly, HTTP status codes are generic; they are required (by [I-D.ietf-httpbis-semantics]) to be potentially applicable to all resources, not just to those of one application.

+1

> When authors believe that a new status code is required, they are encouraged to engage with the HTTP community early, and document their proposal as a separate HTTP extension, rather than as part of an application’s specification.

+1

I'll finish with two more thoughts:

1) There are limits to using the payload for machine-readable status 
information -- it doesn't work well when the request was GET and is 
supposed to make sense in a generic browser, or when it's the result of 
a classic HTML form POST.

2) Maybe it would worthwhile to discuss the usability of a few concrete 
status codes, such as 403, 409, or 422.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 20 Jan 2019 13:39:29 -0800

> On 20 Jan 2019, at 8:51 am, Julian Reschke <[email-masked]> wrote:
> 
> Ok, let's look at that blog post first. I just re-read it twice, and it isn't as far-reaching as the text in bcp6bis. For instance:
> 

Noted. What meaning are you reading into those three words?

> So that is specific to new codes, not using the ones we have.

What you you mean by "that" here?

> So, no, I don't really disagree with that blog post. But that's what bcp56bis says. In particular:

Do you mean that's *not* what bcp56bis says?

> But then RFC 7231 just says "The status-code element is a three-digit integer code giving the result of the attempt to understand and satisfy the request." - much broader.
> 
> I disagree with this narrow definition. HTTP after all is an application-level protocol.

Point taken, but the application it was defined for (web browsing) have latent semantics that are sometimes an awkward fit for other applications.

> I think the more likely case is that the request actually did not reach the origin server, so one is really dealing with a non-authoritative answer.

True. I'll adjust that.

> But how likely is that for the more specific codes such as 409 or 422 (defined for ages, just recently added to the base spec)?
> 
> This sounds like intermediaries can break the status code, but for some reason can't break the payload.
> 
> (I like adding details to the payload, as per RFC 4918 which IMHO did that first, and later RFC 7807, but that does not necessarily imply that more specific status codes do not make sense as well)
> 
> I think using fine-grained status codes actually does no harm, except when developers get carried away with that. Thus the "in doubt" in the blog post makes sense.

I think we're in agreement here. Looking at it again, the underlying issue that's most concerning is that developers try to do a 1:1 mapping between their application's errors and status codes, and quickly tie themselves into knots. The advice should address that problem directly.

Take a look at this edit and see if it's better:
  https://github.com/httpwg/http-extensions/commit/81498701482b44b?diff=split

[snip]

> 1) There are limits to using the payload for machine-readable status information -- it doesn't work well when the request was GET and is supposed to make sense in a generic browser, or when it's the result of a classic HTML form POST.

Sorry, can you expand upon this? (Or provide suggested text...)

> 2) Maybe it would worthwhile to discuss the usability of a few concrete status codes, such as 403, 409, or 422.

PRs welcome.

Cheers,

---

From: Julian Reschke <[email-masked]>
Date: Mon, 21 Jan 2019 07:12:14 +0100

> Noted. What meaning are you reading into those three words?

When not in doubt, that is when I *know* there is a more specific status 
code, it's OK or even better to use that one?

> What you you mean by "that" here?

The whole paragraph. See paragraph title.

Yes.

> Point taken, but the application it was defined for (web browsing) have latent semantics that are sometimes an awkward fit for other applications.

I'm not sure whether "defined for web browsing" is accurate. HTTP/0.9 
yes, but 1.*?

"have latent semantics that are sometimes an awkward fit" - could you be 
a bit more specific? Example maybe?

>    https://github.com/httpwg/http-extensions/commit/81498701482b44b?diff=split

That looks much better. Thanks!

> Sorry, can you expand upon this? (Or provide suggested text...)

Let's assume an HTML form (no JS) posts something, and the server wants 
to return a 4xx such as 409 or 422. It can do that, but it can't send an 
JSON error description in place of the HTML (or it would need to do 
conneg or UA sniffing to decide what format to return).

>> 2) Maybe it would worthwhile to discuss the usability of a few concrete status codes, such as 403, 409, or 422.

The intent was to look at some specific codes to get to general 
recommendations - but we seem to be there already.

Best regards, Julian

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 21 Jan 2019 19:01:16 -0800

High order bit - do you think we need to make more changes, or was that commit sufficient to address your concerns?

Cheers,

> On 20 Jan 2019, at 10:12 pm, Julian Reschke <[email-masked]> wrote:
> 
> When not in doubt, that is when I *know* there is a more specific status code, it's OK or even better to use that one?
> 
>>> Later:
>>> 
> I'm not sure whether "defined for web browsing" is accurate. HTTP/0.9 yes, but 1.*?
> 
> "have latent semantics that are sometimes an awkward fit" - could you be a bit more specific? Example maybe?
> 
> 
> Let's assume an HTML form (no JS) posts something, and the server wants to return a 4xx such as 409 or 422. It can do that, but it can't send an JSON error description in place of the HTML (or it would need to do conneg or UA sniffing to decide what format to return).
> 
> The intent was to look at some specific codes to get to general recommendations - but we seem to be there already.
> 
> Best regards, Julian

---

From: Julian Reschke <[email-masked]>
Date: Tue, 22 Jan 2019 06:27:04 +0100

> High order bit - do you think we need to make more changes, or was that commit sufficient to address your concerns?

I believe this is sufficient. Thanks for the changes!