Subject: [httpapi] rfc7807 errata or just "more"

From: Sanjay Dalal <[email-masked]>
Date: Fri, 15 Jan 2021 10:32:13 -0800

Hello all,

Thanks folks for providing instructions on how to provide feedback to an
RFC. This feedback is for RFC 7807 <https://tools.ietf.org/html/rfc7807>,
"Problem Details for HTTP APIs".

I like RFC 7807. In fact, I have designed Error Catalog Service
<https://github.com/sdatspun2/error-catalog-service> such that the problem
types of RFC 7807 can be managed (CRUDL) and used in error responses, API
definition, API documentation, API testing, etc.

However, I find that RFC 7807 omitted one very common use case in its
proposed schema for Problem Details Object
<https://tools.ietf.org/html/rfc7807#section-3.1> (section 3.1). This
use case is about reporting of multiple errors of the same problem type in
an error response. RFC suggests defining an extension for this use case.

As you know, having multiple schema validation related errors in payload of
HTTP requests is a very common scenario for the HTTP APIs. How can we
improve the Problem Details Object schema to accommodate this common use
case? We should not expect API developers to respond with one error at a
time in the case of 400 scenarios. That would be an unpleasant developer
experience. Asking the API developers to define an extension for such a
common scenario hurts in adoption of the Problem Types.

In my opinion, instead of the proposed singular "*instance*" property of
type string, there should be an array of *instances. *You can find what we
have done for the error response of the Error Catalog service
<https://github.com/sdatspun2/error-catalog-service/blob/master/ErrorResponseInErrorCatalog.md>.There
could be other ways.

I have communicated with both Erik Wilde and Mark Nottingham about this
issue. Both of them are open to discussing a revision under this WG if the
group thinks it is necessary. Would love to get your comments, opinions,
approaches and suggestions.

thanks and regards,
sanjay

>    I am not too familiar with the process to comment an RFC but is there
in general
    a form where I can provide written feedback to existing specifications
or is
    this done through this mailing list?

>On Sat, Jan 9, 2021 at 9:40 AM Salz, Rich <rsalz=
[email-masked]> wrote:
RFC's are published documents, not drafts; the name "request for comments"
is a node to IETF history.  How to provide feedback on published RFC's
depends on the amount and nature of feedback.  If you have a well-contained
technical error, you want to report an "errata" against the RFC.  If it's
more philosophical or just "more", it might be better to find the WG that
published the RFC and post to that mailing list (see above).

---

From: Ben Bucksch <[email-masked]>
Date: Sat, 16 Jan 2021 05:08:17 +0100

Hello Sanjay, hello everybody,

thank you for bringing this up. In fact, this topic is the very reason 
why I joined this mailling list and WG: I find that error responses - 
and more generally error handling - is severely lacking in many systems 
I come across, and it's causing problems during software development.

I wanted to propose to make a new RFC for nailling down not only the 
error responses, but also what information is expected in the response, 
and the behavioral contract. Errors should be part of the protocol and 
API contract.

I had written some document for my project and teams, in my role as 
software architect, based on my experience, and I wanted to share that 
with you as starting point for an RFC. The document I have is not 
suitable as Standard document, it's an software architecture document, 
but I think many of the ideas are viable to be included in the RFC.

I would like to ask you, esp. the leaders of the WG: Would you be 
willing to discuss and adopt such a new RFC? It would probably include 
RFC 7807, but significantly extend it in its scope.

Ben Bucksch

Am 15.01.21 um 19:32 schrieb Sanjay Dalal:
> Hello all,
>
> Thanks folks for providing instructions on how to provide feedback to 
> an RFC. This feedback is for RFC 7807 
> <https://tools.ietf.org/html/rfc7807>, "Problem Details for HTTP APIs".
>
> <https://github.com/sdatspun2/error-catalog-service> such that the 
> problem types of RFC 7807 can be managed (CRUDL) and used in error 
> responses, API definition, API documentation, API testing, etc.
>
> use case is about reporting of multiple errors of the same problem 
> type in an error response. RFC suggests defining an extension for this 
> use case.
>
> As you know, having multiple schema validation related errors in 
> payload of HTTP requests is a very common scenario for the HTTP APIs. 
> How can we improve the Problem Details Object schema to accommodate 
> this common use case? We should not expect API developers to respond 
> with one error at a time in the case of 400 scenarios. That would be 
> an unpleasant developer experience. Asking the API developers to 
> define an extension for such a common scenario hurts in adoption of 
> the Problem Types.
>
> In my opinion, instead of the proposed singular "*instance*" property 
> of type string, there should be an array of *instances. *You can find 
> what we have done for the error response of the Error Catalog service 
> *
> *
> I have communicated with both Erik Wilde and Mark Nottingham about 
> this issue. Both of them are open to discussing a revision under this 
> WG if the group thinks it is necessary. Would love to get your 
> comments, opinions, approaches and suggestions.
>
> thanks and regards,
> sanjay
>
>
> >    I am not too familiar with the process to comment an RFC but is 
> there in general
>     a form where I can provide written feedback to existing 
> specifications or is
> >On Sat, Jan 9, 2021 at 9:40 AM Salz, Rich 
> <rsalz=[email-masked] 
> <mailto:[email-masked]>> wrote:
> RFC's are published documents, not drafts; the name "request for 
> comments" is a node to IETF history.  How to provide feedback on 
> published RFC's depends on the amount and nature of feedback. If you 
> have a well-contained technical error, you want to report an "errata" 
> against the RFC.  If it's more philosophical or just "more", it might 
> be better to find the WG that published the RFC and post to that 
> mailing list (see above).
>

---

From: "Salz, Rich" <[email-masked]>
Date: Sat, 16 Jan 2021 15:04:49 +0000

*   I would like to ask you, esp. the leaders of the WG: Would you be willing to discuss and adopt such a new RFC? It would probably include RFC 7807, but significantly extend it in its scope.

It’s up to the members of the WG to decide if they want to work on this, and it’s up to the WG chairs (working with our Area Director) to work on modifying the charter, if necessary, or pointing to another WG.  In this particular case, since 7807 was done in the now-closed “appsaws” group (https://datatracker.ietf.org/wg/appsawg/charter/) it seems appropriate to discuss it here.  If the WG decides to take this work on, then we’ll make sure the  charter covers it.

So, chat away.

---

From: Darrel Miller <[email-masked]>
Date: Sat, 16 Jan 2021 21:41:37 +0000

Hi Sanjay,

From: httpapi <[email-masked]> on behalf of Sanjay Dalal

As you know, having multiple schema validation related errors in payload of HTTP requests is a very common scenario for the HTTP APIs.

I'm not sure an instances array is going to solve the issue here.  In your example, at the resource level, multiple issues have been detected as schema validation problems and you will need some explicit type to identify schema validation problems..  As you say there may be many validation constraints that were violated within the response payload.  However, it is likely that they are more than one type of validation failure.  Just pointing to the instances of the failures without describing what kind of failures those are would not be a great experience.  That additional information about what kind of failures each of those instances experienced is going to need to be in extensions and the client will need to understand the semantics of the type.

I accept there will be some scenarios where there are multiple instances of problems associated to a single resource that are homogeneous, but there are going to be just as many that are heterogenous.  I don't really see the issue with using the type to define the necessary extensions.

Darrel

---

From: Sanjay Dalal <[email-masked]>
Date: Mon, 18 Jan 2021 09:01:55 -0800

Hi Darrel,

>Just pointing to the instances of the failures without describing what
kind of failures those are would not be a great experience.

I agree. However, if we have a schema that accommodates pointers to
failure(s) and more details about the errors as part of RFC it would help
many API developers. Such a schema could be defined in a very generic
fashion like what we have attempted to do in the error catalog service API
at
https://github.com/sdatspun2/error-catalog-service/blob/master/ErrorResponseInErrorCatalog.md.
These schemas are derived from rfc7807 and from JSON Schema spec, section
10 respectively.

>That additional information about what kind of failures each of those
instances experienced is going to need to be in extensions and the client
will need to understand the semantics of the type...I accept there will be
some scenarios where there are multiple instances of problems associated
with a single resource that are homogeneous, but there are going to be just
as many that are heterogenous. I don't really see the issue with using the
type to define the necessary extensions.

If we can provide a solution for the homogenous use case (I believe that is
80% in 80-20) and suggest to use extension for the heterogenous use case
that would go a long way, imo. In fact, it would increase adoption of
rfc7807 significantly. I did not use rfc7807 while designing PayPal's
proprietary
schema
<https://github.com/paypal/api-standards/blob/master/api-style-guide.md#error-handling>
for the error response. I am sure we were not alone.

Thanks.
sanjay

On Sat, Jan 16, 2021 at 1:41 PM Darrel Miller <[email-masked]>
wrote:

> Hi Sanjay,
>
> *From:* httpapi <[email-masked]> on behalf of Sanjay Dalal
>
> As you know, having multiple schema validation related errors in payload
> of HTTP requests is a very common scenario for the HTTP APIs.
>
> I'm not sure an instances array is going to solve the issue here.  In your
> example, at the resource level, multiple issues have been detected as
> schema validation problems and you will need some explicit type to identify
> schema validation problems..  As you say there may be many validation
> constraints that were violated within the response payload.  However, it is
> likely that they are more than one type of validation failure.  Just
> pointing to the instances of the failures without describing what kind of
> failures those are would not be a great experience.  That additional
> information about what kind of failures each of those instances experienced
> is going to need to be in extensions and the client will need to understand
> the semantics of the type.
>
> I accept there will be some scenarios where there are multiple instances
> of problems associated to a single resource that are homogeneous, but there
> are going to be just as many that are heterogenous.  I don't really see the
> issue with using the type to define the necessary extensions.
>
> Darrel
>
>

---

From: Sanjay Dalal <[email-masked]>
Date: Mon, 18 Jan 2021 09:11:41 -0800

Hello Ben,

>I find that error responses - and more generally error handling - is
severely lacking in many systems I come across, and it's causing problems
during software development.

Huge +1. Lack of it, increases cost of support, affects developer
experience.

> I wanted to propose to make a new RFC for nailling down not only the
error responses, but also what information is expected in the response, and
the behavioral contract. Errors should be part of the protocol and API
contract.

Looking forward to seeing your starting point content and collaborating
with you. Hope you got a chance to look at the Error Catalog Service
<https://github.com/sdatspun2/error-catalog-service> and its OpenAPI
extension
<https://github.com/sdatspun2/error-catalog-service/blob/master/OpenAPIExtensionForErrorType.md>
.

thanks,
sanjay

On Fri, Jan 15, 2021 at 8:08 PM Ben Bucksch <[email-masked]> wrote:

> thank you for bringing this up. In fact, this topic is the very reason why
> I joined this mailling list and WG: I find that error responses - and more
> generally error handling - is severely lacking in many systems I come
> across, and it's causing problems during software development.
>
> I wanted to propose to make a new RFC for nailling down not only the error
> responses, but also what information is expected in the response, and the
> behavioral contract. Errors should be part of the protocol and API contract.
>
> software architect, based on my experience, and I wanted to share that with
> you as starting point for an RFC. The document I have is not suitable as
> Standard document, it's an software architecture document, but I think many
> of the ideas are viable to be included in the RFC.
>
> I would like to ask you, esp. the leaders of the WG: Would you be willing
> to discuss and adopt such a new RFC? It would probably include RFC 7807,
> but significantly extend it in its scope.
>
> Ben Bucksch
>
>
> of HTTP requests is a very common scenario for the HTTP APIs. How can we
> type string, there should be an array of *instances. *You can find what
> we have done for the error response of the Error Catalog service
> issue. Both of them are open to discussing a revision under this WG if
> the group thinks it is necessary. Would love to get your comments,
> opinions, approaches and suggestions.
>
> thanks and regards,
> sanjay
>
>
> --
> httpapi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/httpapi
>

---

From: Darrel Miller <[email-masked]>
Date: Mon, 18 Jan 2021 22:25:07 +0000

Ben, Sanjay,

From: httpapi <[email-masked]> on behalf of Sanjay Dalal <[email-masked]>
>I find that error responses - and more generally error handling - is severely lacking in many systems I come across, and it's causing problems during software development.

Huge +1. Lack of it, increases cost of support, affects developer experience.

I agree that HTTP errors are often not handled well.  However, usually I find that is because devs over simplify client code to something like:

   if status == 200 then success else throw exception.

I find it is this part of RFC 7231 that most developers don't follow,

   HTTP status codes are extensible.  HTTP clients are not required to
   understand the meaning of all registered status codes, though such
   understanding is obviously desirable.  However, a client MUST
   understand the class of any status code, as indicated by the first
   digit, and treat an unrecognized status code as being equivalent to
   the x00 status code of that class, with the exception that a
   recipient MUST NOT cache a response with an unrecognized status code.

Clients must handle each class of status code, but are not required to handle every specific failure condition.

Is it your expectation that a client must understand all service-specific modes of failure in order to make a request to an HTTP API?

On Fri, Jan 15, 2021 at 8:08 PM Ben Bucksch <[email-masked]<mailto:[email-masked]>> wrote:
I wanted to propose to make a new RFC for nailling down not only the error responses, but also what information is expected in the response, and the behavioral contract. Errors should be part of the protocol and API contract.
Errors are part of the HTTP protocol.  That's what 4XX and 5XX status codes are.  Possible errors can be called out in API descriptions like OpenAPI but in my opinion they should never be relied on to be exhaustive by a client.  The layered nature of HTTP means that any intermediary could return an HTTP error response that is not provided by the OpenAPI description.

Creating a further refinement of HTTP status codes using service specific problem types can be useful as long as clients can opt-in to processing those new error types.

Darrel

---

From: Darrel Miller <[email-masked]>
Date: Mon, 18 Jan 2021 22:40:25 +0000

Sanjay,

From: Sanjay Dalal <[email-masked]>

Such a schema could be defined in a very generic fashion like what we have attempted to do in the error catalog service API at https://github.com/sdatspun2/error-catalog-service/blob/master/ErrorResponseInErrorCatalog.md<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsdatspun2%2Ferror-catalog-service%2Fblob%2Fmaster%2FErrorResponseInErrorCatalog.md&data=04%7C01%7CDarrel.Miller%40microsoft.com%7Cbf9181abf8b3408bcc2108d8bbd2c9c1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637465862131622116%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=hhOhNEMnH1EHSZRCT2hFcgf49A7flaibWQeXl%2F6zMG0%3D&reserved=0>. These schemas are derived from rfc7807 and from JSON Schema spec, section 10 respectively.

We define something similar in the Microsoft API Guidelines
https://github.com/microsoft/api-guidelines/blob/vNext/Guidelines.md#error--object  An "error" object has a "details" property that is an array of "error" objects.  I'm not sure how much it is used though.

RFC7807 could add "containedProblems" property (or some similar name) that is an array of problem objects.  That would be the spiritual equivalent.

Darrel

---

From: Roberto Polli <[email-masked]>
Date: Tue, 19 Jan 2021 09:30:20 +0100

Hi @all,

> That would be the spiritual equivalent.

I would like to find a convergent solution for both Problem and
Content-Warning, eg. see
https://github.com/dret/I-D/blob/master/http-warning/draft-cedik-http-warning-02.txt#L295
which proposes the use of a list of Problem.json which - imho -
provide redundant information (eg. `status` should always be the
same).

Kind regards,
R.
-- 
Roberto Polli
API Expert
M. +39 3406522736
MID
DIPARTIMENTO PER LA
TRASFORMAZIONE
DIGITALE
Presidenza del Consiglio dei Ministri - Ministro per l'innovazione
tecnologica e la digitalizzazione
https://innovazione.gov.it/

---

From: Sanjay Dalal <[email-masked]>
Date: Wed, 20 Jan 2021 17:11:07 -0800

>RFC7807 could add "containedProblems" property (or some similar name) that
is an array of problem objects.

Agree. What would be the next step? Should a repo be created so we can
start working on a revision?

>I would like to find a convergent solution for both Problem and
Content-Warning

This could broaden the scope of Problem Details and Content-Warning both
which may not be desirable, imo. Would love to hear from other members of
the group.

https://github.com/dret/I-D/blob/master/http-warning/draft-cedik-http-warning-02
.

"Note that this specification specifically targets warnings and not errors,
meaning that while it may be useful for clients to understand the warning
condition and act on it, they also may choose to ignore it and treat the
response as a successful one."

On Tue, Jan 19, 2021 at 12:30 AM Roberto Polli <
[email-masked]> wrote:

> > RFC7807 could add "containedProblems" property (or some similar name)
> that is an array of problem objects.
> > That would be the spiritual equivalent.
>

---

From: Roberto Polli <[email-masked]>
Date: Thu, 21 Jan 2021 10:15:25 +0100

Hi Sanjay,

Il giorno gio 21 gen 2021 alle ore 02:11 Sanjay Dalal
<[email-masked]> ha scritto:
> >RFC7807 could add "containedProblems" property (or some similar name) that is an array of problem objects.
> Agree. What would be the next step? Should a repo be created so we can start working on a revision?

Before creating a repo, I'd start with a shared gdoc with a basic
proposal. The content can be in markdown, if you like.

> >I would like to find a convergent solution for both Problem and Content-Warning
> This could broaden the scope of Problem Details and Content-Warning both which may not be desirable, imo. Would love to hear from other members of the group.
>
> https://github.com/dret/I-D/blob/master/http-warning/draft-cedik-http-warning-02.
> "Note that this specification specifically targets warnings and not errors, meaning that while it may be useful for clients to understand the warning condition and act on it, they also may choose to ignore it and treat the response as a successful one."

http-warning has a proposal for both warning notification (the
headers) and warning serialization (how to convey the actual
warnings). iiuc the sentence above refers to the headers part, not the
warning serialization.

My opinion is that we should investigate eventual convergence between
the warning serialization and the "containedProblems".

Looking forward to further comments,
R.