Subject: [OAUTH-WG] Adam Roach's Discuss on draft-ietf-oauth-discovery-08: (with DISCUSS and COMMENT)

From: Adam Roach <[email-masked]>
Date: Mon, 22 Jan 2018 18:41:54 -0800

Adam Roach has entered the following ballot position for
draft-ietf-oauth-discovery-08: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-oauth-discovery/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

Thanks to everyone who worked on this specification. I think it's well-written,
clear, and useful. I fully endorse publication, and intend to ballot "yes" once
we come to an agreement on the issue I describe below.

The problem I'm running into is the URL synthesis rules described in section
3.1 for multi-tenancy engage in exactly the kind of behavior that RFC 5785 was
designed to head off: it creates URLs all over the path space of the authority,
rather than coralling all synthesized URLs to live under only one top-level
directory. One of the key aspects of the principles of the web architecture is
URI opacity <https://www.w3.org/TR/webarch/#uri-opacity>, which generally
precludes clients from synthesizing URLs. RFC 5785 was intended as a very
limited carve-out to the principle of URI opacity, and was carefully limited to
a single top-level path element. This specification oversteps that carve-out by
exploding the location that "Well-Known" synthesized URLs can appear: it
literally increases it from one location (the root) to infinite locations (at
the end of any arbitrary path).

Fortunately, this defect is trivial to fix. Rather than placing .well-known
path components *after* the path identified by an issuer identifier, you place
them *before* it, which amends this document's usage to be within the spirit
intended by RFC 5785. For example, the example in section 3.1:

     GET /issuer1/.well-known/oauth-authorization-server HTTP/1.1
     Host: example.com

Would instead become:

     GET /.well-known/oauth-authorization-server/issuer1 HTTP/1.1
     Host: example.com

----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

Section 1.1: [this is an editorial suggestion that I leave to the editors'
discretion] This document makes use of uncapitalized "must", "should", and
"may" in places. Please consider using the RFC 8174 boilerplate instead of the
RFC 2119 boilerplate.

Section 7.2: [this is an important procedural comment that really should be
resolved prior to publication] The addition of restrictions to registries
established by RFC 6749 would seem to require that this document formally
include "Updates: RFC6749" in its metadata, as well as a mention of such an
update in its Abstract and Introduction sections.

---

From: Mike Jones <[email-masked]>
Date: Wed, 24 Jan 2018 00:21:10 +0000

Hi Adam,

Thanks for taking the time to do a thorough read of the specification.  Replies are inline prefixed by "Mike>".
 
-----Original Message-----
From: Adam Roach [mailto:[email-masked]] 
Sent: Monday, January 22, 2018 6:42 PM
To: The IESG <[email-masked]>
Cc: [email-masked]; Hannes Tschofenig <[email-masked]>; [email-masked]; [email-masked]; [email-masked]
Subject: Adam Roach's Discuss on draft-ietf-oauth-discovery-08: (with DISCUSS and COMMENT)

When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)

Thanks to everyone who worked on this specification. I think it's well-written, clear, and useful. I fully endorse publication, and intend to ballot "yes" once we come to an agreement on the issue I describe below.

The problem I'm running into is the URL synthesis rules described in section
3.1 for multi-tenancy engage in exactly the kind of behavior that RFC 5785 was designed to head off: it creates URLs all over the path space of the authority, rather than coralling all synthesized URLs to live under only one top-level directory. One of the key aspects of the principles of the web architecture is URI opacity <https://www.w3.org/TR/webarch/#uri-opacity>, which generally precludes clients from synthesizing URLs. RFC 5785 was intended as a very limited carve-out to the principle of URI opacity, and was carefully limited to a single top-level path element. This specification oversteps that carve-out by exploding the location that "Well-Known" synthesized URLs can appear: it literally increases it from one location (the root) to infinite locations (at the end of any arbitrary path).

Fortunately, this defect is trivial to fix. Rather than placing .well-known path components *after* the path identified by an issuer identifier, you place them *before* it, which amends this document's usage to be within the spirit intended by RFC 5785. For example, the example in section 3.1:

Would instead become:

Mike> Interesting suggestion.  I would agree with your suggestion if we were designing this from scratch.

The problem is that this specification is based the much older and widely deployed specification "OpenID Connect Discovery 1.0" https://openid.net/specs/openid-connect-discovery-1_0.html, where these semantics are defined in the second paragraph of https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfig.  This OAuth spec removes the OpenID-specific parts of this discovery spec and keeps the rest, while remaining compatible with it by design.  As discussed with Mark Nottingham during the IANA registration process for the .well-known URI (which he approved), this specification writes down what is already industry practice for publishing OAuth metadata - which subsets the OpenID Connect metadata to use only the OAuth-specific parts.

The behavior of concatenating the .well-known suffix to the issuer path is hard-coded into at an absolute minimum, the eleven client libraries listed in the "RP Config" column of http://openid.net/certification/#RPs.  In truth, it's hard-coded into essentially every OAuth client library that uses metadata to configure the OAuth client software and all the OAuth servers that support multiple tenants.  These could number in the hundreds.

An example showing a production deployment of the current method is https://login.microsoftonline.com/5608b4e0-e26d-4bd1-8a1a-d57d7ae2af8c/v2.0/.well-known/openid-configuration.  There are many more.

This spec is explicitly paving cowpaths.  Yes, you suggest a better path that could have been taken.  But (torturing the metaphor) it's not where the cows are.  We owe it to the industry to codify the widespread and sufficiently-well-working existing practice.  Thanks for your understanding of this.

Section 1.1: [this is an editorial suggestion that I leave to the editors'
discretion] This document makes use of uncapitalized "must", "should", and "may" in places. Please consider using the RFC 8174 boilerplate instead of the RFC 2119 boilerplate.

Mike>   I'd be glad to update to the RFC 8174 boilerplate.

Section 7.2: [this is an important procedural comment that really should be resolved prior to publication] The addition of restrictions to registries established by RFC 6749 would seem to require that this document formally include "Updates: RFC6749" in its metadata, as well as a mention of such an update in its Abstract and Introduction sections.

Mike> I've actually worked with the IESG and IANA in the past to update the instructions for existing registries.  https://tools.ietf.org/html/rfc7638#section-6 adds to the instructions to the Designated Experts for three existing registries.  The way IANA did this was to add [RFC7638] to the References lists in the registries themselves.  For instance, see that the instructions for the "JSON Web Key Types" registry at https://www.iana.org/assignments/jose/jose.xhtml#web-key-types has a References value of "[RFC7518][RFC7638]".  RFC 7618 did not update RFC 7518 - only the registry established by it.  I propose that we follow this precedent and do the same thing here.

				Thanks again,
				-- Mike

---

From: Adam Roach <[email-masked]>
Date: Tue, 23 Jan 2018 20:14:10 -0600

I understand the difficulty here, as breaking changes -- especially at 
this late state -- are a pretty big deal, and I feel bad that this 
situation has arisen. I'll note that I was not the first to identify 
this specific "better path that could have been taken"; John Bradley 
described the same approach over five years ago in a thread that you 
participated in: 
<https://bitbucket.org/openid/connect/issues/638/discovery-using-issuer-identifer-w-path>.

One of the key reasons existing external specifications are brought to 
the IETF is to file off architectural warts such as this. As is standard 
practice for BOFs involving externally-incubated solutions, there was 
discussion at the OAUTH BOF regarding change control and breaking 
changes. Looking at the minutes, a key proponent asserted "we want oauth 
reviewed, and if problems are found, fixed", while the sponsoring AD 
clarified: "No one expects bit-for-bit compatibility.... Code can change 
over time."  I understand the "code can change over time" argument is 
quite blithe when applied out of the blue, but it seems much more 
relevant when it was one of the arguments that was made for the very 
creation of the working group.

Minimally, if we come to some kind of IETF consensus (after looping in 
the relevant parties) that allows publication of a document that squats 
on the entire path namespace of HTTP URLs -- and I want to be clear that 
I don't believe this is possible -- then this document needs to update 
BCP 190, to amend the following normative requirement on how IETF 
specifications handle URLs:

    Scheme definitions define the presence, format, and semantics of a
    path component in URIs; all other specifications MUST NOT constrain,
    or define the structure or the semantics for any path component.

    The only exception to this requirement is registered "well-known"
    URIs, as specified by [RFC5785].  See that document for a description
    of the applicability of that mechanism.

To be clear: amending BCP 190 is way, WAY outside the purview and 
charter of OAUTH. Overriding this normative language would require input 
from the appropriate community (maybe httpbis, but they're almost 
closed), and you'd need to build consensus that what we did in BCP 190 
-- which was a formal codification of web principles promulgated by the 
W3C, with roots tracing back to Roy Fielding's thesis that coined the 
REST concept itself -- was incorrect.

The fundamental problem here is that the cows have been walking through 
private gardens for years, despite clear rules that say they should not. 
Breaking out the paving equipment to formally seize that land for 
OAUTH's exclusive use would only make that transgression so much worse.

Thanks.

> Mike> I've actually worked with the IESG and IANA in the past to update the instructions for existing registries.  https://tools.ietf.org/html/rfc7638#section-6 adds to the instructions to the Desi…

If there's precedent, I'm okay going along with that.

/a

---

From: John Bradley <[email-masked]>
Date: Wed, 24 Jan 2018 17:07:25 -0300

It seems that I was smarter 5 years ago:)

We went with the easier construction for developers in the end.   That may
not have been the best choice in retrospect.

Mike and I are discussing with the other authors how we can address this
while minimising developer confusion, given that a large number of
implementations use the existing construction.

John B.

On Jan 23, 2018 6:14 PM, "Adam Roach" <[email-masked]> wrote:

>> Thanks to everyone who worked on this specification. I think it's
>> well-written, clear, and useful. I fully endorse publication, and intend to
>> ballot "yes" once we come to an agreement on the issue I describe below.
>>
>> 3.1 for multi-tenancy engage in exactly the kind of behavior that RFC
>> 5785 was designed to head off: it creates URLs all over the path space of
>> the authority, rather than coralling all synthesized URLs to live under
>> only one top-level directory. One of the key aspects of the principles of
>> the web architecture is URI opacity <https://www.w3.org/TR/webarch
>> /#uri-opacity>, which generally precludes clients from synthesizing
>> URLs. RFC 5785 was intended as a very limited carve-out to the principle of
>> URI opacity, and was carefully limited to a single top-level path element.
>> This specification oversteps that carve-out by exploding the location that
>> "Well-Known" synthesized URLs can appear: it literally increases it from
>> one location (the root) to infinite locations (at the end of any arbitrary
>> path).
>>
>> Fortunately, this defect is trivial to fix. Rather than placing
>> .well-known path components *after* the path identified by an issuer
>> identifier, you place them *before* it, which amends this document's usage
>> to be within the spirit intended by RFC 5785. For example, the example in
>> section 3.1:
>>
>> The problem is that this specification is based the much older and widely
>> deployed specification "OpenID Connect Discovery 1.0"
>> https://openid.net/specs/openid-connect-discovery-1_0.html, where these
>> semantics are defined in the second paragraph of
>> https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfig.
>> This OAuth spec removes the OpenID-specific parts of this discovery spec
>> and keeps the rest, while remaining compatible with it by design.  As
>> discussed with Mark Nottingham during the IANA registration process for the
>> .well-known URI (which he approved), this specification writes down what is
>> already industry practice for publishing OAuth metadata - which subsets the
>> OpenID Connect metadata to use only the OAuth-specific parts.
>>
>> The behavior of concatenating the .well-known suffix to the issuer path
>> is hard-coded into at an absolute minimum, the eleven client libraries
>> listed in the "RP Config" column of http://openid.net/certification/#RPs.
>> In truth, it's hard-coded into essentially every OAuth client library that
>> uses metadata to configure the OAuth client software and all the OAuth
>> servers that support multiple tenants.  These could number in the hundreds.
>>
>> An example showing a production deployment of the current method is
>> https://login.microsoftonline.com/5608b4e0-e26d-4bd1-8a1a-d5
>> 7d7ae2af8c/v2.0/.well-known/openid-configuration.  There are many more.
>>
>> This spec is explicitly paving cowpaths.  Yes, you suggest a better path
>> that could have been taken.  But (torturing the metaphor) it's not where
>> the cows are.  We owe it to the industry to codify the widespread and
>> sufficiently-well-working existing practice.  Thanks for your understanding
>> of this.
>>
>
> situation has arisen. I'll note that I was not the first to identify this
> specific "better path that could have been taken"; John Bradley described
> the same approach over five years ago in a thread that you participated in:
> One of the key reasons existing external specifications are brought to the
> IETF is to file off architectural warts such as this. As is standard
> discussion at the OAUTH BOF regarding change control and breaking changes.
> Looking at the minutes, a key proponent asserted "we want oauth reviewed,
> and if problems are found, fixed", while the sponsoring AD clarified: "No
> one expects bit-for-bit compatibility.... Code can change over time."  I
> understand the "code can change over time" argument is quite blithe when
> applied out of the blue, but it seems much more relevant when it was one of
> the arguments that was made for the very creation of the working group.
>
> Minimally, if we come to some kind of IETF consensus (after looping in the
> relevant parties) that allows publication of a document that squats on the
> entire path namespace of HTTP URLs -- and I want to be clear that I don't
> believe this is possible -- then this document needs to update BCP 190, to
> amend the following normative requirement on how IETF specifications handle
> URLs:
>
> To be clear: amending BCP 190 is way, WAY outside the purview and charter
> of OAUTH. Overriding this normative language would require input from the
> appropriate community (maybe httpbis, but they're almost closed), and you'd
> need to build consensus that what we did in BCP 190 -- which was a formal
> codification of web principles promulgated by the W3C, with roots tracing
> back to Roy Fielding's thesis that coined the REST concept itself -- was
> incorrect.
>
> Breaking out the paving equipment to formally seize that land for OAUTH's
> exclusive use would only make that transgression so much worse.
>
>> discretion] This document makes use of uncapitalized "must", "should",
>> and "may" in places. Please consider using the RFC 8174 boilerplate instead
>> of the RFC 2119 boilerplate.
>>
> Section 7.2: [this is an important procedural comment that really should
>> be resolved prior to publication] The addition of restrictions to
>> registries established by RFC 6749 would seem to require that this document
>> formally include "Updates: RFC6749" in its metadata, as well as a mention
>> of such an update in its Abstract and Introduction sections.
>>
>> Mike> I've actually worked with the IESG and IANA in the past to update
>> the instructions for existing registries.  https://tools.ietf.org/html/rf
>> c7638#section-6 adds to the instructions to the Designated Experts for
>> three existing registries.  The way IANA did this was to add [RFC7638] to
>> the References lists in the registries themselves.  For instance, see that
>> the instructions for the "JSON Web Key Types" registry at
>> https://www.iana.org/assignments/jose/jose.xhtml#web-key-types has a
>> References value of "[RFC7518][RFC7638]".  RFC 7618 did not update RFC 7518
>> - only the registry established by it.  I propose that we follow this
>> precedent and do the same thing here.
>>
>
>

---

From: Adam Roach <[email-masked]>
Date: Wed, 24 Jan 2018 15:13:59 -0600

On 1/24/18 2:07 PM, John Bradley wrote:
> Mike and I are discussing with the other authors how we can address 
> this while minimising developer confusion, given that a large number 
> of implementations use the existing construction.

Thanks. I'm not unsympathetic to the deployment issues here, and I think 
there's room for defining a migration path for both servers and clients.

 From the perspective of OAuth service operators, I'll point out that 
putting in place a programmatic "301 Moved Permanently" redirect to 
handle older clients is pretty trivial; e.g. Apache's mod_rewrite can 
accomplish this like so:

RewriteRule "^/(.+)/\.well-known/(.+)" "/.well-known/$2/$1" [R=301]

Ideally, the document would have a short non-normative discussion of the 
operational issues related to migration from pre-standard to standard 
deployments that covers this topic. (If the multi-tenancy arrangement on 
the server only gives tenants access to URL subtrees, it is certainly 
within the operator's purview to reverse the sense of this redirection 
so that tenants have the ability to modify their own discovery documents 
-- so long as it's not the *specification* requiring such a naming 
scheme, doing so is completely in the spirit of the web architecture).

For the clients -- as long as the document makes it clear that the old 
format is deprecated, I'm not opposed to text that requires clients to 
first check the correct location (under the top-level /.well-known 
directory), and if (and only if) it receives a 404 (and only a 404) 
response, says that the client MAY then check the /.../.well-known 
location for compatibility with pre-standard servers. This would need to 
include language that the body of the response from the /.../.well-known 
query might contain something completely unexpected and unrelated to 
OAuth (as site administrators choose the meaning of URLs in general), 
and that they need to be prepared to deal with that. This client 
behavior should be configurable by administrators and/or end-users, and 
ideally default to off.

In both cases, I would also expect a forward-looking statement saying 
that clients and servers are expected to discontinue this behavior at 
some point in the future.

/a