Subject: [art] Fwd: New Version Notification for draft-nottingham-how-did-that-get-into-the-repo-00.txt

From: Mark Nottingham <[email-masked]>
Date: Thu, 16 Aug 2018 15:28:50 +1000

FYI, an idea to help prevent accidental disclosure of authentication tokens. Thoughts? I've circulated it some had had a generally positive response.

Prettier copy at:
  https://mnot.github.io/I-D/how-did-that-get-into-the-repo/

Cheers,

> Begin forwarded message:
> 
> From: [email-masked]
> Subject: New Version Notification for draft-nottingham-how-did-that-get-into-the-repo-00.txt
> Date: 16 August 2018 at 3:27:14 pm AEST
> To: "Mark Nottingham" <[email-masked]>
> 
> 
> A new version of I-D, draft-nottingham-how-did-that-get-into-the-repo-00.txt
> has been successfully submitted by Mark Nottingham and posted to the
> IETF repository.
> 
> Name:		draft-nottingham-how-did-that-get-into-the-repo
> Revision:	00
> Title:		The secret-token URI Scheme
> Document date:	2018-08-16
> Group:		Individual Submission
> Pages:		5
> URL:            https://www.ietf.org/internet-drafts/draft-nottingham-how-did-that-get-into-the-repo-00.txt
> Status:         https://datatracker.ietf.org/doc/draft-nottingham-how-did-that-get-into-the-repo/
> Htmlized:       https://tools.ietf.org/html/draft-nottingham-how-did-that-get-into-the-repo-00
> Htmlized:       https://datatracker.ietf.org/doc/html/draft-nottingham-how-did-that-get-into-the-repo
> 
> 
> Abstract:
>   This document registers the "secret-token" URI scheme, to aid in the
>   identification of authentication tokens.
> 
> 
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat
> 

--
Mark Nottingham   https://www.mnot.net/

---

From: "Manger, James" <[email-masked]>
Date: Thu, 16 Aug 2018 07:27:27 +0000

Your ABNF says a secret-token-URI can only use a 66-char alphabet, then text says other chars are allows as %-encodings. Shouldn't the ABNF explicitly show it allows %xx, or (better still) only allow the 66-char alphabet (so the example needs to change).

The scheme part of URIs is case-insensitive so I'm not sure you can say the URI string is matched "character-for-character, case-sensitive". An example that looks like hex, but supposed to be case-sensitive is not ideal.

--
James Manger

From: art [mailto:[email-masked]] On Behalf Of Mark Nottingham
Sent: Thursday, 16 August 2018 3:29 PM
To: Applications and Real-Time Area Discussion <[email-masked]>
Subject: [art] Fwd: New Version Notification for draft-nottingham-how-did-that-get-into-the-repo-00.txt

Prettier copy at:

Cheers,

Begin forwarded message:

From: [email-masked]<mailto:[email-masked]>
Subject: New Version Notification for draft-nottingham-how-did-that-get-into-the-repo-00.txt
Date: 16 August 2018 at 3:27:14 pm AEST
To: "Mark Nottingham" <[email-masked]<mailto:[email-masked]>>

Abstract:

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org<http://tools.ietf.org>.

The IETF Secretariat

---

From: Graham Klyne <[email-masked]>
Date: Thu, 16 Aug 2018 15:06:42 +0100

>    https://mnot.github.io/I-D/how-did-that-get-into-the-repo/

I think the goal of this proposal is highly desirable, and the idea seems to be 
reasonable.

But I find myself wondering about what needs to happen for this to become 
sufficiently widely employed to be useful (i.e. to significantly reduce 
accidental disclosure of credentials in public repositories).

I could imagine a (somewhat utopian) endgame where all libraries that require 
credentials to be provided accept them only in the form of secret-token URI - I 
imagine this would drive actual use of the scheme.  But that's a big ask, and 
seems unlikely to happen any time soon.  But are there any sufficiently widely 
used libraries that, if modified to accept these URIs, would effectively drive 
its adoption and use?

Or another tack with a similar theme: is there any advantage to be gained from 
using a URI scheme for this that might make a secret-token URI a natural way to 
present a literal credential (which, it occurs to me, makes it a bit like a 
special kind of data: URI)?  For example, if a deployed application reads its 
credentials from a protected file system area (like ~/.ssh for private keys on 
Linux), then using a file: URI could be used for that, yet also allow the 
possibility (e.g. for testing) of using a secret-token URI for a credential 
provided as an immediate literal.

Or could a scheme like this be part of a wider framework of tying in with 
password/credential managers to avoid direct submission of a credential.  (Doing 
for applications what password managers currently do for people.)  Might the 
definition of a secret-token scheme encourage purveyors of credential management 
stores to allow credentials to be accessed indirectly using other URIs (with 
appropriate access controls, of course)?  Is there a possible tie-in here with 
oauth/openid here?

And this raises a couple of other lesser questions in my mind:

1. does this need to be a URI scheme?  (See above for one reason why maybe yes)

2. could the data: scheme with a new MIME type be a reasonable alternative choice?

(I'm not expressing an opinion here, just trying to explore the possible 
solution space a little.)

#g
--

> _______________________________________________
> art mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/art
>

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 17 Aug 2018 11:04:35 +1000

> On 16 Aug 2018, at 5:27 pm, Manger, James <[email-masked]> wrote:
> 
> Your ABNF says a secret-token-URI can only use a 66-char alphabet, then text says other chars are allows as %-encodings. Shouldn’t the ABNF explicitly show it allows %xx, or (better still) only allow the 66-char alphabet (so the example needs to change).

Yes. Will address in an update (possibly by just referring to 3986 ABNF, will see).

>  
> The scheme part of URIs is case-insensitive so I’m not sure you can say the URI string is matched “character-for-character, case-sensitive”. An example that looks like hex, but supposed to be case-sensitive is not ideal.

The token should be emitted, stored and submitted in the same form -- do you really think people will be trying to normalise it?

Cheers,

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 17 Aug 2018 11:08:21 +1000

> On 17 Aug 2018, at 12:06 am, Graham Klyne <[email-masked]> wrote:
> 
> I think the goal of this proposal is highly desirable, and the idea seems to be reasonable.
> 
> But I find myself wondering about what needs to happen for this to become sufficiently widely employed to be useful (i.e. to significantly reduce accidental disclosure of credentials in public repositories).
> 
> I could imagine a (somewhat utopian) endgame where all libraries that require credentials to be provided accept them only in the form of secret-token URI - I imagine this would drive actual use of the scheme.  But that's a big ask, and seems unlikely to happen any time soon.  But are there any sufficiently widely used libraries that, if modified to accept these URIs, would effectively drive its adoption and use?

I was thinking of taking a different approach; getting cloud providers to issue their tokens in this format. Even after a couple of them does it, it should provide value.

> Or another tack with a similar theme: is there any advantage to be gained from using a URI scheme for this that might make a secret-token URI a natural way to present a literal credential (which, it occurs to me, makes it a bit like a special kind of data: URI)?  For example, if a deployed application reads its credentials from a protected file system area (like ~/.ssh for private keys on Linux), then using a file: URI could be used for that, yet also allow the possibility (e.g. for testing) of using a secret-token URI for a credential provided as an immediate literal.

That's interesting, but really now it's just a string in a certain (fairly free) format, to aid identification of *it*. Certainly software could be written to intercept such strings and store them differently.

> Or could a scheme like this be part of a wider framework of tying in with password/credential managers to avoid direct submission of a credential.  (Doing for applications what password managers currently do for people.)  Might the definition of a secret-token scheme encourage purveyors of credential management stores to allow credentials to be accessed indirectly using other URIs (with appropriate access controls, of course)?  Is there a possible tie-in here with oauth/openid here?

Hmm. I'd have to see it sketched out, I think.

> 1. does this need to be a URI scheme?  (See above for one reason why maybe yes)

I don't think so, but it's a very convenient way to make it a "special" string -- and people are already using URI schemes in that fashion anyway.

> 2. could the data: scheme with a new MIME type be a reasonable alternative choice?

I suspect that will raise the barrier (if only in perception) for adoption.

Cheers,

> 
> (I'm not expressing an opinion here, just trying to explore the possible solution space a little.)
> 
> #g
> --
> 
> 
>> 
>> 
>>> Begin forwarded message:
>>>

---

From: John C Klensin <[email-masked]>
Date: Fri, 17 Aug 2018 00:12:59 -0400

--On Friday, August 17, 2018 11:04 +1000 Mark Nottingham
<[email-masked]> wrote:

>> The scheme part of URIs is case-insensitive so I'm not sure
>> you can say the URI string is matched
>> "character-for-character, case-sensitive". An example
>> that looks like hex, but supposed to be case-sensitive is not
>> ideal.
> 
> The token should be emitted, stored and submitted in the same
> form -- do you really think people will be trying to normalise
> it?

First of all, if a string looks like Hex, yes, I can imagine
people trying to case fold it (which is different from
normalizing it).  More generally, while there has been a general
rule all along that running text should be normalized only as
part of matching or comparison operations, the rule is less
clear for identifiers both because special circumstances (as in
IDNA) may suggest getting things into normalized form as early
as possible and keeping them that way and the consensus about
other identifiers is, IMO, shifting.  So, yes, depending on how
those token are processed and through what, I can imagine them
being normalized without those who are processing them paying
much attention to whether or not that is happening.

   john

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 17 Aug 2018 14:19:51 +1000

On 17 Aug 2018, at 2:12 pm, John C Klensin <[email-masked]> wrote:
> 
> depending on how
> those token are processed and through what

Effectively, they're opaque strings that are handed to a user, who uses them to access a service. There's already a common understanding that if they're transformed in that process, they'll fail to work.

---

From: Stian Soiland-Reyes <[email-masked]>
Date: Fri, 17 Aug 2018 13:00:36 +0000

I think this is a good idea, as in theory a CI system just needs to do

  grep -l -r secrettoken: .

to ensure no secrets were checked in (and not output the secret in the error).

I don't think it's a concern if the scheme "kicks off" globally or not, as any institution or open source project can independently choose to use the secrettoken: scheme as a convention in their own code base and do such simple tests.

But of course to be useful, if the service do not expect "secrettoken:" URIs, then the code will also need to do some kind of string replacement (assuming no % escapes):

  secret = configLine.replace("secrettoken:", "") 

but then that source code line (as well as the "grep" tester) would match "secrettoken:" and trigger a fault.

This is in a way the similar trap to what could happen with ps | grep:

stain@biggie:~$ ps -ef | grep secrettoken:
stain    21817 21727  0 13:35 pts/22   00:00:00 grep --color=auto secrettoken:

So perhaps the spec can have a tip in the end, to avoid such self-detection by using it through Regular Expression
patterns like ^secrettoken[:]  

  secret = re.sub("^secrettoken[:]", "", configLine)

or by URL parsing to pick up the authority (which would also resolve any % escapes, and bail out on invalid URIs):

  uri = urllib.parse.urlparse(configLine).path
  assert uri.scheme == "secrettoken"
  secret = uri.path

BTW. here's my shell hack showing how detection could work in bash:

stain@biggie:~/src/foo$   ! ( grep -l -r secrettoken[:] . && echo Secret token detected ) 
./config.ini
Secret token detected
stain@biggie:~/src/foo$  echo $?
1

stain@biggie:~/src/foo$  rm config.init 
stain@biggie:~/src/foo$  ! ( grep -l -r secrettoken[:] . && echo Secret token detected >&2 ) 
stain@biggie:~/src/foo$  echo $?
0

(the status code inversion means the CI job will fail only if the grep succeeds)

--
Stian Soiland-Reyes
The University of Manchester
https://www.esciencelab.org.uk/
https://orcid.org/0000-0001-9842-9718

________________________________________
From: art [[email-masked]] on behalf of Mark Nottingham [[email-masked]]
Sent: 17 August 2018 02:08
To: Graham Klyne
Cc: Applications and Real-Time Area Discussion
Subject: Re: [art] Fwd: New Version Notification for draft-nottingham-how-did-that-get-into-the-repo-00.txt

Cheers,

---

From: John C Klensin <[email-masked]>
Date: Fri, 17 Aug 2018 11:45:16 -0400

--On Friday, August 17, 2018 14:19 +1000 Mark Nottingham
<[email-masked]> wrote:

> Effectively, they're opaque strings that are handed to a user,
> who uses them to access a service. 

Mark,

I understood that.  I also understand that "handed to" is a
little imprecise -- unless the strings are really short and easy
to type (your ABNF allows one-character strings but, as the
draft effectively points out, that would be useless or worse for
the purposes of the draft), users are going to pass these things
through applications or libraries not under your control or even
copy and paste them.  Both have been known in the past to
transform strings.

> There's already a common
> understanding that if they're transformed in that process,
> they'll fail to work.

And we have a great deal of experience that suggests "fail to
work" translates into either high support costs for someone,
lack of adoption and eventual disappearance, or both.   We
should not be standardizing such things (and implicitly or
explicitly recommending them) without making that risk very
clear.

I recognize and agree with you about the underlying problem that
motivates this.  I'm a bit concerned that we are taking an
already complex mechanism (managing adequate authentication
tokens and the equivalent) and trying to solve it by adding
complexity (an identifier for the identifier) and then
recommending surrounding that in a cloak of obscurity (see last
paragraph of Section 4).  I also note that, if the URI scheme
name and syntax are removed and these URIs (in truncated form)
might be used in contexts in which the more-secret bearer tokens
might also appear, you are leaving it to heuristics or trial and
error to determine how a string should be interpreted.  I don't
understand this well enough to know whether that would be an
additional security risk or just a time-consuming nuisance. 

You mention that "This URI scheme is intended to prevent
accidental disclosure; it cannot prevent intentional
disclosure".   With a nod to an old colleague's observation
about the difference between making something fool proof and
trying to make it d**m fool proof, it seems to me that you are
lumping fairly extreme and stupid carelessness in with
accidental disclosure.  I'm not sure whether that it reasonable
or likely to be effective.

So, assuming you can overcome the IETF's traditional dislike for
security by obscurity or argue that it is worthwhile in this
case, let me suggest that a good deal more in the way of
applicability statement-like text and guidance would be be
helpful here... and probably necessary for a standards-track
document.  Without it, I don't believe it is possible to assess
whether this is a mechanism that will solve real problems and
which ones.   If you are not prepared to write that text because
no one really knows how things will work out, that is a case for
an Experimental spec instead of a standards-track one.

best,
    john

---

From: Martin J. Dürst <[email-masked]>
Date: Sun, 19 Aug 2018 17:06:18 +0900

>    https://mnot.github.io/I-D/how-did-that-get-into-the-repo/

A few thoughts:

- Overall, worth a try.

- Some people may want to use base64 for these tokens.
   What about allowing '+', '=' (both of these can appear
   in path elements, or just use pchar and get all of sub-delims,
   and pct-encoded on top of that).
   What about also allowing '/'? This is a bit tricky, but should
   work out because relative secret-token URIs (i.e. without the
   scheme) will defeat the whole purpose of the scheme anyway.

- Please make sure you say that UTF-8 is used for non-ASCII characters
   in %-escaping. Please see the last paragraph of Section 2 of RFC 3986.
   While I don't expect too many non-ASCII characters in security tokens,
   it's better to be on the safe side.

- Committing software that checks for the absence of "secret-token" URIs
   may be difficult. The trick is to artificially compose the string
   "secret-token" in the software, e.g. as "secret" + "-token".

Regards,    Martin.

---

From: [email-masked] (Dale R. Worley)
Date: Sun, 19 Aug 2018 12:05:07 -0400

Various comments:

Yes, of course, software will (incorrectly) attempt to normalize URIs,
or any other string, and you have to underline that it should not.

The grammar doesn't admit %-encodings, although the text does.  That
needs to be straightened out.  OTOH, if %-encodings are permitted, what
happens if you change the case of the hex chars?  IIRC, URI rules
require that to not change the meaning.

"secret-token" seems to be excessively long, but OTOH, if you want
people to filter codebases for a string, you want the string to be long
enough to not appear randomly very often.

> provided as an immediate literal.

This is an interesting point and shows some more value-added of this
proposal.

Dale

---

From: Martin J. Dürst <[email-masked]>
Date: Mon, 20 Aug 2018 02:44:53 +0900

Stian, please be careful. It's secret-token:, not secrettoken: !

We can do some bikeshedding if really necessary (I don't think it's 
necessary), but if we do, then please explicitly.

Regards,   Martin.

On 2018/08/17 22:00, Stian Soiland-Reyes wrote:
> ../config.ini
> .

-- 
Prof. Dr.sc. Martin J. Dürst
Department of Intelligent Information Technology
College of Science and Engineering
Aoyama Gakuin University
Fuchinobe 5-1-10, Chuo-ku, Sagamihara
252-5258 Japan

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 28 Aug 2018 10:06:28 +1000

Thanks for the feedback!

> On 19 Aug 2018, at 6:06 pm, Martin J. Dürst <[email-masked]> wrote:
> 

pchar seems reasonable. 

>  it's better to be on the safe side.

Done.

>  "secret-token" in the software, e.g. as "secret" + "-token".

I leave this as an exercise for the reader...