Subject: [precis] usernames in PRECIS and http-auth

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 07 Mar 2014 11:02:47 -0700

As promised at IETF 89, I have compared the username definitions from 
draft-oiwa-precis-httpauthprep-00 and draft-ietf-precis-saslprepbis-06, 
with the purpose of perhaps harmonizing the two approaches so that we 
can avoid multiplying PRECIS profiles beyond necessity.

(I am writing this email on the flight home, and I neglected to load up 
the meeting minutes before leaving, so I might not address all of the 
relevant points in this message.)

First, let's look at the syntax definitions.

draft-ietf-precis-saslprepbis-06 states that a username can (a) consist 
of one of more userparts, or (b) can be of the form userpart@domainpart. 
The ABNF definition is:

    username   = userpart [1*(1*SP userpart)]
               / userpart ’@’ domainpart

    userpart   = 1*(idpoint)

    domainpart = IP-literal / IPv4address / ifqdn

    ifqdn      = 1*1023(domainpoint)

Where:

* an "idpoint" is a UTF-8 encoded Unicode code point that conforms to 
the PRECIS "IdentifierClass"
* an "IPv4address" is as defined in RFC 3986
* an "IP-literal" is as defined in RFC 3986
* a "domainpoint" is a UTF-8 encoded Unicode code point that conforms to 
RFC 5890

By contrast, draft-oiwa-precis-httpauthprep states that a username 
consists of one or more UTF-8 encoded Unicode code points that conform 
to the PRECIS "IdentifierClass". The ABNF definition is:

    userpart   = 1*(idpoint)

We quickly see that an http-auth username is a legal PRECIS username, 
since 1*(idpoint) is simply a userpart as defined in 
draft-ietf-precis-saslprepbis-06, and a PRECIS username can consist of 
only one userpart. Therefore, in this respect, I think the httpauthprep 
text (whether it appears in a standalone document or elsewhere) can 
simply state that for purposes of HTTP authentication a username is a 
userpart as defined in draft-ietf-precis-saslprepbis.

Second, let's look at the string preparation method.

Here, too, draft-oiwa-precis-httpauthprep-00 is a subset of 
draft-ietf-precis-saslprepbis-06. The preparation method in 
draft-ietf-precis-saslprepbis-06 is:

    1.  The base string class is the "IdentifierClass" specified in
        [I-D.ietf-precis-framework].
    2.  Fullwidth and halfwidth characters MUST be mapped to their
        decomposition equivalents.
    3.  So-called additional mappings MAY be applied, such as those
        defined in [I-D.ietf-precis-mappings].
    4.  Uppercase and titlecase characters might be mapped to their
        lowercase equivalents (see Section 4.2.1 below).
    5.  Unicode Normalization Form C (NFC) MUST be applied to all
        characters.

By contrast, the preparation method in draft-oiwa-precis-httpauthprep-00 is:

    1.  Fullwidth and halfwidth characters MUST be mapped to their
        decomposition equivalents.
    2.  Additional mappings SHOULD NOT be applied, such as those defined
        in [I-D.ietf-precis-mappings], unless there are implementation-
        dependent reasons to do so, or these are exceptionally required
        by specific authentication schemes.
    3.  Case mapping is not applied.
    4.  Unicode Normalization Form C (NFC) MUST be applied to all
        characters.o

These two defnitions agree on the width mapping and normalization rules 
described in the PRECIS framework. They appear to differ slightly with 
regard to the additional mapping and case mapping rules. However, 
draft-oiwa-precis-httpauthprep-00 is merely a bit more restrictive with 
regard to matters that are purely optional according to 
draft-ietf-precis-saslprepbis-06. For example, the option of saying that 
"case mapping is not applied" is allowed by the text "uppercase and 
titlecase characters might be mapped to their lowercase equivalents".

To make this clearer, I suggest that we modify the advice in 
draft-oiwa-precis-httpauthprep-00 so that it no longer defines its own 
PRECIS profile. Instead, I suggest that we phrase the httpauthprep text 
in terms of the username profile in draft-ietf-precis-saslprepbis-06. To 
my mind, something like this would work:

###

For the purposes of HTTP authentication, a username conforms to the 
syntax definition and preparation methods specified in 
[I.D-draft-ietf-precis-saslprepbis], with the following limitations:

* a username conforms to the "userpart" construction from 
[I-D.draft-ietf-precis-saslprepbis]
* case mapping is not applied
* delimiter mappings, special mappings, and other so-called additional 
mappings [I-D.draft-ietf-precis-mappings] are not applied

###

As far as I can see, this simplifies the httpautheprep text quite a bit.

(By the way, draft-oiwa-precis-httpauthprep-00 also seems to copy the 
password profile verbatim from draft-ietf-precis-saslprepbis. I think 
this text can be removed entirely, so that the httpauthprep text can 
simply reference draft-ietf-precis-saslprepbis.)

Yutaka and Alexey (and others), let me know what you think about what I 
have written here.

Thanks,

Peter

---

From: John C Klensin <[email-masked]>
Date: Sat, 08 Mar 2014 17:17:48 -0500

FWIW, this seems about right to me.
    john

--On Friday, 07 March, 2014 11:02 -0700 Peter Saint-Andre
<[email-masked]> wrote:

> As promised at IETF 89, I have compared the username
> definitions from draft-oiwa-precis-httpauthprep-00 and
> draft-ietf-precis-saslprepbis-06, with the purpose of perhaps
> harmonizing the two approaches so that we can avoid
> multiplying PRECIS profiles beyond necessity.
> 
> (I am writing this email on the flight home, and I neglected
> to load up the meeting minutes before leaving, so I might not
> address all of the relevant points in this message.)
> 
> draft-ietf-precis-saslprepbis-06 states that a username can
> (a) consist of one of more userparts, or (b) can be of the
> form userpart@domainpart. The ABNF definition is:
> 
>                / userpart '@' domainpart
> 
>     userpart   = 1*(idpoint)
> 
> * an "idpoint" is a UTF-8 encoded Unicode code point that
> conforms to the PRECIS "IdentifierClass"
> * a "domainpoint" is a UTF-8 encoded Unicode code point that
> conforms to RFC 5890
> 
> By contrast, draft-oiwa-precis-httpauthprep states that a
> username consists of one or more UTF-8 encoded Unicode code
> points that conform to the PRECIS "IdentifierClass". The ABNF
> definition is:
> 
>     userpart   = 1*(idpoint)
> 
> We quickly see that an http-auth username is a legal PRECIS
> username, since 1*(idpoint) is simply a userpart as defined in
> draft-ietf-precis-saslprepbis-06, and a PRECIS username can
> consist of only one userpart. Therefore, in this respect, I
> think the httpauthprep text (whether it appears in a
> standalone document or elsewhere) can simply state that for
> purposes of HTTP authentication a username is a userpart as
> defined in draft-ietf-precis-saslprepbis.
> 
> These two defnitions agree on the width mapping and
> normalization rules described in the PRECIS framework. They
> appear to differ slightly with regard to the additional
> mapping and case mapping rules. However,
> draft-oiwa-precis-httpauthprep-00 is merely a bit more
> restrictive with regard to matters that are purely optional
> according to draft-ietf-precis-saslprepbis-06. For example,
> the option of saying that "case mapping is not applied" is
> allowed by the text "uppercase and titlecase characters might
> be mapped to their lowercase equivalents".
> 
> draft-oiwa-precis-httpauthprep-00 so that it no longer defines
> its own PRECIS profile. Instead, I suggest that we phrase the
> httpauthprep text in terms of the username profile in
> draft-ietf-precis-saslprepbis-06. To my mind, something like
> this would work:
> 
>### 
> 
> For the purposes of HTTP authentication, a username conforms
> to the syntax definition and preparation methods specified in
> * delimiter mappings, special mappings, and other so-called
> additional mappings [I-D.draft-ietf-precis-mappings] are not
> applied
> 
>### 
> 
> (By the way, draft-oiwa-precis-httpauthprep-00 also seems to
> copy the password profile verbatim from
> draft-ietf-precis-saslprepbis. I think this text can be
> removed entirely, so that the httpauthprep text can simply
> reference draft-ietf-precis-saslprepbis.)
> 
> Yutaka and Alexey (and others), let me know what you think
> about what I have written here.
> 
> Thanks,
> 
> Peter
> 
> _______________________________________________
> precis mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/precis

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 12 Mar 2014 17:45:20 +0000

Hi Peter,

On 07/03/2014 18:02, Peter Saint-Andre wrote:
> draft-oiwa-precis-httpauthprep-00 and 
> harmonizing the two approaches so that we can avoid multiplying PRECIS 
> profiles beyond necessity.
>
> (I am writing this email on the flight home, and I neglected to load 
> up the meeting minutes before leaving, so I might not address all of 
> the relevant points in this message.)
>
> draft-ietf-precis-saslprepbis-06 states that a username can (a) 
> consist of one of more userparts, or (b) can be of the form 
> userpart@domainpart. The ABNF definition is:
>
> * a "domainpoint" is a UTF-8 encoded Unicode code point that conforms 
> to RFC 5890
>
> only one userpart. Therefore, in this respect, I think the 
> httpauthprep text (whether it appears in a standalone document or 
> elsewhere) can simply state that for purposes of HTTP authentication a 
> username is a userpart as defined in draft-ietf-precis-saslprepbis.
Agreed.
> These two defnitions agree on the width mapping and normalization 
> rules described in the PRECIS framework. They appear to differ 
> slightly with regard to the additional mapping and case mapping rules. 
> However, draft-oiwa-precis-httpauthprep-00 is merely a bit more 
> restrictive with regard to matters that are purely optional according 
> to draft-ietf-precis-saslprepbis-06. For example, the option of saying 
> that "case mapping is not applied" is allowed by the text "uppercase 
> and titlecase characters might be mapped to their lowercase equivalents".
>
> PRECIS profile. Instead, I suggest that we phrase the httpauthprep 
> text in terms of the username profile in 
> draft-ietf-precis-saslprepbis-06. To my mind, something like this 
> would work:
>
> ###
>
I agree with your proposal.

> simply reference draft-ietf-precis-saslprepbis.)
I agree. I don't see there is a reason to have a separate profile for 
HTTP passwords.

> Yutaka and Alexey (and others), let me know what you think about what 
> I have written here.

---

From: Yutaka OIWA <[email-masked]>
Date: Thu, 13 Mar 2014 09:54:29 +0900

Dear Peter,

I feel that I'm bit confusing about the situation and possible mistake.

As presented in a precis WG long before, and as talked with you with Julien,
my intended set for username in HTTP is almost "every printable string", so
"@yoiwa", "@@@@@", "Yutaka Oiwa",
"Y      O" must be all allowed, and
my understanding is that saslprepbis does not allow most of these.
In other words, http's name must be superset of sasl, and is never be a
proper subset of that.
Did I made some mistake on specification or interpretation of saslprepbis?

-- 
Yutaka OIWA <[email-masked]> on mobile
2014/03/08 23:49 "Peter Saint-Andre" <[email-masked]>:

> with the purpose of perhaps harmonizing the two approaches so that we can
> avoid multiplying PRECIS profiles beyond necessity.
>
> of one of more userparts, or (b) can be of the form [email-masked] ABNF definition is:
>
> * an "idpoint" is a UTF-8 encoded Unicode code point that conforms to the
> PRECIS "IdentifierClass"
> consists of one or more UTF-8 encoded Unicode code points that conform to
> the PRECIS "IdentifierClass". The ABNF definition is:
>
>    userpart   = 1*(idpoint)
>
> text (whether it appears in a standalone document or elsewhere) can simply
> state that for purposes of HTTP authentication a username is a userpart as
> PRECIS profile. Instead, I suggest that we phrase the httpauthprep text in
> terms of the username profile in draft-ietf-precis-saslprepbis-06. To my
> mind, something like this would work:
>
> ###
>
> For the purposes of HTTP authentication, a username conforms to the syntax
> definition and preparation methods specified in [I.D-draft-ietf-precis-saslprepbis],
> with the following limitations:
>
> password profile verbatim from draft-ietf-precis-saslprepbis. I think this
> text can be removed entirely, so that the httpauthprep text can simply

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 14 Mar 2014 19:55:21 -0600

On 03/12/2014 06:54 PM, Yutaka OIWA wrote:
> As presented in a precis WG long before, and as talked with you
> with Julien, my intended set for username in HTTP is almost "every
> printable string", so "@yoiwa", "@@@@@", "Yutaka Oiwa", "Y      O"
> must be all allowed, and my understanding is that saslprepbis does
> not allow most of these. In other words, http's name must be
> superset of sasl, and is never be a proper subset of that. Did I
> made some mistake on specification or interpretation of
> saslprepbis?

What you describe is not consistent wtih the definition in your
document, because your ABNF says that only characters from the
IdentifierClass are allowed (note, for example, SPACE is disallowed in
the IdentifierClass).

When you say "every printable string", does that include all symbol
characters, all punctuation characters, every kind of space character
(not just ASCII space), etc.? What about characters that are
disallowed even by the FreeformClass, such as Old Hangul Jamo
characters, control characters, and the ignorable characters described
in Section 7.13 of the PRECIS framework specification?

I think you need to revisit your httpauthprep document and more
clearly define the allowable characters for your use cases in terms of
the PRECIS framework. If you really want to allow "Y      O" (with six
spaces between 'Y' and 'O') as an HTTP username then even
draft-ietf-precis-nickname is too restrictive for you.

A wider question: why do you think that the definition of an HTTP
username needs to be so loose? Do you define things this way to be
backward compatible with existing implementations, or do you really
think that this is a best practice? I'm truly curious. (And I wonder
if we even want to call this construct a "username"...)

Peter

---

From: Yutaka OIWA <[email-masked]>
Date: Sat, 15 Mar 2014 11:48:26 +0900

Dear Peter and all PRECIS related members,

Thank you very much, and I'm really sorry that our definition
had a serious mistake which have confused many of you.
I'll talk with co-authors (especially, Nemoto-san) again and
revise the document as soon as possible,
to restart the next steps as fast as possible.

# That's why our specification has mentioned about
# the mapping for spaces, which are not included by current definition.

When I talked with Peter and Julian, we mentioned about
another character classes during the talk, and at that time
I should be noticed that I possibly referred the incorrect class.

> if we even want to call this construct a "username"...)
I think the reasons are both.

1: For backward compatibility, we need to keep all ASCII "printable"
    (U+0020 - U+007E, including SP) characters as is, as well as
    Latin-1 printable (U+00A1 - U+00FF, except SHY) be independent.

2: For more semantic reasons, HTTP authentication will be a vehicle
    for many different kinds of existing application frameworks, including
    IMs, Web mails, social network, and others.
    It should be able to accept all kinds of "user name" formats,
    for example a simple "user ID" (yoiwa), user "Name" (Yutaka OIWA),
    a mail address ([email-masked]),
    Social ID formats (@yoiwa or =yoiwa), and many others.
    Unlike SASL or XMPP which have its own semantics in framework,
    the authentication names in HTTP must be semanticless,
    unstructured strings, which can later be added a meaningful
    semantics for each application which uses Web/HTTP.

    We are not likely to correct all possible use cases of
    IDs which are to be used with HTTP (including future uses) and
    then take a union set of these,
    so instead we're defining a "grand-father" ID notations,
    expecting that all ID string use-cases are likely to be subsets of it.

    At the same time, defining it just a "UTF-8" makes users' confusion
    and inter-operability mess about possible "visiblly-same" strings,
    so we must care about that side of string preparation with PRECIS.
    For example, NBSP and other spaces should be replaced.

2014-03-15 10:55 GMT+09:00 Peter Saint-Andre <[email-masked]>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBAgAGBQJTI7MJAAoJEOoGpJErxa2pBu0P/3QCU4zx2i/smvPk5RJAlkYq
> 8b17rjXZfnS+K4zLS89FI7KtBsQjGglT7CTvoe4VJt9Xr62/xpMdgRIhfjOC8ZgA
> hAs0l7zbADArHgcA1AENBVUV0C/GGu5QPgJRgn/Mhnf51JxVgyu2ejG+iL0amr7I
> +eQ+0aEhefItYpXF3rhiB8c3hZK+YRrpj6yK4WUDeSzGUi6MPQyhq6WkfHX87gNj
> SsN0T26wIUq5asXKbTC7Yia1aABBYwp+DwMu4O46CkltRv3MYdX1sIAE4sljgzaE
> 837hQE10WTXgSJOQ/qyIHsIBddAItvwewzYNEqUjT1fo8de/hL2gMQ/KV+VTyBEi
> eq59I5wlvqBNlmGj8mOWtv1aVPk/0ANLHeOJVdgumppkYhRbvmSGfjZ5qsFLMDML
> hDxHeQdYb319KlNTrbRmcPLgBZnv5Mlk8T5p9/jUIkID4gk8lDEOVGHImI7ip/jK
> z0mgrCmbsWJtu12cKMHy34URtYVVN0pfdVGup/83/mK4Aq67Mb2iCnv6xWXsc61l
> 1OtGFh1KNY8fTkpabUPwFj5ZAMARI32EQNUbwF4A7kh4aktItqyq6IWhOq3qABlt
> Z6xl3gofvbOQ3LDrrqY/7XKxWNtz4WaGWa0b9SWe8zgopXxQ5W/QncmtMqkCnrG+
> aWDhAUslrrTuqHqw15mp
> =aaQD
> -----END PGP SIGNATURE-----

-- 
Yutaka OIWA, Ph.D.                 Leader, System Life-cycle Research Group
                               Research Institute for Secure Systems (RISEC)
     National Institute of Advanced Industrial Science and Technology (AIST)
                       Mail addresses: <[email-masked]>, <[email-masked]>
OpenPGP: id[440546B5] fp[7C9F 723A 7559 3246 229D  3139 8677 9BD2 4405 46B5]

---

From: Peter Saint-Andre <[email-masked]>
Date: Mon, 24 Mar 2014 21:36:11 -0600

>      Latin-1 printable (U+00A1 - U+00FF, except SHY) be independent.

I think "saslprepbis" does this.

>      Social ID formats (@yoiwa or =yoiwa),

Here again, I think that "saslprepbis" handles those.

> and many others.

Here we might have some disagreement.

If we think that http-auth needs to allow just about any string as a 
"userid" (to use the term from RFC 2617 - I don't want to call it a 
username), then even a PRECIS profile as loose as 
draft-ietf-precis-nickname is too strict for your purposes.

However, I question whether those purposes are justified.

Do we really expect that an actual userid in http-auth might be any of 
the following?

"Y                     u            taka  O   i    w     a      "
"♖♘♗♕♔♗♘♖" (i.e., the back line of white chess pieces)
"mycatisa	bby" (where those spaces are actually a tab)

and so on.

Do we have evidence from existing applications that we need to support 
strings of characters like those as userids in HTTP authentication? Or 
are we being way too liberal in what we accept?

>      expecting that all ID string use-cases are likely to be subsets of it.

My concern is that this "grandfather set" is every Unicode character, 
and if we allow that then we're really not providing any kind of helpful 
guidance to application developers.

One fundamental assumption underlying the PRECIS work is that it is our 
responsibility as internationalization experts to prevent application 
developers from shooting themselves in the foot. In particular, the 
IdentifierClass in the PRECIS framework tries to provide a safe subset 
of characters, and the username construct in "saslprepbis" profiles the 
IdentifierClass so that application developers can avoid trouble. As far 
as I can see, what you are proposing would invite such trouble, and I'm 
not comfortable with that.

>      For example, NBSP and other spaces should be replaced.

Hmm. So you are saying that if a user inputs the following string:

"Iam&nbsp;cey"

Then "&nbsp;" would be replaced with U+0020 (SPACE) as follows:

"Iam cey"

"Just UTF-8" is just as useless for internationalization as "just TLS" 
is for security. I definitely agree that we need something more than 
"just UTF-8", which is why we've put so much work into PRECIS. Although 
we cannot solve the problem of confusable characters, we can define some 
string classes that "first do no harm" (is there a Hippocratic Oath for 
i18n?). So far, I do not think that what you are proposing does no harm, 
in fact I think it is actively harmful to allow such a wide range of 
Unicode characters into userids.

Peter

---

From: Yutaka OIWA <[email-masked]>
Date: Tue, 25 Mar 2014 15:52:56 +0900

Dear Peter,

I have a side question: looking into saslprepbis spec,

   userpart   = 1*(idpoint)

Is the set of strings generated by "userpart '@' domainpart"
the proper subset of the set generated by userpart rule?
If so, I will be more-than-80% happy by just removing the
redundant second line from the username rule.

We (from HTTP side) do not want any rule which may
even *seem* to handle any characters (here, the @ mark)
specially than others. It's up to the application's own definition.
We're aware of some use case for which
the application defines a meaningful semantics
for strings with 2 @ marks.
(user@domain style identifier used for roaming accesses,
 appended with another @provider-domain suffix.
 So, the result is user@internal-domain@external-domain.)

(Exception: the colon is special in HTTP Basic and Digest,
 but I see it as a known technical issue of these and
 should not be generalized for every HTTP authentication.)

I have to (and will) dig in to the original Unicode table for
more analysis, but the thing I intuitively want seems to be
something between IdentifierClass and FreeFormClass.

> "Y                     u            taka  O   i    w     a      "

Current HTTP allows it, and we don't need to reject that mostly.
(and saslprepbis's username acutally allows it except the last spaces.)

> "♖♘♗♕♔♗♘♖" (i.e., the back line of white chess pieces)

I have a 50-50 feeling on this, little bit offseted to accept.
It's really weired, *I* will never use this, but it's still harmless, I think.
Someone may be using this as a funny identifier.
I think I have to convince HTTP people that it's harmful.

> "mycatisa       bby" (where those spaces are actually a tab)

I think this should be either rejected or replaced with a single space.

The support: most text input field of modern applications accepts
the top two, but for the last case some apps are rejecting TAB.
At least, almost all applications explicitly filter CR and LF out.

I think we can declare that "CR, LF, TAB, SHY and NBSP are harmful
so we should handle it as a part of I18N string handling".
It's the purpose of PRECIS, I agree.
(and that's why I am strongly opposing against "just UTF-8" in HTTPAUTH WG).

2014-03-25 12:36 GMT+09:00 Peter Saint-Andre <[email-masked]>:
> username), then even a PRECIS profile as loose as draft-ietf-precis-nickname
> is too strict for your purposes.
>
> Do we really expect that an actual userid in http-auth might be any of the
> following?
>
> "Y                     u            taka  O   i    w     a      "
> strings of characters like those as userids in HTTP authentication? Or are
> we being way too liberal in what we accept?
>
>
> My concern is that this "grandfather set" is every Unicode character, and if
> we allow that then we're really not providing any kind of helpful guidance
> to application developers.
>
> IdentifierClass in the PRECIS framework tries to provide a safe subset of
> characters, and the username construct in "saslprepbis" profiles the
> IdentifierClass so that application developers can avoid trouble. As far as
> I can see, what you are proposing would invite such trouble, and I'm not
> comfortable with that.
>
>
> "Just UTF-8" is just as useless for internationalization as "just TLS" is
> for security. I definitely agree that we need something more than "just
> UTF-8", which is why we've put so much work into PRECIS. Although we cannot
> solve the problem of confusable characters, we can define some string
> classes that "first do no harm" (is there a Hippocratic Oath for i18n?). So
> far, I do not think that what you are proposing does no harm, in fact I
> think it is actively harmful to allow such a wide range of Unicode
> characters into userids.
>
> Peter
>