Subject: [precis] order of operations (was: Re: Milestones changed for precis WG)

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 4 May 2016 14:46:47 -0600

On 3/28/16 1:58 PM, Christian Schudt wrote:
> Hi,
>
> let me raise one concern, which I had while implementing RFC 7564 and
> 7613. I’ve raised it already on this mailing list, but it got no
> attention.

Sorry about that. Attention cycles come and go. Now that I'm working on 
the bis drafts, I for one will make it a point to reply more quickly.

> Instead of changing RFC 7564 §7 from MUST to SHOULD, why not changing
> RFC 7613 §3.2.2 (and likely other Enforcement sections) to stick to
> the order of rules?

Now that I look at RFC 7613 again more closely, I would say that it 
doesn't really violate the order of operations in §7 of RFC 7564, 
because for purposes of comparison it still applies the width-mapping 
rule before all the other rules. However, it's also true that RFC 7613 
applies the width-mapping rule before the "preparation" (or, perhaps 
better, "pre-processing") steps of (a) ensuring that the string consists 
only of Unicode code points that conform to the PRECIS IdentifierClass 
and (b) encoding the string as UTF-8. That doesn't violate the letter of 
RFC 7564 but perhaps it's confusing for implementers.

Note that the purpose of the order of operations is "to ensure proper 
comparison"; I'm not convinced that what happens during preparation or 
pre-processing needs to follow the same order of operations.

There's also the question of whether preparation / pre-processing is all 
that useful. We added it so that constrained clients (e.g., clients that 
can't realistically perform normalization) could avoid most of the 
problems associated with having their strings rejected by a more 
powerful server that actually does enforcement and comparison. Whether 
we really need to consider such applications is another story.

> For me as an implementer it simply felt more clean and
> straightforward, e.g. when I faced this issue:
>
> It’s still unclear what to do with U+212B (ANGSTROM SIGN) in
> Usernames.

For better thread tracking, I'll reply separately to your original message.

Peter

---

From: Sam Whited <[email-masked]>
Date: Wed, 4 May 2016 17:26:47 -0500

On Wed, May 4, 2016 at 3:46 PM, Peter Saint-Andre <[email-masked]> wrote:
> can't realistically perform normalization) could avoid most of the problems
> associated with having their strings rejected by a more powerful server that
> actually does enforcement and comparison. Whether we really need to consider
> such applications is another story.

I'd be curious to know a "real world" use case for the preparation
step optimization where enforcement is prohibitively expensive and
comparison is not necessary (since comparison necessitates
enforcement)? While benchmarking the implementation I did for Go I
decided that it wasn't necessary to include a preparation step. Even
on a heavily resource constrained server the enforcement step was easy
to run concurrently or in parallel, and was very fast, even in its
unoptimized form (most of the time spent in the algorithm was on
memory allocation, which could easily be reduced). You can't count on
concurrency or parallelism in all systems of course, but since it also
wasn't all that time complex my suspicion is that the preparation step
is premature optimization even on synchronous systems (although I
haven't actually tested this).

—Sam

-- 
Sam Whited
pub 4096R/54083AE104EA7AD3
https://blog.samwhited.com

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 4 May 2016 16:52:19 -0600

On 5/4/16 4:26 PM, Sam Whited wrote:
> enforcement)?

The original suggestion (some years ago now) was browser-based clients 
that could not easily download the entire Unicode character database 
from a web server to the browser in order to do enforcement or 
comparison, but which could do something simpler like allow a user to 
create a username using a restricted repertoire of characters such as 
ASCII or "extended Latin".

Another example might be IoT clients running on microcontrollers with 
very limited memory and processor speed.

However, I have not seen benchmarks for such applications with respect 
to handling of internationalized strings.

> While benchmarking the implementation I did for Go I
> on a heavily resource constrained server

Right, we've always figured that servers have more horsepower.

Peter

---

From: Sam Whited <[email-masked]>
Date: Wed, 4 May 2016 20:47:33 -0500

> The original suggestion (some years ago now) was browser-based clients
that could not easily download the entire Unicode character database from a
web server to the browser in order to do enforcement or comparison,

That makes sense; the trie I use to store the derived properties for all of
Unicode 8.0.0 is 23 KB in memory (it would be a bit bigger over the wire;
though there's also room for improvement there). This seems reasonable at
first glance, however, a similar structure for normalization ends up being
53 KB, and the one for width mapping is 11 KB; I suppose it begins to add
up quickly if you're on a slow connection

I suppose it comes down to a trade off between complexity and portability;
I'm not sure which makes the most sense in this case.

Best,

Sam

---

From: Peter Saint-Andre <[email-masked]>
Date: Thu, 5 May 2016 10:57:54 -0600

On 5/4/16 7:47 PM, Sam Whited wrote:
> On May 4, 2016 17:52, "Peter Saint-Andre" <[email-masked]
> <mailto:[email-masked]>> wrote:
>  > The original suggestion (some years ago now) was browser-based
> clients that could not easily download the entire Unicode character
> database from a web server to the browser in order to do enforcement or
> comparison,
>
> That makes sense; the trie I use to store the derived properties for all
> of Unicode 8.0.0 is 23 KB in memory (it would be a bit bigger over the
> wire; though there's also room for improvement there). This seems
> reasonable at first glance, however, a similar structure for
> normalization ends up being 53 KB, and the one for width mapping is 11
> KB; I suppose it begins to add up quickly if you're on a slow connection

That's an interesting approach. The person who originally brought this 
up (I think it was Joe Hildebrand) might have been thinking that a web 
client would need to import the entire UCD from a web server upon 
initial connection, which would be prohibitive.

Peter

---

From: Martin J. Dürst <[email-masked]>
Date: Fri, 6 May 2016 15:13:21 +0900

>> KB; I suppose it begins to add up quickly if you're on a slow connection

Just for the record, I can definitely confirm the ballpark range of 
these figures, based on implementations of mine.

Regards,   Martin.