Subject: Re: Hashing, and Ted's concerns about wastefulness

From: Graham Klyne <[email-masked]>
Date: Wed, 03 Feb 1999 14:11:00 +0000

At 15:16 02/02/99 -0800, Bill Newman wrote:
>Ted wrote (quoting Graham)
>
>> Yes, but we need to specify exactly how that is represented.  Remember
>> that most hashes on files care about every bit of whitespace and every
>> control character; if we don't specify the representation very well,
>> we will end up with conflicting hashes on equivalent feature expressions.
>
>How bad a problem is this? Is it is worth trying to assign the same
>hash values to differently-represented-but-ultimately-equivalent
>feature sets? It is a feature, but it adds complexity to the
>specification and to each implementation.

Good question.

I don't have a clear answer, but I *suspect* that
"differently-represented-but-ultimately-equivalent feature sets" won't be a
big issue.  I have an unsubstantiated view that automatically recognizing
the same feature set from different authors will not be a big requirement.

>Do you anticipate a pattern of usage where this feature would 
>significantly increase efficiency?

Personally, no.

Also, even if we did not require the hash to be computed over a fully
canonicalized value, a combination of whitespace normalization and
presentation conventions might still match a good number of separately
authored cases.

One thing I do strongly believe:  this matter does not impact correct
functioning of the system (to the extent that appropriate dereferencing
mechanisms exist);  rather, it reates to possible efficiency of operation.

#g

------------
Graham Klyne
([email-masked])

---

From: Ted Hardie <[email-masked]>
Date: Wed, 3 Feb 1999 13:32:42 -0800 (PST)

> presentation conventions might still match a good number of separately

Remember that typical hash functions care about control characters, so
we inherit the whole CR vs. CRLF kinds of problems.  Normalizing it now
is worth the time, not so much for the "seperately authored" case, but
so that the internal representations used to generate the hashes are
consistent across platforms with different conventions.  

				regards,
					Ted

---

From: Al Gilman <[email-masked]>
Date: Wed, 03 Feb 1999 22:06:25 -0500

>consistent across platforms with different conventions.  

If you can wait a while (time measured in months, not years), the work
coming forward as character model and InfoSet for XML will give you a good
foundation so we can have international tags and you can still normalize
text if you wish prior to hashing.

I am not sure we are going to agonize ourselves through the character set
problems any faster than they.

Al