Subject: Re: Charset stuff (Re: DNS SRV for Kerberos)

From: Terry Lambert <[email-masked]>
Date: Tue, 30 Nov 1999 12:21:51 -0800

Masataka Ohta wrote:
> > This is a rather silly statement.
> 
> Any statement on Unicode, including mine or yours, is
> silly ass, of course.
> 
> But, that is not my fault.

Please put aside your personal feelings for Unicode,
and address the issue on technical merits like the
professional you are.

> > No, I am not talking about _any_ multibyte encoding;
> > as you point out, UTF-16 has the same problems as
> > UTF-8, because it is a multibyte encoding.
> 
> Wrong.
> 
> The question is whether applications know character boundary
> or not.

I respectfully submit that sizeof(u_int_16) == 2, and
that the boundary can be known the same way any 16 or
32 bit value's boundary is known.

The only possible place this can fail is a non-error
checked 8-bit link.

For an error checked 8-bit link, or an non-error checked
16 bit link, this is not a problem.

> Everything works fine if an application know some code
> use UTF-8 style character boundary or another code use
> "singlebyte encoding" (that is, "character boundary" ==
> "byte boundary").

Everything _also_ works fine if data streams are treated
as 16 bits wide instead of 8 bits wide.

Your complaint assumes, incorrectly, that I will be
sending my data as octets, not hextets.

> > Anticipating a complaint about byte order in 16 bit
> > values, you can do one of the two following:
> 
> So, your point was 16bit byte fallacy. Sorry to have
> mistaken it as UTF-16 fallacy.

Please explain the "16bit byte fallacy".  I do not
understand why, when I:

	wchar_t		buf[ 128];	/* 16 bit wchar_t*/
	wchar_t		ch;

	...

	ch = buf[ 7];

that "ch" does not contain character 8 from the buffer,
or how in the heck that "ch" could straddle a boundary
between 16 bit characters.

I do not see the problem you are so worried about.

How can the above code malfunction?

-- Terry Lambert
-- Whistle Communications, Inc., an I.B.M. Company
-- [email-masked]
-------------------------------------------------------------------
This is formal notice under California Assembly Bill 1629, enacted
9/26/98 that any UCE sent to my email address will be billed $50
per incident to the legally allowed maximum of $25,000.

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 30 Nov 1999 13:18:17 +0100

At 11:40 29.11.99 +0900, Masataka Ohta wrote:

>Your opsys will display characters, but not bytes, and you cut 2, but
>not 1.5, characters of 4, but not 3, bytes.
>
>The forth character may be systactically important one.

Red herring.

if you truncate data randomly, some data will be syntactically meaningless.
All multibyte encodings (including shift-jis) give you a new way to be 
syntactically meaningless if you truncate on random byte boundaries.

Random truncation of data will get you into trouble no matter what the 
encoding; titech.ac.j is not a registered DNS domain name.

                              Harald T. Alvestrand
--
Harald Tveit Alvestrand, EDB Maxware, Norway
[email-masked]

---

From: Eric Brunner <[email-masked]>
Date: Thu, 02 Dec 1999 07:35:16 -0500

I'd vastly prefer a discussion of Martin's July '98 I-D to the one now
being undertaken.

Cheers,
Eric