Subject: RE: [Sipping] I-D ACTION:draft-fujimoto-sipping-header-lang-00.tx t

From: Adam Roach <[email-masked]>
Date: Mon, 23 Sep 2002 13:06:29 -0500

Jonathan Rosenberg wrote:
> I'm not entirely convinced that this is a real problem; the need is 
> really driven by things like speech to text when the same 
> characters are pronounced differently in different languages.

The text-to-speech problem is somewhat forward thinking, and it's not the
real issue right now. 

The real problem has to do with visually rendering characters in the CJK
range of Unicode. This problem exists today.

The issue, in a nutshell, is this: Chinese Hanzi, Japanese Kanji, and
Korean Hanja have been unified into a single range in Unicode. In
situations where the meaning is the same and the characters are similar,
Unicode has assigned the characters the same code.

Two problems arise from this situation:

 1) Part of it is aesthetic. Without cues about which language to use,
    programs are left to render Unicode CJK symbols however they decide
    to do so. This means that Japanese phrases may end up being a
    patchwork of Japanese and Chinese fonts, with visually horrible
    results.

 2) Beyond the aesthetics, there are real barriers to communication.
    Some unified characters are merely "similar", not "identical." This
    means that some readers may actually not be able to recognize the
    intended meaning. For example, the Unicode character U+76F4 is
    rendered roughly like this in Japanese fonts:

                             @@@
                             @@@@
                            @@@@
                            @@@                   @
                            @@                   @@@@
 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
                          @@
                         @@
                        @@
             @         @
             @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
             @@@                            @@@
             @@@                            @@@
 @@@@        @@@                            @@@
 @@@@        @@@                            @@@
  @@@        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
  @@@        @@@                            @@@
  @@@        @@@                            @@@
  @@@        @@@                            @@@
  @@@        @@@                            @@@
  @@@        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
  @@@        @@@                            @@@
  @@@        @@@                            @@@
  @@@        @@@                            @@@
  @@@        @@@                            @@@
  @@@        @@@                            @@@
  @@@        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
  @@@        @@@                            @@@
  @@@        @@@                            @
  @@@
  @@@                                              @@
  @@@                                             @@@@@
  @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
  @@

While Chinese fonts would usually draw it something like this:

                            @@@
                            @@@@@
                            @@@@
                            @@@@
                            @@@                     @@@
                            @@@                   @@@@@@@@
             @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
                           @@@
                           @@@@
                           @@@
              @@@          @@@             @@@@
              @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
              @@@@                         @@@@@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@
              @@@@                         @@@@        @@
              @@@@                         @@@@       @@@@
 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

I understand that many Japanese speakers won't recognize the second
character, while many Chinese speakers won't recognize the first.
(I can read neither, so I take these statements on faith from people
who can).

Language tagging allows applications to select the correct font
to render text in, which alleviates these sort of problems.

/a
_______________________________________________
Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping
This list is for NEW development of the application of SIP
Use [email-masked] for questions on current sip
Use [email-masked] for new developments of core SIP

---

From: Adam Roach <[email-masked]>
Date: Wed, 25 Sep 2002 18:48:53 -0500

> -----Original Message-----
> From: Eric Burger [mailto:[email-masked]]
> 
> I'm ambivalent on this one.  Whatever is more "standard" 
> would win for me.
> What does e-mail use?  That's what we should use, if possible.

That's not necessarily a useful question to ask in this context.
E-mail is restricted to using 7-bit characters. SIP uses UTF-8,
which gives us much more flexibility. In fact, it allows us to
leverage the Unicode work in this area.

That said, the MIME solution is *similar* to what has been proposed
in the the draft-fujimoto-sipping-header-lang-00 document, but
not the same.

/a

---

From: Adam Roach <[email-masked]>
Date: Mon, 30 Sep 2002 16:07:00 -0500

> I think "Unicode Language Tagging" is also possible alternative, if 
> all SIP implementation will support it as the mandatory 
> specification. But if some of SIP implementation doesn't support it, 
> that may cause problem, which described in Unicode Specification.

The current state of things right now:
 - No SIP implementations support Unicode Language Tagging.
 - No SIP implementations support your proposal.

[Both of these are educated guesses, but I feel rather confident
in claiming them].

So, for at least a transitional period (and probably forever),
there will be some subset of implementations that support no
form of language tagging.

Consequences of sending a Unicode Language Tagged message to
a client that does not understand it: It renders the unknown
character codes as "garbage" (many font rendering implementations
use something like question marks or small squares for this
purpose).

Consequences of sending a message per your proposal to a client
that does not understand it: the language tagging shows up
as odd sequences of character that, to the casual user,
look like garbage. Further, any representation of characters
that must be escaped looks equally funky to users.

I don't see where one approach has much of an advantage over
the other in this particular area.

/a