Subject: Re: [iucg] [Json] Json and U+08A1 and related cases

From: JFC Morfin <[email-masked]>
Date: Sun, 25 Jan 2015 03:50:39 +0100

At 15:44 24/01/2015, Vint Cerf wrote:
>I do wonder whether it is worth considering an attempt to create a 
>new set of properties of UNICODED characters that are of specific 
>use to the DNS. The IDNA 2008 work tried to use properties of 
>characters developed for purposes other than the DNS and the fit is 
>not always perfect.

We are discussing a three steps multi-layer digital name system
(ML-DNS) to be used by multiple applications and network technologies
on the.catenet:

1. I enter a string A in my local querry system which punycodes it
     into string B so it may be used by the next step. (RFC 4895)
2. The string B is entered in the Domain Name Service of the concerned
     CLASS to get an IP address.
3. the string B is restored as A' in the destination system.

The target is that A'=A.

For your information.

As part of the ICP-3 conformant IUser community testing for the
Catenet we want to experiment the following multi-ledger/multi-echnology
approach (ML-DNS)::

1. for the CLASS "IN" ledger of registries of ICANN/NTIA/Verisign:

     the step 1/step 3 back and forth conversion algorithm is:
     - either " A=B/B=A when A and B are restricted  to  the ASCII list".
     - or punycode otherwise.

2. for the private CLASS "FL" (Free/Libre) ledger of registries by DNSLIB:

     the step 1/step 3 back and forth conversion algorithm is:
     - either " A=B/B=A when A and B are restricted  to  the ASCII list".
     - or punycode applied to the UNISIGN semiotic  (Free/Libre) sign set.
     - or no registration permitted/filtering out otherwise.

UNISIGN only includes non-confusable character and non character
signs. Confusable characters share a unique common sign-point.

The UNISIGN table results from a non-confusability algorithm under
financing/work, It should be based upon rastering comparison in an
UNISIGN fount of reference.

jfc

---

From: John C Klensin <[email-masked]>
Date: Sun, 25 Jan 2015 08:49:52 -0500

(address list trimmed to IDNA and IUCG lists, with apologies to
the former, but this one seems to need a response)

--On Sunday, January 25, 2015 03:50 +0100 JFC Morfin
<[email-masked]> wrote:

> (ML-DNS) to be used by multiple applications and network
> technologies on the.catenet:
> 

I either don't understand this or don't see why it is either
necessary or desirable:

If the operation of (1), which I will restate for clarity as
conversion of String A to an A-label as specified by IDNA2008,
produces a String B that can be found in the public DNS (which
means QCLASS=IN for all known Internet applications these days),
then the above is either unnecessary or identical to what
IDNA2008 requires.  Note, in particular, that the dual
relationship between A-labels and U-labels guarantees that A==A'.

Conversely, if you want String A to be something that cannot be
represented directly in the DNS, at least under IDNA rules (or,
given some of your observations about Majuscules, etc., perhaps
even in Unicode), then it seems to me that you should be doing
something that we've discussed on and off for well over a
decade, specifically:

(i) Look A up in some non-DNS database whose matching algorithms
conform to your needs, yielding a string C.  One important
property of that database should be that actual lookups of name
by value (as well as value by name) should be feasible.  FWIW,
note that such lookups were anticipated by the original DNS
design but dropped (and replaced by, e.g., a separate reverse
mapping tree for addresses) when they turned out to be
incompatible with other aspects of that design.

(ii) For maximum interoperability with the rest of the Internet,
look up C normally in the DNS, CLASS-IN, yielding addresses of
other records important to you.

(iii) When and as needed, go back to your database to map C to A
(A' if you like but, if there is only one table and it is used
for both forward and reverse mappings, the identity relationship
is guaranteed).

Interestingly, in the model above, there is no need for C to be
in the encoding produced by the Punycode algorithm, to use the
IDNA "xn--" prefix, or to obey any conventions other than you
own.  You would actually gain some simplicity and resistance
against attacks and the sort of possibly-incorrect matching
issues that have been the topic of this thread (and many others
over the years) by making C a pseudo-random, all-ASCII, value
rather than relying on a trick encoding.   The only requirement
on C is that it be unique, and there are lots of ways to
accomplish that.

Also see your Case (2) below.

>      - either " A=B/B=A when A and B are restricted  to  the

See above.  If A and B are both ASCII, those relationships are
guaranteed by the basic DNS design modulo case-independent
matching and non-preservation of case in a variety of contexts
where compression is involved.  If you need case preservation in
the ASCII range, you either cannot use the DNS or have to use a
trick encoding for everything, including all-ASCII labels.
Whatever that encoding might be, the Punycode algorithm, which
makes some special assumptions about all-ASCII labels, won't be
appropriate.

>      - or no registration permitted/filtering out otherwise.

First, note that the Punycode algorithm was designed
specifically for the Unicode repertoire and block-layout system.
It is an optimization for that system that, in particular, has
better (shorter-string) properties than UTF-8 for strings of
Unicode code points with particular characteristics, notably
when there are short distances among the numerical values of the
code points in the label string and many or all of those code
points have sufficiently high numeric values to require
three-octet encodings.

> signs. Confusable characters share a unique common sign-point.

If you are going to use a repertoire and/or coding system
different from Unicode, then the Punycode algorithm is very
unlikely to make any sense.  If you can use the DNS for your own
purposes and encoding, intend to use a separate CLASS, and can
control the coding so that it meets your needs, then there are
probably no advantages of using anything other than your native
coding.  Remember that the DNS is perfectly happy with labels
consisting of octets with arbitrary values and compares them
perfectly and consistently.  The difficulties that led to IDNA
arise from three things: (a) What now seems like a peculiarity
that octets whose values are consistent with letters in the
ASCII range are compared case-independently but nothing else is
(b) Within CLASS=IN, ASCII is basically assumed for octets in
the range 0x00 to 0x7F; for octets outside that range, the
character repertoire and encoding are unspecified and those
octets are not assumed to be character data at all.  For other
CLASSes, you could easily specify a CLASS-wide character set and
encoding and design that encoding to not trigger any DNS
idiosyncrasies. 
(c) Many applications limit their identifiers and associated DNS
labels to ASCII-only.  IDNA was designed to avoid having to
upgrade all of those applications as a condition for deploying
IDNs.    Because introducing a new coding system or lookups in a
different database or different DNS CLASS requires upgrading all
relevant applications anyway, you don't have that constraint.

> financing/work, It should be based upon rastering comparison
> in an UNISIGN fount of reference.

Good luck with that and do let us know, ideally in the
peer-reviewed literature, how it works out.  It may work if you
can select (and perhaps design) a single type family and
eliminate all others from your enviornment, ideally eliminating
all stylistic variations (like italics or bold for many
Latin-based type families) within that family as well.  That
would leave you in a situation in which characters that were
confusing in your reference font might not be in more normal
ones and vice versa.  It would, of course, work even better if
you restrict the scripts of interest.  Otherwise, for example,
the visual (and likely raster and even vector) similarity
between U+03BB and U+5165, a resemblance that I think we can be
quite confident is entirely coincidental and one where even a
few characters of context would likely be adequate to make a
distinction, would result in a single code point in your system.
I think I've mentioned this to you before, but anyone
contemplating automatic confusability detection based on raster
or vector properties of printed/written characters should
examine the pre-Kurzweil  (mid-to-late 1960s) literature in
optical character recognition as a starting point in
understanding how difficult it is to get from abstractions of
characters to character identification.

---

From: "J-F C. Morfin" <[email-masked]>
Date: Thu, 29 Jan 2015 19:56:31 +0100

Dear John,
I am sorry for the dealy in responding to this. Things are changing 
with some ambitious cooperative project demanding a lot of 
French/Local involvment. This helps maintaing the health.

>I either don't understand this or don't see why it is either 

I understand you do not understand: we do not want to address the same need.

Let me digress
===========

I want a robust basic response to the "if it looks the same, it is to 
be the same" need. You want the best DNS system that can use Unicode.

Your limit is perfect logic. My limit is reality. This is a common 
scientific difference between effective (me) and fundamental (you) 
thinking. Both are correct. The only thing is that yours introduces 
at least two spaces [rules and data] within the same considerations 
(so it is often more complex and more demanding).

This is why I prefer to split them differently, considering 
fundamental as architectonics and effective as operationalism. This 
way I avoid (this is a general consideration of the Universe) to 
answer initial questions on the reality of the time that Einstein and 
many others before and further on have made to disapear. In a 
nutshell, I live in a single universe, rather than discussing a 
multiverse: this allows me to consider a rough multi-internet in a 
single universe, rather than a single perfect internet in each 
universe of a multiverse.

I certainly think that networking brings basic metaphores to 
cosmology, but no need for now :-)

Specific of this case
===============

The difference is who is the master: the machine (theoretical and 
logic, in a local context) or the person (practical and "agoric", 
i.e.in an holistic global context).

- UNICODE is a codepoint based system, i.e. the reference is what 
UNICODE has decided to be a character.

- UNISIGN is a semiotic signpoint based system, i.e. what a common 
reader decides he/she sees/feels. We try to reduce it to UNIGRAPH, 
i.e. what the machine AND all the readers (statistics) are supposed to accept.

Your system makes sure that A -> A-label -> B=A. when there  re 
computers with the same software at both ends.

Reality does not insure that A =H= B =H= A  if =H= means "equals" by 
mutiple/different-humans reading.

This work story
===========

Remember when we started this WG I said I would support its work as 
long as there would be no MUST on the end to end path, and I would 
document a middle layer IDNA2008 based ML-DNS solution further on.

There was some layer confusion but the RFC 5895 proposition addressed 
them, permitting a consensus, introducing the subsidiarity principle 
in order to embed INDA2008 into real-life diversities (except for 
orthotypography and French Majuscule like issues).

IMHO, this was a major advancement in the overall Internet 
architectural design. But it was permitted by the RFC 5895 unlocking, 
the IESG did not accept as a part of the WG work; and the French 
majuscules generic issue was not addressed (we would not face a 
problem today if it had been).

As for IDNA2003, time was necessary to observe the field results of 
IDNA2008 as an Unicode end to end "IDNS". What I observe (my 
diagnosis) is that fringe to fringe ML-DNS is necessary and either 
calls for a non-confusability algorithm or for a Unicode replacement 
(either a complex one as UNISIGN (the whole semiotic), or more rustic 
one as UNIGRAPH (only the character sets)).

The new context of this debate
=======================

Now, your points are made with CLASS="IN" in mind and in a 
pre-20150108 context 
(http://www.ietf.org/blog/2015/01/taking-a-step-towards-iana-transition/).

Mine's are those of a member of:
- an RFC 6852 global - non-NTIA dependent - community
- which is in unformal standard technical innovation coopetition with the IETF
- which respect the ICP-3 recommendations and only consider 
CLASS="FL" (Free/Libre), [for the time being as a private CLASS].

So, I am not anymore discussing IETF considerations for a global 
standard to be documented by http://iana.arpa. IETF leaders decided 
they did not want to assume this responsibility. I am just 
contributing on behalf of a young and loose Libre Community WG about 
what we consider in an multioperable context (shared ISP switchers - 
compatibility with our own UNINAME need, i.e. for a member directory).

>incompatible with other aspects of that design.

If you consider strings we are back to the preceding problem: strings 
are chains of characters. The difficulty is with the man/machine 
possible disagreement over characters identification.

>other records important to you.

At this time CLASS-IN would be establishing too much habits. This is 
testing, and ICP-3 is reasonably clear and prudent.

>for both forward and reverse mappings, the identity relationship

This is precisely what we do not want. This means that I would need 
***three*** separate non-ASCII DNS systems to be maintained in parallel!

>on C is that it be unique, and there are lots of ways to

Correct. But this is not human oriented anymore. So why not just to 
use an IPv6 address?

>\›ØXÚ
>@L-DNS)::
>‚F†R7FW÷7FW2&0k and forth conversion algorithm is:
>where compression is involved.

Yes. But remember I want to protect majuscules/other metafancies so I 
must differentiate them (usual way is upper cases, but 
orthotypography has others).

>If you need case preservation in

Right. Actually I must consider ASCII as possible IDNs (as in Elysée.fr)

>makes some special assumptions about all-ASCII labels, won't be

No. See above. For example I can decide that ^e = E, so Elysee.fr is 
^elysee.fr.

>‹ˆ›ÜˆHš]˜]HÀLASS "FL" (Free/Libre) ledger of
> > registries by DNSLIB:
> >
>HZ]\ˆˆOP‹ÐPHÚ[ˆH[™ˆ\™H™\Àtricted  to  the
>TÐÒRH\Ý‹‚‚Ò÷"Vç6öFRÀied to the UNISIGN semiotic
> > (Free/Libre) sign set.

My priority is something that effectively works. If a solution works 
somewhere RFC 1958 advises to use it elsewere. The idea is to use 
xu-- domain names where punnycode handles pseudo-Unicode that are 
coded/uncoded at both fringes.

>purposes and encoding, intend to use a separate CLASS,

Yes. This is the premise.

>and can
>coding.

No, because I do not plan replacing, but adding on top. This is a 
practical issue (at least as long as we are smaller than Google+USG 
:-))  Remember: we have a need and the need for the solution to be 
stable. This need is common with Banks, Police, Corporations, etc. We 
have no problem in transcoding further on to any "INTERNATIONALCODE".

Our problem is simply that people do not discriminate at the machine 
level as Unicode supports it. Unicode is too good for us.

>Remember that the DNS is perfectly happy with labels
>arise from three things:

>(a) What now seems like a peculiarity

Yes. We need to support that since you made IDNA reduce upper-cases 
to lower-cases during the IDNA DNS process.

>encoding and design that encoding to not trigger any DNS

This is the very basis of the "FL" CLASS. Again, this is ICP-3 
recommendation. I do not want to innovate and take risks. Just to get 
out of the 1986 status-quo, fully respecting the stabilized RFC 
(including the now completed RFC 6852). This is the new 
"permissionless innovation" motto.

>relevant applications anyway, you don't have that constraint.

No. It only requires people like Firefox to accept that the users are 
grown enough and can decide to send to the DNS resolver what they 
actually entered, and the DNS resolution to be handled at the "IUI" 
or "MYCANN Plug-in" level, where entries are read and massaged 
appropriately (this is RFC 5895).

>Latin-based type families) within that family as well.

I do not need that. The fount is virtual (i.e. only in establishing 
the graphic average confusability. Depending on the environments it 
can be Arial, use, or the law). Please remember it is just deciding 
that roman o and cyrillic o are a graphic o and giving the receiving 
end the semiotics metadata to use it (language, script, 
orthotypography). Why do you think I made sure langtags would not be 
confusable?

>That

Yes. No phishing.

>characters to character identification.

There is not automatic confusability detection. There is only a table 
based transcoding algorithm, between UNICODE and UNIGRAPH in the 
UNISIGN framework (there are not only character signs). It can be 
established manually, but it will probably be easier/cheaper and more 
operationally oriented to prepare it with published/legal existing 
rasters with the advantage to produce raster printing as well for 
easy international identifications (ex. Customs, forms, signings, 
etc.). The point is not to know what Chinese use to confuse but what 
people from anywhere may confuse in Chinese.

What pre/post=Kurzwell may have writen on raster confusability is 
certainly be of interest. We will leave that to experts. Our need is 
just to start a Libre momentum and protect the UNISIGN and UNIGRAPH 
terms so all this does not land into a closed commercial consortium 
but in a DECLIC approach (development for Catenet Libre, 
Institutional and Commercial use or whatever better English wording 
you can find for the French "DEstiné au Catenet/Libre Institutionnel 
Commercial" (DECLIC) license we work on.

jfc