Subject: [iucg] key questions for the IDNA2010-2012 works

From: JFC Morfin <[email-masked]>
Date: Fri, 19 Feb 2010 00:00:12 +0100

This exchange with Patrik Fälström certainly benefit from his wide 
experience of the matter. It should be compared with the "Mapping" 
document the AD hesitates to forward to the IESG.
Best
jfc

NB. Reminder. Due to the wrong usage ccNSO made of the term 
"IDNA2010" (as a replacement for IDNA2010) we have registered the 
idna2010.org and idna2012.org in order to stop that confusion. 
IDNA2008 is the IDNA building block on the Internet side, IDNA2010 is 
understood as the way to implement IDNA2008 on the user/zone manager 
side. IDNA2012 is the background to permit the joint administration 
of IDNA2008 and IDNA2010. These terms may not be the best, but they 
impress the main point: that IDNA2008 is here to stay.

At 16:33 09/02/2010, Patrik Fältström wrote:
> > At 23:15 08/02/2010, Patrik Fältström wrote:
> >> Part from standing behind what Vint wrote in his response, let me answer
> >> more directly the questions from Lisa.
> >>
> >> I am a developer/designer of mostly web based software for management of
> >> domain names, web sites, email services and what not. I will 
> (and want to be
> >> able to) in my interfaces first of all be pretty sure over what data comes
> >> to me as a "server". I.e. I want to know what the end user 
> really wanted to
> >> give me (I did explicitly not say "typed" here for reasons I hope people
> >> understand).
> >
> > I understand that you are on the server end. You are related to a user
> > and you do not know if this is a direct relation, or it goes through a
> > bowser, an iPhone, a Client, a User Agent. Correct? Yet, you also consider
> > the case where you may have to develop the User Client/Agent and 
> you control
> > things down to the user's fingers?
>
>My point is that what people historically believe is "the user 
>interface" sometimes is a split relationship between what today is a 
>client and a server, where the human have access to the client, and 
>the access to the server is from the client via some protocol like HTTP.
>
>I.e. we have often said that potential mappings should be done in 
>the user interface, and I agree with this, but I wanted to give one 
>example where it technically is very hard to do what is the user 
>interface. IF there is a mandate on some mapping, what software 
>should do it? What software will be compliant, and what software 
>will not be compliant?
>
>It is not as easy to answer those questions as some people might think.

Correct. This is where I identify there are four possible IDNA user 
localisations:

1. in the user interface to the client
2. in the user client.
3. as a unique IDNApplication that serve the client
4. as a part of an IUI (Internet Use Interface) in between the client 
and the plug to the network where the server is plugged.

1 & 2 are OK for testing, not for user operations. There are as many 
experiences and possible different resolutions as there are 
applications or clients.

3. is more secure, can have a control panel, support local config 
files. But user applications must know that service application and 
how to dialog with it.

Only 4 permit localization issues to be locally centraly managed and 
networkization issues dealt through it with the ISP in order to 
permit a semantic relation with the server cleared from the context 
and identical with every existing protocols (the IUI looks to them as 
a pseudo-network they can ignore). You may have scores of 
applications using IDN without knowing it if they are UTF transparent.

> >> Now, if the web browser (if the web browser was used) did some trick and
> >> converted some characters, then I do not get the characters the end user
> >> typed (for example), but regardless of this, I will do whatever 
> I want to do
> >> to make the end user happy. Exactly what that implies will 
> differ depending
> >> on context. Is this domain name management or management of a web site, or
> >> the management of email addresses or sites?
> >
> > So you mean that the reference is what the user wants to achieve?
>
>To some degree, yes.
>
>A user should not be surprised. But then what is surprised?

We agree. And this is where I say that the IUI must be intelligent 
enough to be able to make a difference between localization and 
personalization issues.
Also, it means that there might actually be different Ulevels 
depending on personal, local, etc. mapping (i.e. for example a simple 
equivalent to Host.txt being supported)

This is why I refer to a Multi-Layer Domain Name System pile with
- what the user actually types,
- which can be different from what the client will accept
- that can be different from the real Ulevel
- and then the actual Alabel

All of them must strictly polynym (synonym in their context) for the 
user (this may change from users to users).
This can be very common if the resolution may also be made outside of 
the DNS in other environments also using the IDN (IRI, semantic 
addressing, etc.)
This is why I prefer to refer to IDNs as "Internet Domain Names" with 
a single simple syntax that covers DN and IntlDNs alike.
This permits to consider that IntlDNs only are internet domain name 
of the "xn" presentation. The IUI having the job of sorting 
presentations and classes.

Presentations are simply the two letters before double dash (IDNA 
uses the xn-- "extended names" class). Classes other DNSupported pay 
loads can be introduced by single letter + --.

1. the Internet presentation layer emerges from this, with years of 
validation behind by i-DNs and IDNA2003
2. I have a universal syntax available to support presentations and 
any semantic. To make it universal I consider that Internet Domain 
Names syntax is alphadigital labels and sub-labels with "." (or 
something else) as intelabel, and "-" or (something else) as 
intersublabel. Double dash corresponds to an empty sublabel that 
separates metadata payload from naming payload.

> > And
> > your understanding of what he wants to achieve will depend on different
> > context inputs, and obviously of the protocol being used and the task to be
> > performed?
>
>Potentially, yes.

Agree.

> > Also from the zone?
>
>No. And the reason is because of the following:
>
>To be able to give context dependent responses (such as language, 
>username or whatever), the selection mechanism must get information 
>about the context.
>And that is not possible to do in the DNS protocol.

Yes you can.
I can decide that .fr do not support French majuscules because it 
never had. And that .fra does, because it is designed to be semanticaly clear.

>So some layers in the architecture must be exactly the same, all 
>over the world, for all users, or else it will be impossible to get 
>the base interoperability that is needed to build the systems people wants.

Correct. This is why the architecture needs to be more precise and 
support the needed flexibility up to the point IDNA2008 defines. This 
is this way that IDNA2008 defines multiplicity. It is the ultimate 
required point where architecture must be the same, all over the 
world, for all users. As such it defines the boarder of the IETF 
scope. This is why I identify:

- the Internet system, the IETF scope of responsibility : IDNA2008
- the Internet exotem, the user area that wants to use IDNA2008, 
documenting it is what I call IDNA2010 in order ICANN does not 
continue to use the word wrongly
- the Internet peritem, the fringe between the Internet system and 
the user's Internet exotem. Documenting the way this peritem 
information must be presented is what I call IDNA2012.

As Project.FRA I have a need. I know that to solve it I need to get 
my users to treat .fra differently from .fr.
How can I do that?

Very simply: in having my users using "punyplus" as an algorithm 
rather than "punycode".
When they find a TLD or an upper level domain (can be several level) 
("ULD", also a "User Level Domain") listed in their "netlocale" their 
IUI is going to localize it as indicated in the netlocale file.

The same, if they need something special (for example kidprotection 
filtering) the punyplus algorithm can be used to filter as documented 
in a "personale" file.
Punyplus and the netlocale file formats and possible management 
system need to be documented.

My suggestion is that IETF can use:

- MUSTs for its system
- SHOULDs for the peritem fringe
- MAY for the user exotem, like Mapping does.

>Another example, the HTTP and HTML protocols are the same all over 
>the place, but HTTP include a negotiation phase regarding screen 
>size, type of browser etc that makes it possible for an HTTP server 
>to give back different data depending on such information.
>
>HTTP is not different. HTML is not different.

Absolutely. This kind of functions (server front end and IUI can be 
performed by OPES - the HTTP OPES has been specified).

Now, the problem you raise that it is not possible to do it through 
the DNS protocol is more a question of capacity if there are millions 
of TLDs to filter, update, etc. This is true on the Internet network 
side. This is not on a user side.

If instead of a unique unique root file, you consider a unique 
virtual root file that people can partly implement through their 
"personale" file. They just maintain the TLDs they are interested in. 
There are 25.000 sociolinguistic entities: I am able to read or grasp 
just a tiny few. I wish to filter out what I do not want (and not 
what Unicode wants, hence the dispute about RFC 4646).

> >> Is this mapping, if applied, to be implemented in the browser or in the
> >> web server? I would like to get as raw data as possible, without 
> having the
> >> browser doing anything, so that I know as much as possible on the server
> >> side. So I can "help" users the same way browsers today "help" users by
> >> adding ".COM" TLD if that is not added. "guessing" I think it 
> can be called.
> >
> > I understand this is in the case where you have to develop the user
> > client interface?
>
>Sometimes yes, sometimes no. If you for example on a web server use 
>Wordpress software, you as a user of the site do not develop the 
>software. Are you developing the user client interface? I do not 
>know. I would claim I am not doing that. Do the one creating the 
>wordpress theme do that? I do not know.
>
>It is blurry.
>
>But I want to trust all of those developers "to do the right thing".

You cannot. First a whole because some are breakers, phishers, pirates, etc.

This is why I tend to say, what the user feel as the "network" MUST 
be what his machine perceives as the network (this can include 
non-Internet networks).
In order to achieve this, I consider that "user side network 
applications" must be considered as forming a dedicated network 
layer, where they appear as "smart local operating tasks" (slots) to 
clearly identfy the from their brother at networking the Internet 
network applications (Mail, DNS, etc.).

They locally operate tasks the user consider as parts of his "smart 
Internet system", whatever it may really be. The user will access 
them through the browser or some dedicated local interface. Hence I 
call the slot layer "pseudo-network" as it is seen by the user as an 
intelligent network shell.

Now, I put that slot layer in the IUI, so there is a clean overlay :

- through the IUI the network expand on the User side.
- through the IUI the user controls the fringe of the network. This 
permits to differentiate :
-- dump traffic (passive content) from end to end
-- smart traffic (ambient and active content) from IUI to IUI

Then I observe such slots have much in common and that it could be 
nice for them to interoperate with other local slot servicing them, 
or with remote fellow slots to support metatraffic. This is why I add 
interapplication layer - between the Internet network application 
layer and the IUI network application layer.

Then I obseve that there can be advantages in locating network 
applications in the Internet (global range network applications) and 
in the IUI (local range network applications).

Let take the example of the DNS. If I put the DNS in the IUI it is 
both part of the Internet DNS and dedicated to my 
machine/small-home-network (SnHn) or duplicated within my own 
Internet context (cybship: all the digital resources I control). To 
be sure there is no pollution I turn recursion down, making it 
secure.This means that now I can use my own local root, with just the 
TLD/ULDs I am interested in from the unique virtual root classes (I 
can add my own ULDs as China does with its own IDNgTLDs).

> >> And if HTTP/Web browser is not in use, but instead for example an iPhone
> >> application that then interact over JSON/Netstring/TCP with some 
> server API,
> >> then I can in a more direct way control what is sent to the server (if I
> >> also write the client). I will only be confused (possibly) over 
> the mapping
> >> that is in the operating system / user interface libraries.
> >
> > OK. Did you think of a method for you to clarify that possible confusion?
>
>Only by giving me the ability to implement mapping or not depending 
>on what "makes sense" for me, and my application.

OK. This is where I say there is an urgent need for Internet 
Adminance (i.e. technically managing the internet in cooperation, so 
all this can be documented by who is actually responsible). But this 
calls for flexibility, simple and robust rules, etc. This is why I 
say the IUI needs an interapplication system that permits network 
applications. Naturally called it "Netix". And I can easily support 
it in using the ML-DNS syntax.

This way Netix in the IUI makes a complete interoperable addition to 
the Internet basis able to match the user's expectation for an 
intelligent open network of his own understanding, using its own 
classes and presentations, ML-DNS pile, interslot authentication

>I am VERY nervous over having any kind of mandatory mappings. 
>Mandatory for whom? Who must implement it?

Sure !!! But having a mandatory syntax to express a free mapping 
semantic may help. Moreover it can be multilingual and users can 
enter parameters or use scripts.

> >> All of this might sounds like if it will make things messier for the
> >> user, but in reality I do not think so. *NO* developer have the 
> interest in
> >> making this messier for the user. And exactly what it means to make it
> >> easier for the user depends on context. Very much. Specifically 
> when we have
> >> so many protocols and environments and contexts where domain names are in
> >> use.
> >
> > As I said, I am not that much competent in Web level applications, rather
> >
> >
> > with user level interapplications. Which documentation do you use to
> > identify how you should understand what to do with a given protocol, in a
> > given environment, for a given context?
>
>I will use whatever I can find. Often I am looking at what is done 
>in local language communities. At the moment IANA Language tables.

Correct.
This is why I need all this and much more to be federated in a 
structed open repository that applications can directly access. A 
local customized IANA.

I call it the Multilingual Distributed Referential System (MDRS). It 
should be ISO 11179 compatible to interoperate with many other registries.

As you say, most is documented on a local language community basis. 
This is why I documented the ccTAG concept with people from ISO, 
MAAYA, Linguasphere, etc. They are just a reference grid, a 
continuation of ISO 3166 (we use to call ISO 3166-4). ISO 3166 
includes for each country the administrative (i.e. normative) 
languages and scripts using ISO 639 and 15924 codes. So ccTAGs 
proceed in the logical categorization order: country, script, 
language, region - supported by the core document of international 
consistencuy, including Internet and ccTLDs.

The technical interest is that being polylingual (French/English) and 
listing the Govs, i.e. the standardization authorities, it can be 
understood as the French and English instantiations of a single 
multilingual standard with equal validity in every normative language 
(what we call "polynym", synonym in different context).

Due to the universal acceptance of ISO 3166 it continues the Internet 
intrastructure as it is since 1978. Very stable, accepted, intuitive. 
This means that I can tag every application linguistic instantiation 
in a congruent manner with its administrative forms, sales contracts, 
vendor rates and support, local regulations, etc.

Also, that through ISO 11179 compatibility the MDRS can be 
indefinitly expanded as a metastructure, ready to support the 
Intersem (semiotic/semantic internet) needs.

> >> Saying just "no mapping at registration time" is not good enough for me.
> >> What about if you want to change the holder or adminc or techc 
> of a domain?
> >> Should you do mapping then, if the domain name is typed by the 
> user or not?
> >> Is a transfer of a domain name "registration time"?
> >
> > Good point. What if the Registry sotfware does not consider
> > holder/adminc/techc and uses a different system?
>
>They all do. They might use different terminology.

All ccTLDs do not adhere to the ICANN WHOIS concept which actually is 
illegal but tollerated in most of the countries (violation with local 
privacy laws). Zone Managers can be at any level and be sure that 
many ULD managers will not even know that ICANN exists or what it 
could have to do with their Private Class TLD. ICANN is only for class IN.

> > What is the Registry only use local script/language interfaces?
>
>Many only support interfaces in local script and local language. I 
>do not see any problem with that.

Sorry, I meant at application layer, not at user layer.

Internationalization means that local language is embedded in English.
Multilingualization means that local language replaces English.

This is the conflict I had with Unicode because their Globalization 
is internationalization+localization+flangtag-filtering. In the fussy 
process we identified the existence of multilinguistics as the 
discipline of the cyberntics of the linguistic diversity. How 
languages interelates among other languages and/or cultural entitities.

> >> No, I rather see that I as a developer can do whatever I want to make
> >> things easier for the users, and that might include that I try to do some
> >> mappings of codepoints that are not PVALID, or complete changes of some
> >> things that might be domain names (hard to know sometimes) to 
> something that
> >> are domain names (different separators between labels for example).
> >
> > and French/Latin language majuscule support?
>
>I can not be more precise than what I was above.
>
> >> So to answer your question Lisa: No, it is not clear what characters are
> >> suggested to be mapped, but we do not even know where or who has the
> >> responsibility to do the mapping -- if any. In the case of HTTP, is that
> >> done in the web browser or on the server side, in the cgi?
> >
> > I have some difficulty here: how can an IDN be mapped on the server side?
> > Should it not necessarily be mapped on the user side so it can be properly
> > resolved to that very server?
>
>"server" as "server side" in HTTP. Not server side as in DNS server.

Yes? I still miss it. I might need an map.

> >> For example how do a web browser know whether the ajax / json call is to
> >> be "a registration" or not? A situation when mapping should explicitly not
> >> occur? It can not, and because of that browsers should not include any
> >> mapping at all. Ever.
> >> That is one view, and of course you can find the arguments the other way
> >> around as well.
> >
> > I agree with your position. Would you know of an online
> > document/discussion on this matter, or a con/pro list?
>
>I think the IDNA mailing list archive is a good source... :-)

OK :-)
But I thought about something more formal and organised.
I compiled all the WG/LC remarks in a Draft. Indicating which had 
been answered. I lost the count of the non-answered ones.
http://www.ietf.org/id/draft-iucg-wgidnabislc-01.txt
(or better reading: http://wikidna.org/index.php?title=WG-IDNA_LC)

> >> And this is why I think the current mapping document is as good as we
> >> can get it -- and why I think (like Pete) TR46 is too strong. 
> Because it is
> >> very wrong to explicitly ask for mappings the way it does.
> >
> > Do you know if other possibilities have been expressed by W3C, or SDOs?
>
>TR46 is one of the documents. I know ITU and UNESCO also are doing 
>work, but they have been waiting for the RFCs to be published. ISOC 
>might come out with something. ICANN might write recommendations. Etc.

Dont you think a common work could be usefull ? Users are not 
interested in consensus, but in multiconsensus, i.e. a consensus in 
each of the position groups, and a general consensus on how each 
position bridges with the others. Beging in using a single 
terminology and possible comparison tables.

> >> What *might* be ok and good to do is to say what codepoints can be
> >> mapped in various contexts to other characters, but I think for 
> example the
> >> definition of bundles in the IANA registries to some degree do that. When
> >> not mapping from PVALID to other PVALID of course. So that 
> people know that
> >> IF one work in a specific context (certain language for example), THEN it
> >> MIGHT be interesting to map from A to B.
> >
> > Do you consider the possibility of mapping on a TLD basis?
>
>No.
>I think the user wants mapping based on other criteria.

For example Project.FRA wants majuscule support based on .fra while 
.fr may npt mind or address the need differently.

>But of course the TLDs already today via the language tables do 
>implement in some cases bundling and various policy based issues.

I am interested in what I call mecalanguages. i.e. natural languages 
as to be spoken by machines. The risk is that they become the 
referents and that languages rigidifies or split (machine/human). But 
the risk is, now on an interactive basis, equivalent to dictionnaries 
and grammars. The mecalanguages must be able to adapt and support 
dolwn to avalects (languages of each individual avatar's styles) 
(hundreds of billions). The only serious non-physical personal 
authentication is based on avalects as it betrays the cerebral status.

Now, the challenge is to work on the invarients, i.e. the semantic 
traits or atoms, that can then be passed as metadata (as one passes 
Alabels) and restored through different mecalanguages. This gives the 
idea of Ulabels that could display differently depending on the context.

This is certainly possible as part of the ML-DNS (multilayer domain 
names pile system - the minimum of it being the Alabel/Ulabel pile. 
Domain Names cannot refer to invariant, but semantic addressing can 
do it while using the DNS as a support. This is one of the targets of 
the Project.FRA in using its namespace as a taxonomy for an open ontology.

> >> So I would say the mapping document we have is good.
> >
> > How would you qualify it? a reminder, a protection against too stringent
> > or lax behaviours, a guidance, a reference, a open link to further work?
> > Is its informational nature not a problem while it completes a standard
> > track work?
>
>We do not know yet. Let people try, and we see. Having guidelines is 
>ok, and probably what developers need.
>But mandatory rules, nope.

Who in your opinion should work these guidleines after the experience 
of this WG, i.e. working relations with Unicode, IUCG, i-DNs, ICANN, 
ITU, etc. ?

We have a MAAYA Unesco meeting on 23/24 to prepare a book on 
multilingualism in cyberspace. I think we should start with some kind 
of ontology of multinguinstics and cyberlinguism, i.e. the way 
languages behaves together and the way cyberspace affects languages. 
But we will know better when we know what others may mean when we use 
the same words as us.

This is why I say that now we have built the Internet side IDNA, one 
has to consider the User side and the intrastructural side 
(guidelines, tables, information, registries, etc. ): where and how 
the developper information should be stored and found.

jfc