Subject: Re: [iucg] referencing IDNA2008 (and IDNA2003?)

From: jean-michel bernier de portzamparc <[email-masked]>
Date: Fri, 22 Oct 2010 14:37:35 +0200

2010/10/22 J-F C. Morfin <[email-masked]>

> IMHO in the cookie case, there are three types of cookies:
>
> - those documented as part of the Internet technology, subject to standard
> track document, that can only document their use within the Internet Iron
> Curtain (i.e. lowercase A-labels)
> - those being used by IUI level applications that are to be documented by
> the Internet Users community that can be documented through BCP or for
> information like RFC 5895.
> - those being used by the users applications that are documented by their
> developers.
>
> The IU community has many needs and solutions that can be supported in
> respecting the now existing and confirmed Iron Curtain separation. Let try
> to not erode it and create instability.
>

I just wish to add, for Peter to best understand, that "U-labels" as
conceived by the IUCommunity  may be transformed into lowercase A-label with
many different "presentation prefixes" (other than "xn--") and from many non
U-labels origins (like user gesture, computed result, audio input,
sentences, Netix commands, etc.) that may only be occasionnally used on the
Internet. The ML-DNS implies a domain name stack (different classes) that
may be used as their class IN lowercase A-Label occurence.

In the future we will strive to document a UN class that will support a non
homographically confused NUCS (network universal character set) UCS subset
for ML-DNS unique registrations (avoiding confusion like .br and .6r) on a
first come first served basis.

Last but no least we plan to fully use not only the identification
(centralized management), but also designation (individual management) and
communication (mutual management) capacities. Therefore, only the Internet
DNS (i.e. lowercase A-Label IN and further UN classes as accepted by the
Internet community (presently the IEISOCANN enhanced cooperation IANA root
file)) is considered as being in the IETF standard track area.

Best
Portzamparc

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 22 Oct 2010 10:20:03 -0400

On Fri, Oct 22, 2010 at 03:30:45PM +0200, J-F C. Morfin wrote:

> better we stay in our respective areas. And Internet IETF RFCs stay  
> within the boundaries of lowercase A-Labels.

There is no guarantee that A-labels will never have capital letters in
them, is there?  RFC 5891 says this in section 3.1, number 2:
"A-labels MUST be compared as case-insensitive ASCII (as with all
comparisons of ASCII DNS labels)."

A-labels are, to an IDNA-ignorant application, just plain old DNS
labels that happen to conform to the LDH hostname convention.  They
can be in either case, AFAIK.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Vint Cerf <[email-masked]>
Date: Fri, 22 Oct 2010 10:29:19 -0400

andrew,

we were pretty explicit that the algorithm that produces A-labels
produces only lower case. check with John Klensin.

v

On Fri, Oct 22, 2010 at 10:20 AM, Andrew Sullivan <[email-masked]> wrote:
> _______________________________________________
> Idna-update mailing list
> [email-masked]
> http://www.alvestrand.no/mailman/listinfo/idna-update
>

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 22 Oct 2010 10:48:55 -0400

> produces only lower case. check with John Klensin.

I remember talking about it, and I remember this being an issue
because Punycode does not actually require lower case.  But I can't
put my fingers on the text where it says this right now.  I haven't
looked that hard, however.

The reason I haven't looked hard is that it doesn't matter.  There is
absolutely no way we can enforce any restriction in the DNS that
requires the label to remain lower case.  Though DNS is supposed to be
ASCII-case-preserving but ASCII-case-insensitive, the plain fact is
that not every implementation does this, or does it correctly.  (I
recall quite clearly pointing this out during the WG discussions,
because some implementations use compression pointers to the original
query string and therefore get whatever was asked by an application.)
Applications can put their LDH queries in _in any case at all_ and
have them work.  An IDNA2008-unaware stack with an IDNA2008-aware
application above might do anything, including converting everything
in the label to upper case (try logging into an old-fashioned UNIX
console with the caps lock on.  You'll do a lookup for XN--SOMETHING
no matter what you intend).  If it does that, and happens to query
through a caching name server, the upper case form will persist in the
cache.  You still have to treat that label as matching the lower case
U-label.  We couldn't do all this above the DNS if you didn't have to.

The case-preserving, case-insensitive feature of DNS was, in my
opinion, a grave error.  But it's an error we have to live with
forever if we're going to continue using DNS.  You simply cannot build
a case-sensitive layer atop the DNS if any of the US-ASCII code points
in the labels you want to use are themselves to be case sensitive.  If
you want to do something clever like Punycode, only for the entire
Unicode range (i.e. including that which overlaps with US-ASCII) so
that you never have a transparent map between the DNS name and the
user-presented name, then you have the possibility of introducing case
sensitivity to the naming system (but not the DNS).  Otherwise, you're
out of luck.

A

-- 
Andrew Sullivan
[email-masked]
Shinkuro, Inc.

---

From: Adam Barth <[email-masked]>
Date: Fri, 22 Oct 2010 11:38:20 -0700

I'm not sure I quite understood everything that was mentioned during
this thread.  Concretely, here's what cookies need to do:

1) We receive a sequence of octets from the network, which we convert
to lower case.  Let's call this X.
2) We have a URL that the user agent has used to generate an HTTP
request.  Let's call the host name component of this URL Y.
3) X is a sequence of octets that's has all the crazy xn-- stuff.
4) We need to transform Y to the crazy xn-- form to see if it's in
some relation to X.
5) For our sanity, we'd like to use octet-by-octet comparison, without
reference to any kind of folding (e.g., case-folding, IDNA-folding,
etc).

As an added constraint, we don't feel its our place to mandate to user
agents whether they ought to use IDNA2003 or IDNA2008.  User agents
are free to make the decision independently of what the cookie spec
says.  You should feel free to lobby them one way or another, but
we're not going to impose that requirement on them in this document.

My understanding is that the text in the spec meets our requirements.
If that's not the case, please let me know.

Adam

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 22 Oct 2010 12:59:56 -0600

On 10/22/10 12:38 PM, Adam Barth wrote:
> this thread. 

Don't worry, it takes a long time to grok i18n, let alone grok it in
fullness. I for one feel like I'm always standing at the foot of the
mountain and not making any progress toward the summit...

> Concretely, here's what cookies need to do:
> 

Yep, it's an A-label.

Possible trap: is the domain-value provided by the HTTP server in the
Set-Cookie header really an A-label (if the domain is an IDN)?

> reference to any kind of folding (e.g., case-folding, IDNA-folding,

One conclusion from this thread is that it will come to you already
folded (lowercase, in simplified terms) because that's the way an
A-label is.

> we're not going to impose that requirement on them in this document.

Right -- the user agent should use whatever IDNA technology it already
has, although we hope that they'll migrate to IDNA2008 (but that hope is
out of scope for the cookie spec = draft-ietf-httpstate-cookie).

> If that's not the case, please let me know.

Here's where our understandings diverge. :)

My reading
<http://www.alvestrand.no/pipermail/idna-update/2010-October/006770.html> of
this thread is that in the cookie spec it would be best to proceed as
follows...

1. Expunge the concept of a "canonicalized host name" and thus Section
5.1.2.

2. In Section 5.3, wherever we say this:

      If the domain-attribute is identical to the canonicalized
      request-host:

   Change it to something like this:

      If the domain-attribute and the request-host are identical
      according to the domain comparison rules of the implementation,
      including comparison of internationalized domain names (IDNs)
      using either IDNA2003 or IDNA2008:

   Similarly change this:

      Set the cookie's domain to the canonicalized request-host.

   To:

      Set the cookie's domain to the request-host.

   (I don't see a strong need to "canonicalize" it before storing it.)

   Etc.

3. Remove Section 6.3 entirely.

Adam, do you read this thread differently?

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Vint Cerf <[email-masked]>
Date: Fri, 22 Oct 2010 15:21:13 -0400

be aware the IDNA2008 speaks to "labels" so you may receive a sequence
of labels. I think we have agreed that before processing of the domain
name, the separators need to have been rendered in ASCII 2E ("period")
form.

If the label is entirely ASCII in the range A-Z, a-z, 0-9 and "-" and
does not begin with "xn--" or "XN--" you have what sounds like a
standard form NR-LDH (non-reserved Letter/Digit/Hyphen) label.
Comparison of list label with potential matches can be done with case
insensitive DNS method.

If the prefix is "xn--" (or even "XN--" ??), you may have an A-label
but this needs to be checked for successful conversion to U-label (and
back?).

Is the process by which you receive input labels also subject to
random Unicodes (in 8-bit, 16-bit or 32-bit forms)? If so, the process
gets more complex because these strings should not be directly
compared but, in my opinion anyway, converted into A-labels, if
possible. If that fails, the object isn't a valid label.

It's more complex than I have said, I think, depending on the form and
context in which candidate labels appear.

vint

---

From: Adam Barth <[email-masked]>
Date: Fri, 22 Oct 2010 12:17:05 -0700

> Set-Cookie header really an A-label (if the domain is an IDN)?

I believe so.  Or rather, I think they'll need to be to work properly
today.  We could run some experiments to be sure, but I was told that
putting non-ASCII characters in HTTP headers is bad news bears.

> folded (lowercase, in simplified terms) because that's the way an

Great.

> Here's where our understandings diverge. :)

Ok.  :)

>      using either IDNA2003 or IDNA2008:

This text seems symmetric.  The user agent needs to know that it
should not apply the U-label => A-label conversion to the
domain-attribute but that it should apply the conversions to the
request-host.

>   (I don't see a strong need to "canonicalize" it before storing it.)

We need to canonicalize it before storing to make the rest of the
storage model work out properly.  In particular, we count the number
of cookies that share common host names, etc.  Rather than having to
do IDNA-folding comparisons everywhere, we canonicalize and then do
simple octet-comparisons.

> Adam, do you read this thread differently?

That approach doesn't meet requirement (5).  In particular, this text
uses IDNA-folding comparisons instead of first canonicalizing and then
applying octet-by-octet comparisons.

Adam

---

From: Adam Barth <[email-masked]>
Date: Fri, 22 Oct 2010 12:26:21 -0700

It doesn't matter if X (described below) is a valid label.  If it's
not an octet-by-octet match for something that we know is well-formed,
we just ignore the whole cookie.

Adam

---

From: Adam Barth <[email-masked]>
Date: Fri, 22 Oct 2010 14:00:41 -0700

On Fri, Oct 22, 2010 at 1:46 PM, J-F C. Morfin <[email-masked]> wrote:
> At 20:38 22/10/2010, Adam Barth wrote:
> To make sure we all understand well I think it would be nice to be sure we
> talk of the same things in documenting:
>
> - who is "we"

When I saw "we", I mean the httpstate working group.

> - what does mean "receive from the network"

More precisely, "receive as part of an HTTP response"

> - what is the user agent

The user agent is a concept from HTTP.  It refers to a class of HTTP
clients, namely those with users (as opposed to network
intermediaries).

> - where do we have the URL from ? (in reference for example to the RFC 5895
> special case).

That's inessential.  Sometime in the past, the user agent generated an
HTTP request for a particular URL.

> - X is a lowercase A-label

Not necessarily.  X is a sequence of octets.

> Suggestion: to be sure you cannot be mistaken, do not consider U-Label as
> Unicoded Labels you could match one way or another, but as User-Labels which
> can be many different things ranging from ASCII label to a converted snap of
> user fingers as Portzamparc hinted it. You will _never_ know which UTS 36,
> RFC 5895, etc. etc. process has been used to convert U-labels into lowercase
> A-labels. Just remember that at Project.FRA we do not know yet (we want to
> experiment first with people from many other languages) how we will support
> plain French language "U-labels" as far as majuscules are concerned (same
> for other Latin languages). Being a majuscule as impact on the words meaning
> and this is one of the metadata information Unicode does not support.

This paragraph was entirely mysterious to me.

> I understand that you want to "specify the syntax and semantics
> of these headers as they are actually used _on_ the Internet." IMHO
> you follow the same thinking as IDNA2003, while IDNA2008 should make you
> say "_through_ the Internet". You want to document "on the
> rope" something now documented as being "off the rope"
> (still networked but controlled by the user). You meet the same problem
> as the WG/IDNABIS met. A problem of IS/MUST/SHOULD/MIGHT that demands
> that you clarify the scope of the communication process you want to
> stabilize to understand where you can use MUST, SHOULD and MIGHT - and if
> IS/ARE have to be considered that are imposed on you.

This paragraph is also entirely mysterious to me.  Where did the rope come from?

> IMHO to be off universal good value cookies should only be Internet
> documented semi-stable data containers (therefore relying only on inner
> Internet controlled elements such as A-labels).

We're not taking a position about whether something is or is not a
universal good.  We're taking a position as to what is useful for
folks who would like to build HTTP user agents.

> You cannot know all the different ways people will use it. But you must
> tell them that if they want to use it in your way, they MUST follow that
> you say. And, because you write a MUST you MUST be in control of the
> feasibility of what you demand. On an "end to end" basis you
> only control the "inner Internet" (within the Iron Curtain) DNS
> entry/output, i.e. the lowercase A-label. And you can also trust it: they
> are the xn-- labels Registries have actually registered, under IDNA2003
> and IDNA2008.

I have very little power to enforce my will upon others.  I believe
the Iron Curtain was lifted near the end of the cold war, so I'm not
overly concerned with what network infrastructure they might have used
that far in the past.

> By nature a robust architecture is synergetic. This is synergetics.

These statement are mysterious to me.

> Seen that way, your cookies makes things simple and the whole net simpler
> and more robust (RFC 3439) and inscrease the robustness of the whole
> internet.

I'm glad you hold that opinion.  From my perspective, cookies are
neither simple nor to the make the whole Internet more robust.
However, they're what we've got.

Adam

---

From: Adam Barth <[email-masked]>
Date: Fri, 22 Oct 2010 20:22:27 -0700

I'm sad that you've invented your own language for describing things
that other folks call by other names.  That makes it difficult to
understand what you're saying.  However, it appears we are in
agreement.

Adam

On Fri, Oct 22, 2010 at 5:35 PM, J-F C. Morfin <[email-masked]> wrote:
> At 23:00 22/10/2010, Adam Barth wrote:
>>
> Dear Adam,
> As I explained IDNA2008 has built a separation between the Internet
> standards as documented by the IETF (RFC 5890 to 5894) and the Internet Use
> standards as initiated by "unusual" RFC 5895 (*). We (small IU Community
> kernel) need words to describe the result.
>
> We have get accustomed to call this separation "Iron Curtain" for many
> reason. And to call "rope" the network wire plus its "off the rope"
> extensions outside of the "Iron Curtain" through what we call the IUI
> (Internet Use Interface) We also dubb the IUI as the "no-mans land" because
> that part is for extended network use processes, not for users or user
> agents and because a proposition is about "smart mines" protection in that
> area.
>
> We found the wording better than "iron wall" initially proposed by analogy
> to "fire wall", and well received by people having to understand the
> IDNA2008 architectural implications
>
> (*) This is well expressed in this key RFC 5895 part: "It should be noted
> that this document does not specify the behavior of a protocol that appears
> "on the wire". It describes an operation that is to be applied to user input
> in order to prepare that user input for use in an "on the network" protocol.
> As _unusual_ as this may be for a document concerning Internet protocols, it
> is necessary to describe this operation for implementors who may have
> designed around the original IDNA protocol (herein referred to as IDNA2003),
> which conflates this user-input operation into the protocol.
>
>> > - who is "we"
> Got it. I thought you were talking about the processes having to deal with
> the protocol.
>
> Through an OPES or not (RFC 4236) ? One way to implement the IUI is to used
> OPES or ONES (networked OPES).
>
>> > - what is the user agent
> I understand. But there are many possibilities to implement IDNA2008 on the
> user side. They may use their own DNS-agents or OPES-agents, on an user
> individual basis. Or networked.
>
> You implied that this URL was not in lowercase A-label form. Now or in the
> past it is essential to know where did the system get it, to understand its
> nature and format.
>
> OK, then you have changed your schema. If X is a sequence of octets and Y is
> of unknown format, I have some difficulty to understand how IDNA2008 can
> help.
>
> It may not be obvious if you do not dissociate Internet and Internet Use.
> Try to compare with telephone. Telephone works for people from every culture
> and languages. Yet if you want to build a phone service to the users you
> need to know more than sequences of octets and probably one of their 25.000
> languages. Here it is the same. IDNA2008 has relieved the Internet side of
> the Iron Curtain from the worry of languages, scripts and languages+scripts
> (orthotypography).
>
> You will not be interoperable/compatible with the Internet IDNA2008 if you
> reintroduce them.
>
> I introduced the origin of the word rope. The difference from the wire is
> that it is optional, depending from the user side architecture being chosen
> or planned (at this time, investigated/tested). Some users may want to get
> IDNA implemented in the way the IETF imaginated it: a straight wire is OK
> between servers/user agents.
> Others may want something more secure with a core DNS agent, supporting HTTP
> or other protocols user agents.
> Others may want more related services.
>
>> > IMHO to be of universal good value cookies should only be Internet
> Your cookies are not limited to HTTP. As such their universal value can be
> good if they bring an open plus to the network architecture, and bad if they
> limit what could be done.
>
> MUST, SHOULD etc. have a meaning in the Internet standard process. In the
> new reading of the Internet architecture IDNA2008 has exemplified, it turns
> out that the IETF area seems to be where IETF can say MUST, and that in the
> Internet Use area the IETF can at most say SHOULD. This is a good way to
> know where what you consider may stand.
>
> Richard Buckminster Fuller. RFC 3439 establishes that the internet has to be
> simple. Synergetics as a scientific discipline has somewhat helped
> understanding simplicity the way thevery large  Internet system needs it.
>
> Dont get confused about what I said. I did not say cookies are simple, but
> that your cookies (i.e. the definition of them you try to build) should help
> making things simple and the net simpler. They could compare with the nano
> buckies (again from Bucky Fuller). At least this is the way we consider them
> and explore them - as remote datacontainers, some of us say "information
> bubbles".
>
> I just tried to help from my own exploration of similar issues in
> multilinguisation context. I think now I am done. I will certainly spend
> time on your draft as it is a construed document capitalizing on the input
> of a WG.
> jfc
>
>
>> Adam
>>
>>

---

From: jean-michel bernier de portzamparc <[email-masked]>
Date: Sun, 24 Oct 2010 04:05:20 +0200

2010/10/24 Mark Davis ☕ <[email-masked]>

> I'm in agreement about the usefulness of storing the punycode form. As to
> what you would like to see, Patrik, I'm in agreement there as well; that the
> goal is IDNA2008. And I think we'll get there eventually, when the major
> registries disallow the registrations of non-IDNA2008 names.

Dear Mark,

whatever the policy of the "registries", their transitions, their interest
in Unicode, their commercial, cultural or political strategies, etc.  they
only use A-labels as far as the Internet and the Internet DNS are concerned
(them having "xn--" headers or not - remember that until IDNA2003 every
cooky was A-label only). These A-label having been initially registered as
IDNA2003, IDNA2008 or xn-ascii does not make any difference. Cookies are not
interested in the origin of the domain name, but in the value of the domain
names. Every IDN has one and only one lowercase A-label value. And this
value is here to stay.

Considering anything else for cookies, is to reintroduce the confusion that
IDNA2008 clarified.
Remember the sensible ".su" position: they will not register U-labels, but
only A-label whatever the reverse punycodability.

The problem for implementers is not there. The problem is to obtain a local
user authoritative A-label, something the AD was told not to ask but IAB
will have to document.

Portzamparc

---

From: jean-michel bernier de portzamparc <[email-masked]>
Date: Sun, 24 Oct 2010 04:18:56 +0200

JFC, Adam,

something Adam may easily understand: there are now two ways of reading all
the RFCs. One is IDNA2003 like (diversity is loosely handled inside the IETF
technology) and one is IDNA2008 (diversity is handled outside the IETF
technology in using it in a very strict IETF decided common way). These two
ways consider the _same_ internet (not a single bit being changed in the
code) in a very _different_ maner. If we used the same language it would
mean we would use the same terms in a rather idfferent or even opposite
manners. We would definitly not understand oneanother anymore.

Cheers !
Portzamparc

2010/10/24 J-F C. Morfin <[email-masked]>

> At 05:22 23/10/2010, Adam Barth wrote:
>
> do not be sad. I am happy we are in agreement.
>
> IDNA2008 has acknowledged an evolution of the world digital ecosystem
> network (WDEN) functions assumed by the Internet. We can compare this to the
> Copernican evolution, with the Newtonian revolution equivalent under way. It
> would certainly be strange if the vocabulary of the Ptolemeean side/period
> did fit all of our brave new world needs. It is better (and well  in the
> Internet way) to start using images before coining appropriate sophisticated
> terms.
>
> Never mind, as I underlined it, we will still have to bring the Internet
> up-to-date with the early XXth scientific paradigm. In a certain way p2p,
> cookies, etc. are just doing that: entering the world of the general
> relativity. This is why I say your work is important. The internet strata,
> over the telecom strata, is the content strata. Up to now the Internet deals
> mostly with static content (end to end). Cookies (memory stocking in the
> fringe) permit to some extent to deal with ambient content (a content
> related to some kind of location) as p2p may relate to diffuse content
> concepts. More generally the IUI, with OPES Smart Mines, or ONES or
> Interplus, permits to investigate active content (what you receive is what I
> wanted you to receive but I did not send) supported at fringe to fringe
> level.
>
> Once that stage proven, my current work to try to experiment it without
> creating a conceptual mess, the technology will start being able to support
> its next "end XXth/early XXIst" scientific paradigm and the Intersem
> (semiotic as in Kinect). The Internet of thoughts. Through semantic add-ons
> as a possibly the synergetic evolution of mathematic (the science of
> patterns and structure) and engineering. Until IDNA2008 it was difficult to
> forecast when this could practically happen and how. Because it was
> intellectually blocked by a too limited and conservative reading of RFCs.
>
> Since IDNA2008 (various current examples shows it) it is possible to say
> "it is now". Yet this will call an entire newly coordinated (networked)
> approach of the network concepts and notions (starting with a new network
> topology: centralized, decentralized, distributed, intricated). At least
> epistemologists and philosophers gave us a name for this endeavor:
> "diktyology", the science of networks. The internet is a very interesting,
> living, global diktyologic phase of the "bit-bang" we can all observe and
> influence (RFC 3935 states something like this: IETF is to influence those
> that design, use and manage the Internet so it works better). Time has come
> to consider what "better" may mean and how to achieve it.
>
> However, beware. Just as IAB felt necessary to correct my reference to
> "diversity" as if it only was a reference to "[linguistic] diversity", it
> would be better if you discussed "cookies" and not just "[http] cookies".
>
> Best
> jfc
>
>
>
>

---

From: jean-michel bernier de portzamparc <[email-masked]>
Date: Sun, 24 Oct 2010 21:21:46 +0200

> > These A-label having been initially registered as IDNA2003, IDNA2008 or
> xn-ascii does not make any difference.
>
> There is a bit of a problem in terminology. I used the term "punycode
> label" to include labels of the form "xn--...", where the ... is valid
> Punycode representations (in ASCII) of a Unicode string (with non-ASCII
> characters).
>

This seems to be an IDNA2003 and IDNA2008 well acceptable definition?

> If we are speaking precisely, the term "A-Label" is defined in IDNA2008,
> and is more restrictive. It does not include all the punycode labels that
> are valid in IDNA2003. So "http://xn--1-wpn.blogspot.com/" (= http://€
> 1.blogspot.com/) does not have an A-Label in it, but is a punycode label,
> and is valid in IDNA2003.
>

This is also my understanding. But I understand it is only more restrictive
because the IDNA2008 conversion terms are more restrictive.

>
> Because A-Label is defined in IDNA2008 (not in IDNA2003), we should follow
> the IDNA2008 definition precisely. Otherwise communication becomes difficult
> -- two people think they are in agreement about a point involving A-Labels,
> when they mean different things, and are thus not in agreement.
>

Agreement. IDNA2003 stated (RFC 3490) : "While all ACE labels begin with the
ACE prefix, not all labels beginning with the ACE prefix are necessarily ACE
labels." How should I call non-ACE/non-A-label "xn--" ASCII domain names?
Such domain names exist and therefore can be used by Cookies.

However, the problem I raised was that for the network DNS A-labels are the
reference, while U-labels are the IUser's reference, and that no concept has
been suggested (*) (on the IETF side) and no mechanism has been defined (*)
on the IUse Architecture side to make sure they strictly correspond
throughout applications. (*): I am aware of except the IDNApplication
concept (i.e. to centralise punycoding on each machine/network) and the
ML-DNS JFC is working on a running test.

Frankly, my understanding is that IDNA2008 is a vertical begining and real
IETF horizontal work is to come, as per
http://www.iab.org/documents/iabmins/iabmins.2010-04-07.txt and their
http://www.iab.org/appeals/2010-08-20-morfin-response.pdf committed reponse
.

Portzamparc

> Mark
>
> *— Il meglio è l’inimico del bene —*
>
>
> On Sat, Oct 23, 2010 at 19:05, jean-michel bernier de portzamparc <
> [email-masked]> wrote:
>
>> Considering anything else for cookies, is to reintroduce the confusion
>> that IDNA2008 clarified.
>> The problem for implementers is not there. The problem is to obtain a
>> local user authoritative A-label, something the AD was told not to ask but
>> IAB will have to document.
>>
>> Portzamparc
>>
>
>

---

From: John C Klensin <[email-masked]>
Date: Sun, 24 Oct 2010 16:15:35 -0400

Folks,  

There is a bit of the sound of FUD here and I don't think that
is useful, especially when some of the group are complicating
things with their own vocabulary.

Observations:

(1) We had experimental confirmation that the terminology in
IDNA2003 created a certain amount of confusion.  That was
partially due to not making a distinction between a "Unicode"
(really, native-script" or Unicode encoded in one of the
Unicode-standardized encodings) label that could be encoded by
ToASCII (after Nameprep processing and so on) and a similar
label that could be obtained from applying ToUnicode to such a
valid label encoded by ToASCII.  There was also confusion about
the exact status of strings that were encoded using Punycode,
had the "xn--" prefix, but were not actual, valid, IDN labels.
And, finally, there was confusion about whether strings encoded
by Punycode should be looked up without validating them
(although I personally believe that IDNA2003 was very clear on
that point, there was confusion nonetheless).  For exactly those
reasons, IDNA2008 contains much more precise definitions.  That
creates a situation in which, depending on how one looks at
things, there is a much broader range of strings that can be
considered valid (particularly those that depend on mapping) in
IDNA2003 than there is in IDNA2008.  That difference, however,
has a lot more to do with the validity of different presentation
forms (again, including forms that require mapping to be valid)
than fundamental validity.   And every valid A-label (under
IDNA2008) that represents Unicode characters that are valid in
Punycode-encoded labels that are valid output of ToASCII under
IDNA2003 is valid under IDNA2003 and has the same interpretation
when converted back to a native character string (U-label in
IDNA2008).

(2) Mark is quite correct in pointing out that the IDNA2003
considers many characters, especially symbols and punctuation,
valid that IDNA2008 does not.  That issue has nothing to do with
mapping and is really an entirely separate issue where validity
checks are concerned.  However, those who are concerned about
that difference in terms of lookup compatibility should read
Section 5.3 of RFC 5891 very carefully if they have not done so
already.  The case that it quite intentionally involves seems to
me to be precisely the one that Adam ideally has: a pair of
domain names that contain things that look like A-labels which
he wants to compare by a simple octet-by-octet comparison.  

In the less ideal case in which one of the two strings to be
compared is in native-character from, not Punycode-encoded, is
complicated for another reason.  IDNA2003 quite intentionally
left policies about what characters could actually be registered
for use in labels to registries, while IDNA2008 first eliminates
symbols punctuation, and other non-letter/ non-digit characters.
But every set of recommendations I've aware of for IDNA2003
recommended against registering a label containing any character
that was not a character validly used to write "words" in some
language.   Modulo some hand-waving, that is the same rule that
IDNA2008 specified as part of the PVALID/ DISALLOWED
distinction.   So, for those punctuation and symbol characters
(etc.) the difference between IDNA2003 and IDNA2008 has more to
do with something that one can get away with under IDNA2003 (in
some registries) but is explicitly prohibited under IDNA2008
rather than a new incompatibility.

How serious an incompatibility that is depends on how far one
wants to go to support characters that have been recognized as
bad practices since 2003 and earlier just because someone,
somewhere, may have successfully registered and used them
(remember that we are talking about domains in cookies here, not
what might have been used in an IRI).

regards,
   john

--On Sunday, October 24, 2010 21:21 +0200 jean-michel bernier de
portzamparc <[email-masked]> wrote:

>> There is a bit of a problem in terminology. I used the term
>> "punycode label" to include labels of the form "xn--...",
>> where the ... is valid Punycode representations (in ASCII) of
>> a Unicode string (with non-ASCII characters).
>> 
> 
>> If we are speaking precisely, the term "A-Label" is defined
>> in IDNA2008, and is more restrictive. It does not include all
>> the punycode labels that are valid in IDNA2003. So
>> "http://xn--1-wpn.blogspot.com/" (= http://€
>> 1.blogspot.com/) does not have an A-Label in it, but is a
>> punycode label, and is valid in IDNA2003.
>> 
> 
> This is also my understanding. But I understand it is only
> more restrictive because the IDNA2008 conversion terms are
> more restrictive.
> 
>> 
>> Because A-Label is defined in IDNA2008 (not in IDNA2003), we
>> should follow the IDNA2008 definition precisely. Otherwise
>> communication becomes difficult -- two people think they are
>> in agreement about a point involving A-Labels, when they mean
>> different things, and are thus not in agreement.
>> 
> 
> Agreement. IDNA2003 stated (RFC 3490) : "While all ACE labels
> begin with the ACE prefix, not all labels beginning with the
> ACE prefix are necessarily ACE labels." How should I call
> non-ACE/non-A-label "xn--" ASCII domain names? Such domain
> names exist and therefore can be used by Cookies.
> 
> However, the problem I raised was that for the network DNS
> A-labels are the reference, while U-labels are the IUser's
> reference, and that no concept has been suggested (*) (on the
> IETF side) and no mechanism has been defined (*) on the IUse
> Architecture side to make sure they strictly correspond
> throughout applications. (*): I am aware of except the
> IDNApplication concept (i.e. to centralise punycoding on each
> machine/network) and the ML-DNS JFC is working on a running
> test.
> 
> Frankly, my understanding is that IDNA2008 is a vertical
> begining and real IETF horizontal work is to come, as per
> http://www.iab.org/appeals/2010-08-20-morfin-response.pdf
> committed reponse .
> 
> Portzamparc
> 
> 
> 
>> Mark
>> 
>> *— Il meglio è l'inimico del bene —*
>> 
>> 
>> On Sat, Oct 23, 2010 at 19:05, jean-michel bernier de
>> portzamparc < [email-masked]> wrote:
>> 
>>>> what you would like to see, Patrik, I'm in agreement there
>>>> as well; that the goal is IDNA2008. And I think we'll get
>>>> there eventually, when the major registries disallow the
>>>> registrations of non-IDNA2008 names.
>>> 
>>> 
>>> Dear Mark,
>>> 
>>> whatever the policy of the "registries", their transitions,
>>> their interest in Unicode, their commercial, cultural or
>>> political strategies, etc.  they only use A-labels as far as
>>> the Internet and the Internet DNS are concerned (them having
>>> "xn--" headers or not - remember that until IDNA2003 every
>>> cooky was A-label only). These A-label having been initially
>>> registered as IDNA2003, IDNA2008 or xn-ascii does not make
>>> any difference. Cookies are not interested in the origin of
>>> the domain name, but in the value of the domain names. Every
>>> IDN has one and only one lowercase A-label value. And this
>>> value is here to stay.
>>> 
>>> Considering anything else for cookies, is to reintroduce the
>>> confusion that IDNA2008 clarified.
>>> Remember the sensible ".su" position: they will not register
>>> U-labels, but only A-label whatever the reverse
>>> punycodability.
>>> 
>>> The problem for implementers is not there. The problem is to
>>> obtain a local user authoritative A-label, something the AD
>>> was told not to ask but IAB will have to document.
>>> 
>>> Portzamparc
>>> 
>> 
>>

---

From: Mark Davis ☕ <[email-masked]>
Date: Sun, 24 Oct 2010 10:55:10 -0700

There is a bit of a problem in terminology. I used the term "punycode label"
to include labels of the form "xn--...", where the ... is valid Punycode
representations (in ASCII) of a Unicode string (with non-ASCII characters).

If we are speaking precisely, the term "A-Label" is defined in IDNA2008, and
is more restrictive. It does not include all the punycode labels that are
valid in IDNA2003. So "http://xn--1-wpn.blogspot.com/" (= http://€
1.blogspot.com/) does not have an A-Label in it, but is a punycode label,
and is valid in IDNA2003.

Mark

---

From: Mark Davis ☕ <[email-masked]>
Date: Mon, 25 Oct 2010 12:17:05 -0700

Well, there is definitely FUD involved, but often FUD is in the eye of the
beholder.

> We had experimental confirmation that the terminology in
IDNA2003 created a certain amount of confusion.

We had anecdotal evidence, not 'confirmation'. I do find some of the
conclusions reasonable -- for example, it is credible that people
registering labels that are mapped (eg ÖBB.at) might believe that what they
had registered was the unmapped version, whereas what they actually
registered was the mapped version (eg öbb.at <http://xn--bb-eka.at>), and
that such misunderstandings can cause problems.

But to justify the backwards incompatibilities, we *never* got any hard data
(no statistics by TLDs, none for the many other registries) to justify the
need for the thousands of backwards incompatibilities. No measurable impact
of what were felt to be "bad" practices; just anecdotes. So speaking of
'confirmation' is misleading.

Don't get me wrong; now that IDNA2008 is in place, we need to move to it.
But there will be a period of transition. It is easy to someone who doesn't
have product responsibilities to shrug that off, but those who do have
user-facing products are faced with dealing with the transition issues.

Mark

---------- Forwarded message ----------
From: John C Klensin <[email-masked]>
Date: Sun, Oct 24, 2010 at 13:15
Subject: Re: referencing IDNA2008 (and IDNA2003?)
To: jean-michel bernier de portzamparc <[email-masked]>, Mark Davis ☕ <
[email-masked]>
Cc: [email-masked], IDNA2010 Documentation <[email-masked]>,
[email-masked], internet users contributing group <[email-masked]>,
Peter Saint-Andre <[email-masked]>, Adam Barth <[email-masked]>

Folks,

Observations:

regards,
  john

---

From: "J-F C. Morfin" <[email-masked]>
Date: Sat, 30 Oct 2010 01:39:36 +0200

At 04:18 24/10/2010, jean-michel bernier de portzamparc wrote:
>JFC, Adam,
>This is something that Adam may easily understand: there are now two 
>ways of reading all the RFCs. One is like IDNA2003 (diversity is 
>loosely handled inside the IETF technology) and one is IDNA2008 
>(diversity is handled outside the IETF technology in using it in a 
>very strict, IETF decided common way). These two ways consider the 
>_same_ Internet (not a single bit being changed in the code) in a 
>very _different_ manner. If we used the same language, it would mean 
>that we would use the same terms in a rather different or even 
>opposite way. We would definitely not understand one another any more
.
Dear Jean-Michel,

I took my time before commenting on this, so that people may feel 
more relaxed about the matter. You are correct. On one side there is 
the IETF building the Internet in "influencing those who design, use, 
and manage it for the Internet to work better". On the other side 
there is at least our French Users' Diktyology effort (diktyology is 
the name of the network science) :-) that tries to understand what 
the Internet really is, so that its users can better benefit from the 
way it is designed, used and managed.

However, please remember that this is the IETF side. The IETF comes 
from RFC 1958 through RFC 3439 and progressively up to the RFC 
5890/5895 set. On our own side, we are coming from our need for much 
more than IDNA2008 (the Intersem, the internet of thoughts) and we 
consider that their support by the Internet architecture is implied 
in RFC 1958 and 3439 unless some   unfortunate errors woluld be made. 
However, we may be wrong in our own analysis, which is why, even if 
the IETF has eventually accepted to match some of our requirements, 
they may not accept others. Their job is to make sure that they could 
or could not be accepted based upon what they think their Internet 
technology is and should be.

Please also remember that RFC 3935 states: "The Internet isn't 
value-neutral, and neither is the IETF.  We want the Internet to be 
useful for communities that share our commitment to openness and 
fairness. We embrace technical concepts such as decentralized 
control, edge-user empowerment, and sharing of resources because 
those concepts resonate with the core values of the IETF community. 
These concepts have little to do with the technology that's possible, 
and much to do with the technology that we choose to create."

Our Internet use is not value-neutral either. However, our values are 
those consensually voted on by the users' communities (governments, 
industry, civil society, etc.) of the world at the WSIS (World Summit 
on Information Society) that they ignore because they refused to 
attend (by IETF Chair's clear policy).  Our values also includes the 
respect of the users' pragmatism, authority and liberty like the 
".su" (and many, many others) use of IDNA2008 or French majuscules, 
and many other cases they cannot consider because they cannot fit 
into their architectural scheme. And this is why we need our own 
hacking but not breaking additional other scheme.

Beware: they do not really understand it, and what the ML-DNS can be; 
and they are as much afraid of it as we are from their vision of the 
user interface. The reason why is that we cannot technically dialog 
with them before we have completed its experimentation. This  is due 
to the ISOC copyrights and IAB/IESG. I tried to get an IETF use area 
investigated that could have been granted further on a public domain 
stream. IESG and IAB were not ready for that and may-be this is 
better. Therefore, we will have to set-up our own IUTF/IUSG, using 
the IUCG to liaise with the IETF under an ISOC copyright limited to 
the document presentation, but without any possible claim on the content.

This is not that bad: they still have, and we may share or help, 
their rough consensus, and we will rendezvous on running code.

In the meanwhile, what we can offer for clarification sake in my 
spare time is a public domain conceptual architectural framework 
(ALFA) for them to understand where we think the Internet fits and 
why we are not interested in IETF side's documented "MUST"s until 
they get being respected among the network side's "ARE (period.)"s. 
The same as our own "MUST"s need to get respected by them as their 
own external world "ARE (period.)". This is what I engaged (cf. the 
end of my IAB appeal) and am trying to advance in parallel to our 
live ML-DNs experimentation. .

Our only potential problem are the inconsistencies that IETF may 
document for mid-term reasons while we are interested in short-term 
(operations) and long-term (metastructures). We have such an 
inconsistency in the RFC 2826, a document that definitely needs to be 
expended. We could have had some in RFC 4646 and RFC 5890. We have 
two pending ones with the precautionary principle and the "intertest" 
charter (for securely using the Internet as its own test-bed), and 
may be could we reduce some of the Unicode/Internet through an NUCS 
(networked universal character set).

For everyone to feel at ease, everyone has to accept this parallel: 
the same as IETF does not want and cannot change telecom standards as 
established by the ITU,  IUsers do not want and cannot change the 
IETF standards, words, and definition. This is because the ITU 
standards are not only used by the Internet, and Internet Use words, 
definitions, and the coming standards do not only concern Internet 
users. This is that  uncoupling which permits each telecom, Internet, 
use, and coming semantic stratum to freely progress.

This is why the "I" in ITU, IETF, IUI and IUser, and Intersem 
successively stands for international, internet, intelligent, and 
individual. Today, we are crossing the architectural middle: the IUI 
network use interface. On the Internet side the "I" stands for 
"Internet" and for "Intelligent" on the user side, as in "good 
intelligence". This "good intelligence", i.e. good metanetworking is 
not necessarily easy to get (but has to be) adhered to - in the 
short-term user (operational) range, in the governance (policies) 
middle-term range, and in the adminance (normative) long-term range.

Look at our own changes since we started investigating the ML-DNS and 
the implications that we found regarding things that we never 
considered first. Let us first work, test, and check. Our only 
interest was (as in the RFC 5988 case that we need to check):

- that Adam's Cookies are documented as coherent with the DNS core 
coherence becoming definitive "ARE" we can trust.
- to save time, that they are not limited to HTTP,
- that their management/utilization rules are well defined but also 
open and simple enough so that they can be used for whatever they can 
deliver outside the Internet, and not only for what their Internet 
designers had in mind.

jfc