Subject: [Ietf-not43] issues and questions on FIRS

From: anewton at ecotroph.net (Andrew Newton)
Date: Sat Aug 16 12:17:41 2003

I have a couple of questions regarding FIRS, some of them being 
complaince issues.
So that the lines between these issues do not blur, I've broken them out:

1) Naming Syntax of inetDnsDomainSyntax (Section 3 in 
draft-ietf-crisp-firs-dns).
2) The implementation of 3.1.8.
3) The implementation of 3.2.8.
4) Enumeration of error codes.

If I have simply overlooked something, I apologize.  Please hit me over 
the head with the pointers.

#1 - Naming Syntax of inetDnsDomainSyntax (Section 3 in 
draft-ietf-crisp-firs-dns).

First, I do not understand why the domain name is being specified as 
UTF-8.  I understand that this is being done in consideration of IDN's, 
but this is a service about the actually registered domain names.  While 
the UTF-8 equivalences are important, the key for the domain names (the 
rdn in this case) should be the wire equivalence since this about the 
registration of these domain names.  In other words, I think the key 
should be the 7-bit ASCII versions that show up in zone files and via 
dig, etc... and not the 8-bit versions.

Second, why are the 8-bit versions escaped?  Is there an issue with 
supporting UTF-8 or Unicode?  Just curious.

Third, point (a) in that section states how the escaped values must be 
stored.  I don't know if this is valid or not.  It could be right, but 
I'm a little concerned that this is making an assumption about how 
information is stored in a registry/registrar that just doesn't exist. 
As in, it MUST be this version of the ASCII escaped UTF-8, when what is 
stored might actually be the ASCII version and the Unicode version 
(non-escaped).

#2 - The implementation of 3.1.8.

There was mention of 3.1.8 in the meeting by Peter, but I do not see it 
addressed in the drafts.  Looking through the jabber logs, Peter 
proposed various methods to do this.  These were defined as: using the 
bind operation, special policy entries, and server-side extended operations.

The use of the bind operation would be overloading of LDAP's 
authentication mechanism and would likely cause confusion.  Plus I do 
not see how it would be tied to a query.  The second option of 
special-purpose policy entries would probably work, but I think the 
convention for doing it would be hard to coordinate.  The third option 
of an extended server-side operation I do not understand.

It would seem that this would be easily done with another control.  As 
in, the control for "only check permissions" is set and then the query 
is issued.  The control is then "released" after the answer.  I don't 
know.  This is just me theorizing.

Because LDAP supports a generic query syntax and because service 
providers are likely to deny all queries not explicitly allowed to 
prevent data mining, this requirement seems more important for the FIRS 
proposal.

#3 - The implementation of 3.2.8.

Looking at the jabber logs, there is also discussion of this item with 
theorizing about how it might be done but no mention of it in the 
drafts.  Nor can I find it.  I think the conversation in the jabber logs 
is correct with regard to this only needing to be done on the LDAP 
attribute level and not the LDAP value level, which Peter seemed to 
think would be harder to do.  However, either the control or the 
reserved values (or whatever) do need to be specified.

#4 - Enumeration of error codes.

Given the recent discussion on error codes and the mention of these in 
the jabber logs, I think it is necessary to fully enumerate them so that 
we can better understand what needs further clarification.  Some of the 
error codes would naturally map to existing LDAP error codes, but as we 
discussed with the bags, some will not.  This will help with listing 
which new codes needed to be defined.

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Sun Aug 17 14:58:56 2003

> dig, etc... and not the 8-bit versions.

This is a complex issue but the simple answer is that the UTF-8 form will
be the preferred form eventually, and it will be simpler in the long run
to do UTF-8 now rather than try to accomodate two systems later. Now for
the complex answer:

1) I had to choose one. It is likely that clients will exist that only
accept seven-bit ASCII for input/output, but it is also likely that there
will be clients which will use UTF-8, and it is also likely that some
clients will require some other local encoding (eg, UCS-2 on Win32).
However, protocol efficiency requires that a single canonical form be used
for the message data. The "majority" is going to have to do some kind of
conversion no matter what. None of them are significantly cheaper at this
exact moment in time.

All systems are required to do validation anyway. The cost of local
conversion is likely to be relatively minor if it done during the
validation process, so there's no significant incentive for any format
here either.

The wire format for LDAP protocol is UTF-8. Since all LDAP libraries
already have to support UTF-8 conversion in order to render LDAP data for
local input and output (EG, a company name, or a street address), this
means that reusing the existing UTF-8 conversion logic is going to be the
cheapest for everybody, since they code would have to exist already.

Furthermore, non-trivial comparisons are easier using UTF-8 than ACE.
Sub-string searches, soundex searches, etc., any of which may already be
implemented in LDAP servers for UTF-8, can be reused pretty easily (or
extended partially to allow for things like dot-separators between
labels). With something like IDNA, however, the servers would need
separate searches for all of these, with the first step being to produce a
normalized (UCS) instance. Using UTF-8 as the default makes this stuff
much much easier, and this is a very compelling argument.

2) i18n domain names have a canonical representation as UCS characters.
The IDNA (ASCII-compatible) representation is just one of many possible
UCS-to-<target> encodings for IDNs. The RFCs don't hammer this point
strong enough for my tastes, but the references to different "slots" makes
this clear: applications can use whatever representation of a canonical
IDN they wish, including IDNA but also including any other encoding that
the application and/or protocol supports.

At the CURRENT TIME, delegation entities are seeing IDNA as the dominant
encoding, but this is symptomatic rather than deliberate. For example, at
this particular moment DNS delegations and nameserver entries are limited
to the hostname syntax (letter-digit-hyphen), but that doesn't mean that
alternative encodings cannot be used in the future (I've done a fair bit
of work in this space, and my research shows that it is possible, and that
the problems are political not technological). Similarly, tools like dig
are currently constrained to ASCII output, but there is no reason to
believe that folks are not going to be developing IDN-aware tools that
perform IDNA-to-local conversions for input and output (this would
actually be pretty simple). I mean, there's nothing about dig that
prevents this; if we expect stuff like web browsers and email clients to
perform conversion, then certainly dig and hostname and the like can too.

Cumulatively this means that the use of ASCII-compatible sequences for
domain name ~management is an artificat of the CURRENT generation of
services and tools, and not a restriction of the technology. In all
likelihood, the tools and services will adapt, and then sooner or later
we'll have local encodings of the canonical IDNs being the default view,
rather than IDNA being the default.

3) If FIRS is chosen, and if the directory expands into the edges of the
user-to-user space, the use of LDAP tools to manipulate raw data will
become more common. This means that more and more people will eventually
be exposed to the protocol representation directly (modulo any wholesale
conversion that is needed for their platform).

People will want to work with the IDN they bought and/or see, not an
illegible encoding of that domain name. I mean, we certainly don't expect
that people will *want* to work with IDNA encodings in web pages or email
addresses, and I think that the CRISP user community is going to have the
same kinds of desires for the ~whois service too. With this in mind, the
smart play in terms of adoption is to give them what we already know
they're going to want, which is internationalized views. Ask your
marketing department if they are selling i18n domain names or encoded
representations of i18n domain names, and let that answer apply to all of
the other domain-based usages that might be plugged into the directory in
the long haul.

This is true from the user side too. When I get my first IDN spam, I'm
going to want to find details about the "ex?mple.com" domain, not
"xn--exmple-cua.com".

This is also where points 1 and 2 come together. The default UTF-8
encoding is mandatory in all cases and is therefore guaranteed to be
supported, while the tools which are currenlty used to manage domain names
suffer from version-specific restrictions and are likely to be eventually
upgraded so that they provide i18n representations by default. In other
words, the tools are likely to catch up to the user's desires and the
capabilities of the technology, rather than being a permanent restriction
on the entire system.

There is another minor point here, which is that the official IAB policy
calls for UTF-8 as the preferred encoding. Since we are implicitly hoping
for reuse of common data, this is a strong argument in favor of using
UTF-8 for (most) directory data regardless of its underlying definition.

So given all of that, using the UTF-8 representation makes the most sense
both in the short-term and the long-term.

On the other side of the coin is the operator convenience issue. Working
with a representation of the domain names which is different from the
representation used by the current generation of tools and services is
admittedly inconvenient. I don't think this is compelling enough to
justify forcing all users and layered applications into working with an
illegible encoding format by default. The tools will change eventually.
Furthermore, as was already stated, all names have to be validated anyway
and doing the conversion as part of the validation process (or more
likely, as part of populating the database) is not an egregious expense.

> supporting UTF-8 or Unicode?  Just curious.

DNS uses raw octet values, but there are no "characters" in the UCS
repertoire for referencing octet values. Instead, the octet values would
refer to other characters which do not represent the values themselves.

For example, a valid DNS domain name (not a hostname) can contain the
octet value of 0xC4, and this value must be preserved across all instances
of that domain name. In UCS, however, the character code C4 refers to
uppercase "A" with diaeresis. In many instances, the UCS character would
get normalized to lowercase "a" with diaeresis, which has the character
code value of E4. If this value is mapped back to DNS, the original domain
name would be destroyed.

The escape syntax is provided in order to support octet values in domain
names while preventing them from being interpreted as characters (so that
they aren't normalized, for example).

In theory, this could be avoided by limiting the domain name syntax to
hostnames, except for a couple of non-trivial issues. First and foremost,
there is not a strict definition of what constitutes a valid IDN, so all
values have to be allowed anyway. But if all values are allowed then
there's nothing to stop the users from entering octet values either, so an
escape mechanism of some kind is mandatory in any event. Secondarily, if
we want this service to be useful for domain names in the general case
(rather than being limited to the narrow purpose of "delegation
management") then we need to design for legitimate domain names rather
than attempt to enforce an arbitrary restriction tp the hostname subset.

> stored might actually be the ASCII version and the Unicode version 

As to the first part, domain registrations are limited to the hostname
subset by definition (the owner and data domain names of an NS RR have to
fit in the hosts.txt database). As such, these rules are really provided
for usages beyond "delegation management".

EG, the inetDnsRR specs implicitly allow the contents of a zone to be
stored in the directory (this isn't the stated goal, but a possibility
here is out-of-band zone replication, benefiting from ACLs and the other
buttons and knobs), and that some of the domain names in a zone are not
likely to conform to the hostname rules and will need to be escaped.
Whether or not a registry or registrar chooses to offer an ancillary
service will probably determine the extent to which they need to be
concerned with this kind of stuff. In the usual case, it won't be
necessary since input masking filters should keep the extended syntaxes
out of the delegation-specific data.

I'm not sure I understand the second part. If you're concerned that I'm
dictating underlying database formats, that's not the goal, and I can be
clearer in the text if you want. The real need here is for the protocol's
view of the database to be consistent. However that happens is an
operational concern.

> bind operation, special policy entries, and server-side extended operations.

Peter and I talked about this some in a private exchange. My suggestion
was to leave it as described in section 5.3.3 of firs-core-02 (with
"unwillingToPerform" as the generic response) for a later point where we
could talk about it in sufficient detail. Since we can add "explicit"
restrictions at a later point without breaking the implicit default
restrictions, there's no immediate rush.

However, if folks want to go ahead and start working on this, we certainly
can. I'd like to get a better analysis on the objective first though. Are
we wanting to provide something like a counter ("50 out of 50 possible
queries already performed, try back tomorrow"), a simple tooManyQueries
response, or an array of different responses?

Note that there are also some possible synchronization issues here, such
as the timezone the server is using.

> prevent data mining, this requirement seems more important for the FIRS 

Yeah, I agree.

> reserved values (or whatever) do need to be specified.

Section 5.3.4 of firs-core-02 tries to lay this off onto the LDAP specs:

 |  Clients MUST NOT equate the absence of any attributes with the
 |  absence of data, and SHOULD assume that the user is not authorized
 |  to view any data which has not been provided.
 |
 |  If a client specifically requests an entry or an attribute which
 |  the server is unable or unwilling to provide due to policy
 |  constraints, the server MUST use the appropriate LDAPv3 error
 |  message. For example, if the user is unable to view an entry or a
 |  requested attribute because it has not yet provided sufficient
 |  authentication credentials, the server MUST return the
 |  "invalidCredentials" error. Similarly, if the client has request
 |  an entry or attribute which the server is unwilling to provide due
 |  to policy reasons, the server MUST return the unwillingToPerform
 |  error to the client.

Those examples don't enumerate all of the possible reasons on purpose,
although I could do so. See the response to your next question below.

> which new codes needed to be defined.

I recognize and appreciate the desire to be explicit. However there is
another equally valid consideration, which is over-specifying to the point
where changes to LDAP are not accomodated, or cannot be accomodated,
resulting in FIRS becoming version-locked. I'd really like to just point
people to the LDAP specs for the proper response, and only give examples
where ambiguities exist or where illustration is needed. I'll accomodate
the consensus of course, but let's not go overboard.

I don't mind detailing new codes where necessary either.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

---

From: leslie at thinkingcat.com (Leslie Daigle)
Date: Sun Aug 17 22:58:09 2003

Eric,

You and I have already disagreed about this off-list, but:

> to do UTF-8 now rather than try to accomodate two systems later. Now for

I disagree that it is appropriate to be architecting the CRISP
system such that UTF-8 is sent in a query that is supposed to be
about domain names.

Today, domain names are still defined as ascii, and the
requirements talk about the ability to lookup domain names, not
"internationalized domain names".

For the very reasons you cite below, the client has to do the
normalization, and for DNS that means drilling down to
the ascii-encoding.

If this working group believes it is important to be able to
do queries on the internationlized form of domain names, then
that should be added to the requirements documents.

Otherwise, IMO, we wind up in a world of hurt.

Leslie.

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Sun Aug 17 23:43:04 2003

> "internationalized domain names".

These are valid concerns but at the same time, it's important to recognize
that it will be much more difficult to graft deep i18n support onto the
service later than to build it in up-front. Slapping a facade on it later
isn't going to hold up when everybody is poking around at the database
innards (and this becomes more likely as more services get layered on),
while changing the database contents won't be an option. Much better to
bite the bullet now and build in the deep support from the start. That's
my vote anyway.

---

From: Peter.Gietz at daasi.de (Peter Gietz)
Date: Mon Aug 18 08:05:51 2003

Some comments on #2, #3, and #4:

> the head with the pointers.
[...]

The first proposed solution (I):
> authentication mechanism and would likely cause confusion.  

This would be a control to the bind operation, which could do FIRS 
version negotiation and be used for the server to tell the client what 
kind of data it can expect to retrieve.

> Plus I do 
> not see how it would be tied to a query. 

Yes right, this would be more general information, not information about 
a special query. But to be honest, I don't understand this requirement 
in this caser. What is the difference between the two following client 
server dialogues?

1.) Without feature 3.1.8:
client binds to server
client makes request
server sends back I won't answer you this request

2.) With feature:
client binds to server
client asks server: will you answer this request
server answers no

The only difference IMO is that in cas 1 you don't need an additional 
circle in the case the server wants to answer the request. Thus I think 
for per request you don't need to check acls.

What makes more sence is to generaly ask "will you provide me with this 
sort of data?" and this could be checked inside an additional control to 
the bindrequest and response.

The second proposed solution (II):
 > The second option of
> convention for doing it would be hard to coordinate.  

Again this solution would only give back general rules about which 
client can expect which kind of data. Such policy could of course be 
places at many places in the Directory Information Tree using the 
concept of subtree subentry and thus be less general than solution I.

The client could check e.g. the policy of a subtree of the whole tree 
the server has as naming context, and evaluate itself if it can expect 
to retrieve data or not. I once wrote a schema for LDAP-crawler policy 
and would be willing to write something similiar for FIRS.

Solution III:
 > The third option
There are two similiar LDAP protocol extension mechanisms, one is the 
control, which enhances existing operations, like the control defined 
for enhancing the search operation with the relay bag feature.

The second LDAP protocol extension mechanisms is called extended 
operation, which is a total new operation with request and response 
defintions. Since check-ACLs would be a new operation I think, it is 
better to have an extended operation instead of another search control.

extended operations have to be understood by client and server. I am not 
sure how the "server-side" came into the jabber logs.
> know.  This is just me theorizing.

are you talking about a search extending controll here?

> prevent data mining, this requirement seems more important for the FIRS 

I still think that a per request granular implementation of this feature 
is rather useless (see my remarks to solution I)

> reserved values (or whatever) do need to be specified.

Isn't this addressed in section 5.1.4. of 
http://www.ietf.org/internet-drafts/draft-ietf-crisp-firs-core-02.txt,
where three standard LDAP error codes are used, as proposed by me in the 
  meeting?

> which new codes needed to be defined.

My current understanding is that we only need new error codes for the 
relay bag stuff. All other FIRS errors can be mapped by standard LDAP 
error codes. That is what Eric said and I have no objections to this.

Of course it may be a good thing to have an appendix in firs-core ore 
firs-arch listing all error codes used in FIRS, but this will produce 
dependencies with the relay bag draft...

Cheers,

Peter

> 
> -andy
> 
> _______________________________________________
> Ietf-not43 mailing list
> [email-masked]
> https://lists.verisignlabs.com/mailman/listinfo/ietf-not43

-- 
_______________________________________________________________________

Peter Gietz (CEO)
DAASI International GmbH                phone: +49 7071 2970336
Wilhelmstr. 106                         Fax:   +49 7071 295114
D-72074 T?bingen                        email: [email-masked]
Germany                                 Web:   www.daasi.de

Directory Applications for Advanced Security and Information Management
_______________________________________________________________________

---

From: leslie at thinkingcat.com (Leslie Daigle)
Date: Mon Aug 18 10:48:43 2003

And I don't think we know (precisely) which bullet we're
talking about biting.

Which is why it makes sense to me to split it into 2 separate
types of query.

Note that, were I designing software for an enterprise product,
I would probably choose differently.  But this is Internet
infrastructure -- which changes slowly, and where operators
of such services want precision in their requirements because
they have systems to run at 5 9's.

Leslie.

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Mon Aug 18 11:01:31 2003

> And I don't think we know (precisely) which bullet we're

I personally went back and forth on this subject for many months, being
unable to decide if the large list of work items would produce a benefit
that was worth the effort (Andy can tell you how late I was). But in the
end there is the unavoidable conclusion that it is just easier for
everybody to do the work up front. As complex as the rules appear to be
(although they aren't any more complex than the existing DNS and LDAP
rules, at least in terms of escapes), it is going to be much much more
difficult to treat the identifiers differently while still providing the
benefits that we know people want.

> Which is why it makes sense to me to split it into 2 separate

It's "two sets of queries". You need separate internationalized queries
for inetDnsDomainMatch, but you also need separate queries for
inetDnsDomainEqualityMatch, inetDnsDomainSoundexMatch, and all of the
other queries that make LDAP useful as a CRISP repository in general.

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Mon Aug 18 11:47:40 2003

My comments are in-line:
(I now regret not starting a seperate thread for each item)

> to do UTF-8 now rather than try to accomodate two systems later. Now for
[...snip...]
> for usages beyond "delegation management".

My concern is that the xn-- 7-bit versions of the domains are the things 
that will be flying around the Internet and people will need to refer to 
them, even if some of their software does the Unicode conversion.  The 
fact is, the xn-- 7-bit version is what IS going to be in the zone 
files, and for troubleshooting purposes we need to be able to refer to them.

I have no problem with the protocol also allowing a user to specify the 
name in Unicode.  IRIS takes care of both cases.  I just think it is 
important to also specify it for people mucking around in the 7-bit world.

> response, or an array of different responses?

I don't quite follow this.  It sounds like you are talking about the 
first part of 3.2.5., which is something that is already defined in 
stock LDAP (I think).

> although I could do so. See the response to your next question below.

As I understand it, LDAP error codes are returned per response, not per 
result or per attribute.  If I'm wrong, then this changes things and 
please forgive me.

Let's say I submit a query and get back the following:

START RESPONSE

Contact: bob-192
Name: Bob Smurd
Email: [email-masked]
Phone: 555-1212

Contact: lisa-543
Name: Lisa Smith
Phone: 121-5555

Contact: charles-777
Name: Charles Taylor
Email: [email-masked]
Status: deposed

Error: unwillingToPerform

END RESPONSE

What happened here?

1 - What bit of information did I not get because of a policy decision? 
  Was that the phone number for charles-777 or the email address for 
lisa-543?

2 - What was the server unwillingToPerform?  It gave me back 3 entries! 
  It certainly was willing to do something.

(Sorry for the last example entry... too much coffee this morning!)

> the consensus of course, but let's not go overboard.

See my example on unwillingToPerform in #3.  It is this type of thing 
that makes me think this excercise should be undertaken.

Also, I don't follow your comment about version locking with respect to 
FIRS and LDAP.  Is there a dependency that has implications to CRISP?

-andy

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Mon Aug 18 12:05:02 2003

Peter, please see my reply to Eric regarding #3 and #4.  I think they 
speak to your points.

As for #2:

> are you talking about a search extending controll here?

I believe I am thinking of the former, but it could probably be done 
using either.  I assume by your description that a search extending 
control (the first protocol mechism you mention) is easier or better, so 
let's go with that.  Could it be done that way?

-andy

---

From: Peter.Gietz at daasi.de (Peter Gietz)
Date: Mon Aug 18 12:19:48 2003

[...]
>>> Looking at the jabber logs, there is also discussion of this item 
>>> with theorizing about how it might be done but no mention of it in 
>>> the drafts.  Nor can I find it.  I think the conversation in the 
>>> jabber logs is correct with regard to this only needing to be done on 
>>> the LDAP attribute level and not the LDAP value level, which Peter 
>>> seemed to think would be harder to do.  However, either the control 
>>> or the reserved values (or whatever) do need to be specified.
>>
>>
>>

There are two LDAP errors that indicate that only a partial list has 
been given back: timeLimitExceeded and sizeLimitExceeded.

unwillingToPerform should not be used for partial lists. Now that we do 
define FIRS specific error codes, I would vote for adding two more for 
this. What about:

authorizationLimitExceeded and privacyConstraintsLimitExceeded

>>> Given the recent discussion on error codes and the mention of these 
>>> in the jabber logs, I think it is necessary to fully enumerate them 
>>> so that we can better understand what needs further clarification.  
>>> Some of the error codes would naturally map to existing LDAP error 
>>> codes, but as we discussed with the bags, some will not.  This will 
>>> help with listing which new codes needed to be defined.
>>
>>
>>

I more and more tend to agree.

Peter

Peter Gietz (CEO)

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Mon Aug 18 12:31:49 2003

Peter Gietz wrote:
> 
>>
>> As I understand it, LDAP error codes are returned per response, not 
>> per result or per attribute.  If I'm wrong, then this changes things 
>> and please forgive me.
>>
>> 1 - What bit of information did I not get because of a policy 
>> decision?  Was that the phone number for charles-777 or the email 
>> address for lisa-543?
>>
>> 2 - What was the server unwillingToPerform?  It gave me back 3 
>> entries!  It certainly was willing to do something.
>>

When you define them, is it possible that they are accompanied by a list 
of dn's + attribute names so that the client can correlate which 
attributes in which entries are being redacted?  (see the first question.)

Or am I making a false assumption?  Can an error code be returned per 
entry or attribute?

On other possible solution is to have the server return a special entry 
(perhaps last) being a list of dn+attribute name / reason pairs.  Again, 
just theorizing here.

-andy

---

From: Peter.Gietz at daasi.de (Peter Gietz)
Date: Mon Aug 18 13:11:16 2003

Andrew Newton wrote:

>> unwillingToPerform should not be used for partial lists. Now that we 
>> do define FIRS specific error codes, I would vote for adding two more 
>> for this. What about:
>>
No, you can give back a result set and one error code for he whole request.

> (perhaps last) being a list of dn+attribute name / reason pairs.  Again, 

This could be done, but would be very FIRS specific and we woulld have 
to define schema for that. I don't think we really want to go this 
direction.

wouldn't it be enough to tell that more entries would have been returned 
if authentication were different? That would be indicated by the error 
code authorizationLimitExceeded.

Peter

Peter Gietz (CEO)

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Mon Aug 18 14:19:04 2003

> code authorizationLimitExceeded.

Well, I'm not sure it is useful otherwise given my example.  Besides, 
3.2.8 talks about these errors with respect to values not result sets.

-andy

---

From: davidb at verisignlabs.com (David Blacka)
Date: Mon Aug 18 14:29:53 2003

> code authorizationLimitExceeded.

Maybe I missed something here, but I thought Andy's question was about 
various *attributes* being removed (say, for privacy reasons).  How, in 
FIRS, is the client able to tell which attributes were removed and why 
(why being either "insufficient authorization", "private", or no 
explanation according to 3.2.8.1)?

-- 
David Blacka    <[email-masked]> 
Sr. Engineer    Verisign Applied Research

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Mon Aug 18 14:50:58 2003

on 8/18/2003 11:14 AM Andrew Newton wrote:

> My concern is that the xn-- 7-bit versions of the domains are the
> things that will be flying around the Internet and people will need to
> refer to them, even if some of their software does the Unicode
> conversion.  The fact is, the xn-- 7-bit version is what IS going to be
> in the zone files, and for troubleshooting purposes we need to be able
> to refer to them.

Not to belabor this particular point, but rather the issue as a whole...
The format of a domain name in any particular zone file is going to be
determined by the feature set of the server in use. Somebody is eventually
 going to figure out that administrators in ~Japan would prefer to have
their zone files use ~Katakana instead of ASCII. There is absolutely no
reason that a server cannot convert the data into ACE when the zone data
is loaded into memory (exceptions exist for products that ignore the DNS
in-band replication service in favor of things like rsynch, but those are
also marketing choices). So again all of this stuff is symptomatic of the
current state of the products and not a deliberate restriction. We should
be planning for where we think we'll be (or where we want to be) in 10
years, not where the last 20 years forced the current offerings to be.

> important to also specify it for people mucking around in the 7-bit

I'm not sure we're on the same wavelength. What inputs or outputs are you
worried about specifically?

In the current model, the client is required to normalize input to the
UTF-8 representations of the canonical UCS name, and most of the
protocol-level traffic is required to use UTF-8 . That's JUST the
protocol-level traffic.

The format of the database contents are intended to represent the richest
possible view since those can always be down-converted. So for something
like an entry name (the cn attribute) that would be the UTF-8 form.
However, there are other classes of attributes where this is not going to
be an option. For example, the mail and labeledURI attributes are locked
to seven-bit syntaxes so they cannot (currently) use UTF-8. Meanwhile,
stuff like nameserver attributes would be stored in their rich form, but
would not use any of the escape mechanisms given their restrictions from
the DNS service.

Separately, the client is allowed to convert response domain names into
IDNA for local display purposes if necessary. For example, I could add an
"--output=idna" switch to firs.pl which would force the client to convert
any domain attribute to IDNA for display purposes. That would give users
the option of working with IDNA in all cases.

For anybody else's client (especially one which is intended to be highly
portable), IDNA might even be the default representation of the domain
name attributes.

So in all cases, the user has the option of working with whatever form
they find best suited to their needs.

All of this would also be possible if we reversed it around, such that
IDNA was used for the protocol messages, the database storage and the
default view, except that it would be much more difficult to implement
holistically and as deeply (it would require multiple search filters,
server-side conversion, etc).

Is there some other area of concern that I'm missing?

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Mon Aug 18 16:50:42 2003

on 8/18/2003 11:14 AM Andrew Newton wrote:

>>> There was mention of 3.1.8 in the meeting by Peter, but I do not
>>> see it addressed in the drafts.  Looking through the jabber logs,
>>> Peter proposed various methods to do this.  These were defined as:
>>> using the bind operation, special policy entries, and server-side
>>> extended operations.
>> 
>> However, if folks want to go ahead and start working on this, we
>> certainly can. I'd like to get a better analysis on the objective
>> first though. Are we wanting to provide something like a counter ("50
>> out of 50 possible queries already performed, try back tomorrow"), a
>> simple tooManyQueries response, or an array of different responses?
> 

3.2.5 talks about maximum results in queries, while my understanding of
this particular issue is that it is related to the ability to issue any
queries at all, possibly because of rate-limited (too many queries today
alredy), or possibly because of some other requirements.

The general cases are already defined as bind errors (unwillingToPerform
essentially means "just go away"). I'm trying to get a handle on exactly
what kind of limits we are trying to express so that I can describe them
properly. So, is 3.1.8 specifically about fixed query volume (try again
after HH:MM UCT), the query interval rate (wait five minutes), both of
these, something else, or what?

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Mon Aug 18 17:05:01 2003

> these, something else, or what?

Think of it as the "dry run" -n switch for rsync or something.  The 
results maybe different depending on the subdirectory/mount point/file 
permissions you may have, but you will know if it can do what you want 
it to do without having it do it.

Does this make it clearer?

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Mon Aug 18 21:34:20 2003

> Does this make it clearer?

Not really. As it stands, it seems like the requirement is already being
satisfied. Quoting section 3.1.8.1 of reqs-5:

| 3.1.8.1 Protocol Requirement
|
|  The protocol MUST provide a mechanism allowing a client to determine
|  if a query will be denied before the query is submitted according to
|  the appropriate policies of the operator.

There are already several bind error responses that tell a client that
they won't get service before they issue the query, so the requirement is
already being satisfied by the protocol (Peter said as much during his
presentation as well, and talked about other mechanisms as alternative
mechanisms). The alternatives have other costs, and I can imagine some
other mechanisms that could be useful for specific situations (like
hit-counters), but I'd like to hear what the operators think is needed
before I start extending the protocol even more.

Maybe we can revisit this at a later time.

---

From: hardie at qualcomm.com ([email-masked])
Date: Tue Aug 19 14:38:48 2003

>before I start extending the protocol even more.

There seem to me too different cases here:

1) Server needs to say "I won't provide $YOU with service"

and

2)Server needs to say "I won't return results to $QUERY to $YOU".

I understand how the bind mechanisms apply to 1.  The question,
as I understand it, is how to satisfy 2.  If there are multiple ways
it could be solved in FIRS, could you please pick any one of them
and describe it in medium-level detail, so that we can move past
the question of whether this requirement is met?

			thanks,
				Ted Hardie

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 19 17:54:59 2003

> the question of whether this requirement is met?

I have added the following sentence to the discussion on the firsVersion
control in core-03 (not-yet published):

  Servers MAY adjust the list of supported resource types as necessary
  or desirable.

This essentially means that a server can tell client-X that a particular
resource is supported while telling client-Y that the exact same resource
is not supported. Since the firsVersion control is encouraged to be
returned in response to a successful bind, this mechanism imposes no
additional mandatory round-trips.

---

From: Peter.Gietz at daasi.de (Peter Gietz)
Date: Wed Aug 20 07:41:27 2003

> additional mandatory round-trips.

Either the client has to include the query it wants to send in the bind 
control, or Ted's 2) is not fulfilled.

I still don't really understand 2) anyway. What is the difference between:

2a) client sends $QUERY in a search request; server says "I won't return 
results to $QUERY to $YOU"

2b) client asks what would you do if I send $QUERY; server says "I won't 
return results to $QUERY to $YOU"

That is why I thought 2b would make more sence if the client can 
retrieve general server policy published as LDAP data (Solution A).

A search control with the function "don't perform the search, just tell 
me, if you would answer this request" as alternative is even easier to 
specify. (Solution B)

It would sort of be the same as specifying this in the bind control, 
with intended query data given by the client with the bind request. But 
then the answer would be only for one specific query, or the client 
would send a list of all intended queries. (Solution C1)

Or and this seems to be the path Eric wants to follow, the server gives 
back a general answer together with the bind response to the client what 
kind of queries it is willing to answer to this particular client. 
(Solution C2) This would be similiar to the policy publishing solution, 
but not expressed in LDAP data, but in the bind control response.

What I want to say here is
- However 3.1.8. is to be interpreted, there are several solutions to 
fulfil it

- Solution C2, as Eric seems to propose is doable, but it needs a 
detailed specification of what kind of entries (defined by object class) 
and what kind of attributes (defined by an attribute list) will be 
returned to this particular client. If there are different policies for 
different subtrees, the DN of the subtree has to be specified as well. 
So an answer could be something like:

dn: x=y, dc=xxx, dc=yyy$oc: inetOrgPerson$at: attr1, attr2, attr3$oc: 
inetIpv6Network$at: attr4, attr5, attr6
dn: x=z, dc=xxx, dc=yyy$oc: inetOrgPerson$at: attr1, attr2, attr3

With this, a client can compute, which kind of queries will be serverd 
and which won't.

Cheers,

Peter

Peter Gietz (CEO)

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Wed Aug 20 07:58:20 2003

> me, if you would answer this request" as alternative is even easier to 

I agree.  Solution 2 seems to be the easiest thing to do.  I'm not sure 
why it is so difficult.

> With this, a client can compute, which kind of queries will be serverd 

This is an interesting approach.  It would probably need to be worked 
out more thoroughly on paper first.  However, you would also need to 
include the qualifying matching rules and search scope.

-andy

---

From: Peter.Gietz at daasi.de (Peter Gietz)
Date: Wed Aug 20 09:13:43 2003

Andrew Newton wrote:

> Peter Gietz wrote:
> 
>>
>>
>> Either the client has to include the query it wants to send in the 
>> bind control, or Ted's 2) is not fulfilled.
>>
>> 2a) client sends $QUERY in a search request; server says "I won't 
>> 2b) client asks what would you do if I send $QUERY; server says "I 
>> won't return results to $QUERY to $YOU"
>>
>> A search control with the function "don't perform the search, just 
>> tell me, if you would answer this request" as alternative is even 
>> easier to specify. (Solution B)
> 
> 
>> detailed specification of what kind of entries (defined by object 
>> class) and what kind of attributes (defined by an attribute list) will 
>> be returned to this particular client. If there are different policies 
>> for different subtrees, the DN of the subtree has to be specified as 
>> well. So an answer could be something like:
>>

Why matching rules? Do you want to distinguish between string and 
substring search here?

With scope I am not so sure either, since the policy should rule for 
whole subtrees. It shouldn't matter if the client makes basedn, onelevel 
or subtree searches, the access policy should be the same.

I don't want to get this too complex. What might be added is the server 
side sizelimit (the maximum of entries a server will be willing to give 
back to the client), which should be one number for all naming contexts 
of the server.

Cheers,

Peter

Peter Gietz (CEO)

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Wed Aug 20 09:32:47 2003

> back to the client), which should be one number for all naming contexts 

People will be attempting to mine data from these servers.  That is a 
fact of life.  I know our service will not allow somebody to do a 
subtree search in the contacts DIT.  For RLDAP, we excluded any partial 
string matching if they didn't give use the first three letters of the 
name (e.g. cn=*, cn=b*, and cn=bob*).  Therefore matching rules, search 
filters, and scope will be allowed and disallowed depending on what the 
person is attempting to do.

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Wed Aug 20 14:19:13 2003

> I agree.  Solution 2 seems to be the easiest thing to do.  I'm not sure 

Something more comprehensive than "act like rsynch" will probably get a
more direct solution.

Right now, the mechanism is modelled after the CAPABILITY output in the
various email protocols. The server returns a list of suported features,
and which can vary based on the user identity (this is returned after
auth). Clients don't have to burn session-setup time by probing for the
server's feature set ("do you support F1? how about F2?"), but instead can
just look at the list of features in the CAPABILITY output. It's faster
since it eliminates round-trips for distinct probes, and it's cumulatively
easier on the server than having to issue errors for every failed command
from every distinct client.

> person is attempting to do.

One thing I've been considering is changing the list output to reflect
available query types instead of or in addition to object class versions,
and this seems to indicate a desire for similar effect. Perhaps an
additional bind control (instead of overloading this one) can do that the
best while still preserving the efficiencies of the CAPABILITY model.

I'll start looking into this.

---

From: leslie at thinkingcat.com (Leslie Daigle)
Date: Tue Aug 26 12:17:34 2003

Catching up on a couple of threads:

> years, not where the last 20 years forced the current offerings to be.

What we are chartered to address is a protocol for a core infrastructure
service -- i.e., we have to define something that is usable by the 
registrars & registries (which will be the backbone providers of data)
to provide a clear, uniform service globally.   By uniform, I mean that
it is possible to write clients that have expectations set appropriately.

Taking an example from another area -- there were far more brilliant
hypertext systems implemented long before the world wide web, none
of which would have succeeded at the scale of the Web.  Essentially,
http's simplicity & uniformity made it possible for completely unrelated
systems to be built and interoperate as a servICE, not just a protocol.
And then the Web blossomed, driven (indeed) by edge-user demand and
creativity.  Without that simplicity & uniformity, we would have
had islands of connectivity (pretty much like the more brilliant
hypertext systems).

My point:  we know we have to serve domain names properly, today.
We believe we will have to handle IDNs properly, but we don't yet
(entirely) know what that will mean (more on that below).  While
we need flexibility in the protocol, speculation, infinite extensibility
and configurability by individual operators will be the death
of global deployment and interoperation as a servICE.

> portable), IDNA might even be the default representation of the domain

IMO, a globally-deployed servICE is going to have to have a uniform
policy on what format domains are expressed in to the servers.  It
makes little sense, then, to overload the attribute's characteristics
in the protocol. 

> Is there some other area of concern that I'm missing?

IDNs have a lot of (as-yet-unfinalized) baggage in terms of 
variants.  Therefore, I suspect that "matching" for an IDN is
going to have to be looser than for 7-bit ascii.  I.e., doing
exact-match on UTF-8 bytes is going to be unsatisfying as a 
search tool (requiring the client to generate all possible variants
seems unwieldy, generates unnecessary queries to the servers, and 
besides, the server may well have a better notion of what "local"
variants are allowed, etc).

My point:  no matter how you slice it, domain names and IDNs have
different characteristics  and different matching/searching
requirements.  Handling IDNs as UTF-8 domain names will, I believe,
do a disservice to IDNs.  If we need to handle both, fine -- but
let's separate them out explicitly.

Leslie.

-- 

-------------------------------------------------------------------
"Reality:
     Yours to discover."
                                -- ThinkingCat
Leslie Daigle
[email-masked]
-------------------------------------------------------------------

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 26 13:46:53 2003

This discussion is going all over the map, so I'm going to focus on
specific issues. Personally I'd rather talk about larger considerations,
like whether we should be more externally-focused than internally-focused
and putting user-side considerations above operator considerations.

How does IRIS intend to satisfy the actor-specific requirements for
substring searching within an IDNA-encoded name?

How does IRIS justify the imposition of IDNA-encoded naming when the
community space of an edge-based application consists of native language
speakers who would prefer to work with resources in the representation
they paid for, instead of the represntation that is convenient for the
operators of a supposed "service"?

on 8/26/2003 11:14 AM Leslie Daigle wrote:

> What we are chartered to address is a protocol for a core
> infrastructure service -- i.e., we have to define something that is
> usable by the registrars & registries (which will be the backbone
> providers of data) to provide a clear, uniform service globally.   By
> uniform, I mean that it is possible to write clients that have
> expectations set appropriately.

FIRS provides exactly that: all identifiers (including netblocks for that
matter) are collapsed and normalized to their absolute canonical form, and
no other forms are allowed INSIDE THE PROTOCOL. What happens between the
protocol and a local presentation subsystem is a matter of preference for
each particular client.

The real area of concern (and where all the real work is done) is ensuring
that the names are properly formed when they are put into the system
(either inserted into the database, or generated as a query value). The
current specs describe a mandatory double-transformation mechanism that
ensures that the data can only exist in a single canonical form.

There is nothing better than this. For example, it produces stricter
output than "just use ASCII" for IDNs, but also in those cases where the
domain names are not IDNs. Not only is the canonical domain name validated
(which wouldn't be the case with reusing the ASCII fields), but the ASCII
representation is also validated more strictly than it would be in the
normal case since the ASCII representation doesn't accurately account for
all of the STD13 rules but which are represented in the stricter rules.

So by every measure, the rules that require absolute normalization produce
the best formed domain names possible, even in those cases where the
domain names aren't IDNs. In short, edge-based strict normalization
delivers exactly what you are asking for, and delivers it better than any
other mechanism.

It also happens to be the cheapest to implement. For example, all
implementations have different presentation targets (UCS-2, IDNA,
whatever), but all implementations also have to support UTF-8 in order to
use the underlying technology, so using UTF-8 at the back and allowing
client-specific local transformations is the exact same behavior that the
applications ALREADY SUPPORT for all of the other attributes that contain
non-ASCII text.

This says nothing about the cost of having servers perform conversion for
comparison purposes, or having to build and support multiple search
filters, or having to maintain 2x the entries (one for IDNA and one for
"other"), or...

Trying to avoid the "problem" only makes the damages worse, and the cost
of repairing the resulting damage is multiples more expensive. Meanwhile,
using the canonical form is cheaper and more efficient, and produces
stricter output for legacy domain names as well. That's what led me to
this design, and without hard evidence to the contrary... I'd be lying to
myself if I said that supporting two weakly-formed representations made
any kind of sense for anybody.

> hypertext systems implemented long before the world wide web, none of
> which would have succeeded at the scale of the Web.  Essentially, 
> http's simplicity & uniformity made it possible for completely
> unrelated systems to be built and interoperate as a servICE, not just a
> protocol. And then the Web blossomed, driven (indeed) by edge-user
> demand and creativity.  Without that simplicity & uniformity, we would
> have had islands of connectivity (pretty much like the more brilliant 
> hypertext systems).

This is a bad analogy for several reasons ("are you referring to HTTP .9,
1.0, or 1.1 as successful?"...).

I think we're all trying to deliver on functional and scalable and simple
but taking different tacts.

> My point:  we know we have to serve domain names properly, today. We
> believe we will have to handle IDNs properly, but we don't yet 
> (entirely) know what that will mean (more on that below).  While we
> need flexibility in the protocol, speculation, infinite extensibility 
> and configurability by individual operators will be the death of global
> deployment and interoperation as a servICE.

My rules would produce the least amount of variance.

> makes little sense, then, to overload the attribute's characteristics 

I don't know where you got that. My rules are entirely about producing a
single canonical form within the protocol.

Different clients have different requirements for local presentation, but
that's beyond the scope of the protocol, other than the fact that a
heavily normalized canonical form is the easiest to work with.

> IDNs have a lot of (as-yet-unfinalized) baggage in terms of variants.
> Therefore, I suspect that "matching" for an IDN is going to have to be
> looser than for 7-bit ascii. I.e., doing exact-match on UTF-8 bytes is
> going to be unsatisfying as a search tool (requiring the client to
> generate all possible variants seems unwieldy, generates unnecessary
> queries to the servers, and besides, the server may well have a better
> notion of what "local" variants are allowed, etc).

How do you plan to do this with IDNA, given that substring matching with
IDNA is impossible in many cases? No matter what you're answer is, it's
going to be easier with UTF-8.

> requirements.  Handling IDNs as UTF-8 domain names will, I believe, do
> a disservice to IDNs.  If we need to handle both, fine -- but let's
> separate them out explicitly.

That is twice the work to guarantee strict normalizations, twice the
storage, twice the filters, and in the end doesn't provide users with the
domain names they actually paid for.

---

From: leslie at thinkingcat.com (Leslie Daigle)
Date: Tue Aug 26 14:45:16 2003

Hmm.  
> and putting user-side considerations above operator considerations.

Then you fundamentally and utterly missed my point.  

If we don't build the core properly (and that is the part that
is intimately entwined with operator considerations you so determinedly
brush off) there won't BE any useful user-side services because
there will be no interoperable infrastructure service for them
to use:  there will be pockets of cute data systems.

YOU do not know the half of the universe of potential user side applications
any more than the next person, or this group collectively.  That's
one reason why we aren't spec'ing for it.

> operators of a supposed "service"?

I don't really know why you're bringing IRIS into this.

And I didn't bring up the issue from the standpoint of what is more
convenient for the operator.

I brought up the point because I don't believe you, or anyone
else, adequately understands what "match" is going to mean for
IDNs.  That's going to bite users as much as service operators.

With respect to my point on variants, which I think you also
missed, I was referring to the language-specific variant tables
currently under discussion in Taiwan, China, Japan and CENTR,
and the possibility (probability?) that registering one IDN
is going to necessarily block out a *set* of IDNs.  You can
normalize user input all day long, but if they've entered
a recognized variant of the IDN, is it appropriate to return
"no match" or a fuzzy match?

So, my point is: conflating domain name and internationalized
domain name in the protocol is a mistake, because these things
will have inherently different handling requirements.

Leslie.

--

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Tue Aug 26 15:03:18 2003

> and putting user-side considerations above operator considerations.

What user-side considerations are we putting aside?  I don't understand 
the concern.

> substring searching within an IDNA-encoded name?

Good question!  First, I'll be the first one to admit we do not fully 
understand all the IDN implications.  So if anything is not quite right, 
I'm sure we can change it for the better.

One the search side, IRIS defines two separate searches: 1) find domains 
by name (<findDomainsByName>) and 2) find domains by internationalized 
name (<findDomainsByI18NName>).  I realize that it could be one search 
with a modifier switch, but for the sake of simplified server code paths 
I thought this might be better.  Anyway...

The first search wants a 7-bit ASCII domain name.  The second search is 
wants UTF-8 and allows for optional language tags.  Because each 
registry will be defining its own rules for the registering of IDN's, 
the language tags allow the client to hint to the server about any 
special normalization/canonicalization/transformation processes that 
might need to take place during the search.  Finally, there is a search 
too wide error (<searchTooWide>) which allows the server to tell the 
client, "I would return something if you'd narrow your search."

On the lookup side, entity names can be UTF-8, so a registry can allow 
lookup by either the 7-bit ASCII name, the 8-bit UTF-8 name, or both. 
Its their choice.

For results, there are two types of results.  The first is the obvious 
domain object (<domain>) and the second is the domain variant object 
(<domainVariant>).  The decision to use the domain variant object is up 
to the policies of the registry.  Some will have them, some won't.  Some 
might have them but not show them.  That's cool too.  IRIS allows them 
to express a difference between the two if they so desire.

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 26 15:14:18 2003

on 8/26/2003 1:42 PM Leslie Daigle wrote:

> brush off)

I'm not brushing it off, I'm trying to point out a fundamental principle
that continues to be ignored: the demands of an operator-to-operator
service are different from the demands of an operator-to-user or
user-to-user service, and the technologies which may be appropriate for
the former are not necessarily suitable to the latter. The demand to use
IDNA because it's the same as the data in the ~com. zone file is a pure
example of this in action: the community doesn't give a flying crap about
what bits are in that specific text file. It's the wrong focus point.

> there won't BE any useful user-side services because

That's obviously a given, but it doesn't apply to this case much. We're
talking about the canonical format of the data and (related) where the
normalization and validation should occur, which is a trivial point of
argument given the clarity of the choices: two unrelated hierarchies with
independent filters and rules versus a single canonical format that
everybody has to support anyway? If we could keep the extra-protocol
interests out of this debate we'd have been finished with it by now.

> I don't really know why you're bringing IRIS into this.

It seems apropos to the discussion at hand given the direction you are
arguing, and it's on my list of IRIS issues anyway. I can change it: How
do you propose to satisfy the actor-specific requirements for substring
searching with an IDNA-encoded name? This is unsolvable using IDNs, and
running parrallel systems is outrageously expensive and makes the whole
thing fragile. Using a single canonical form for everything makes it
cheap, efficient, and gives people what they paid for anyway. What was
your argument again?

> "no match" or a fuzzy match?

As you describe it, each variant would get an entry, which would exist as
a referral to the canonical entry. The variant query would match the
variant registration and get redirected.

> will have inherently different handling requirements.

I disagree.

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 26 15:47:16 2003

> understand all the IDN implications.  So if anything is not quite
> right, I'm sure we can change it for the better.
> 
> One the search side, IRIS defines two separate searches: 1) find
> domains by name (<findDomainsByName>) and 2) find domains by
> internationalized name (<findDomainsByI18NName>).  I realize that it
> could be one search with a modifier switch, but for the sake of
> simplified server code paths I thought this might be better.  Anyway...

So we have two searches against two data-stores, where all of the entries
and search values require edge normalization.

I don't see any algorithms for normalizing or validating data.

Are errors supposed to be blocked at the client, caught by the server's
query parser, or bounced off the data-store (aka, where should data be
normalized and validated)?

I don't see any requirements for synchronizing the contents of the
entries, or for synchronizing indexes across a shared pool of entries. Is
it okay if a server doesn't return the same answer data for two different
query forms? Is it okay if a server doesn't support one of the query forms
at all?

> lookup by either the 7-bit ASCII name, the 8-bit UTF-8 name, or both.

How does the client know which form is acceptable as user input? On the
other hand, if the client is supposed to normalize regardless of input,
then there needs to be an algorithm.

---

From: leslie at thinkingcat.com (Leslie Daigle)
Date: Tue Aug 26 15:59:21 2003

Eric,

> what bits are in that specific text file. It's the wrong focus point.

Ah.  I didn't realize you were not responding to the points
I had raised.  I was neither focused on operator-operator nor
was I urging the strict use of IDNA formats (let alone
for ~com zone convenience).

> cheap, efficient, and gives people what they paid for anyway. What was

I believe you'll have read Andy's response by now -- separation
in protocol doesn't necessarily mean duplication of representation
in the database.  What was yours, again?

> variant registration and get redirected.

That is one way to implement a fuzzy match for this problem.

>>will have inherently different handling requirements.

Ain't nothin' I can do about that :-)

Leslie.

--

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Tue Aug 26 16:04:34 2003

> and search values require edge normalization.

Why two data stores?  We certainly put both in a single data store.

> normalized and validated)?

Admittedly, the specification for the IDN's is not quite complete... but 
I stated that before.  We do not know all the issues right now.

I good client should stop the user from entering malformed data.

> query forms? Is it okay if a server doesn't support one of the query forms

Shared pools?  Synchronization?  Why do we need to specify this?  How 
the operator stores, synchronizes, backs-up, etc is up to them.

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 26 16:11:27 2003

on 8/26/2003 2:56 PM Leslie Daigle wrote:

> I had raised.

I did respond to your other points, as well as this one ("brush off").

> I was neither focused on operator-operator nor

You were arguing that registry adoption is critical (which we agree on),
and the implicit statement that ~IDNA is critical to registry adoption
(which we certainly don't agree on).

I'm still interested in hearing how parallel systems are supposed to be
more efficient towards this particular point, or towards broader adoption
outside the peer-club in general.

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 26 16:17:29 2003

on 8/26/2003 3:32 PM Andrew Newton wrote:

> Eric A. Hall wrote:
> 
>> So we have two searches against two data-stores, where all of the
>> entries and search values require edge normalization.
> 

Unless you mandate two indexes against one data-store, it will be two
data-stores.

>> Are errors supposed to be blocked at the client, caught by the
>> server's query parser, or bounced off the data-store (aka, where
>> should data be normalized and validated)?
> 
> Admittedly, the specification for the IDN's is not quite complete...
> but I stated that before.  We do not know all the issues right now.

Do you consider this to be a requirements miss?

The other subordinate issue is probably a bigger one. If servers are
allowed to arbitrarily choose not to support the UTF-8 searches, then
substring searches are not possible from all clients against all servers
(modulo the natural issues of ACLs and permissions).

> I good client should stop the user from entering malformed data.

What is malformed for one client would be legitimate for another if you
don't specify the algorithm.

My discussion on escape syntaxes for octet values is a demonstration of
this; in some cases, octet values could be considered as legitimate
charset-codepoints.

>> entries, or for synchronizing indexes across a shared pool of
>> entries. Is it okay if a server doesn't return the same answer data
>> for two different query forms? Is it okay if a server doesn't support
>> one of the query forms at all?
> 

I'm asking if the same answer data must be returned for both queries. This
isn't stated anywhere either.

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Tue Aug 26 17:03:19 2003

> Unless you mandate two indexes against one data-store, it will be two

I do not presume to know how all registries/registrars store their IDN 
data.  Some may store both the originating Unicode and the 7-bit 
version.  Some may store only one and calculate variants.  Some may not 
support IDN's at all.

> don't specify the algorithm.

Specifying that the input should be punycode/nameprepped/whatever should 
give client writers enough direction.

> this; in some cases, octet values could be considered as legitimate

Why are they escaped if the field is UTF-8 capable?

> isn't stated anywhere either.

If you ask for something that is a domain variant, then you get a domain 
variant.  If you ask for something that is a domain, you get a domain. 
I don't understand the issue here?

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 26 17:18:21 2003

> version.  Some may store only one and calculate variants.  Some may not

That's fine, except where some people want to get the contacts for
ex?mple.com and others want to get the contacts for xn--exmple-cua.com.

I'm not saying you need to dictate operational behavioral, but AFAICT you
haven't specified whether or not the queries should even return the same
data, which is the point here not the operational stuff.

>> What is malformed for one client would be legitimate for another if
>> you don't specify the algorithm.
> 
> Specifying that the input should be punycode/nameprepped/whatever
> should give client writers enough direction.

It's a start, but it won't guarantee consistent results. You need to
require two-pass conversions for all input to guarantee that all of the
data in the system is in a consistent form.

>> My discussion on escape syntaxes for octet values is a demonstration
>> of this; in some cases, octet values could be considered as
>> legitimate charset-codepoints.
> 

Again, octet values are legal in domain names. Secondarily, the user side
of the client has a high probability of using something else (UCS-2,
8859-1, 2022-JP, whatever). Cumulatively, the user can enter an octet
value in a local charset which will get mapped to UTF-8 as part of the
canonicalization routines, so data that is explicitly an octet value
(rather than a character at that codepoint) needs to be protected.

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Tue Aug 26 18:00:22 2003

> data, which is the point here not the operational stuff.

If the client is doing a lookup and sends in the punycode, they will get 
a consistent answer.

> data in the system is in a consistent form.

See above.

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 26 18:09:46 2003

on 8/26/2003 5:28 PM Andrew Newton wrote:

> Eric A. Hall wrote:
> 
>> I'm not saying you need to dictate operational behavioral, but AFAICT
>> you haven't specified whether or not the queries should even return
>> the same data, which is the point here not the operational stuff.
> 
> If the client is doing a lookup and sends in the punycode, they will
> get a consistent answer.

Back to square one then. IDNA doesn't work for substring comparisons.

At a minimum, You must specify complete normalization and validation
rules, and you must specify where they will be used.

If you want to have two namespaces, searches, etc., you need to specify
all of this stuff twice.

---

From: hardie at qualcomm.com ([email-masked])
Date: Tue Aug 26 18:31:15 2003

Eric,
	Doesn't this text in firs-contact indicate the same
problem:

5.      Query Processing Rules
      Queries for contact entries have several special requirements, as
      discussed in the following sections.
      Refer to [FIRS-CORE] for general information about FIRS queries.
   5.1.    Query Pre-Processing

      Clients MUST ensure that the query input is normalized according
      to the rules specified in section 3 before the input is used as
      the assertion value to the resulting LDAP query.
      The authoritative partition for a contact entry is determined by
      mapping the domain element of a normalized email address to a
      sequence of domainComponent labels.

      Since the domainComponent attribute is restricted to seven-bit
      characters, the domain element MUST be converted to its IDNA form
      using the "ToASCII" conversion operation specified in [ RFC3490 ],
      with the "UseSTD3ASCIIRules" flag disabled (FIRS applications MAY
      reuse the output from the conversion performed in step 3.c if the
      entire conversion process is known to have completed
      successfully). The resulting sequence of ASCII labels are used to
      form the domainComponent sequence which represents the
      authoritative partition for the email address.
      As a simple example, "[email-masked]" would be mapped to the
      "dc=example,dc=com" authoritative partition, with this partition
      being used to seed the query process.

In other words, if the query pre-processing requires normalization to the
IDNA form using ToASCII, aren't we limited to using matching forms that
work on IDNA for contacts?

Sorry if I'm missing something obvious here.
			regards,
				Ted Hardie

---

From: anewton at ecotroph.net (Andrew Newton)
Date: Tue Aug 26 18:34:55 2003

> rules, and you must specify where they will be used.

Yes.

So it would seem that the <findDomainsByI18NName> search should specify 
that the input is nameprep, not just utf-8.  I think that is your point. 
  In addition, it should allow the option for exact match (something I 
noticed).

I'll make this change in the next rev.

Thanks.

-andy

---

From: ehall at ehsco.com (Eric A. Hall)
Date: Tue Aug 26 19:17:53 2003

on 8/26/2003 5:31 PM [email-masked] wrote:

> Eric,
> 	Doesn't this text in firs-contact indicate the same problem:

>    5.1.    Query Pre-Processing

No. The pre-processing is where the query value gets mapped to a
partition, so that the SRV RRs for the target partition can be located.

The UTF-8 form of the query value is still used for the LDAP queries.