Subject: [URN] more info on edits for NAPTR draft
From: Ron Daniel <[email-masked]>
To: [email-masked]
Date: Thu, 29 Aug 1996 17:16:01 -0600

A note on process - this note delves into details of the
regular expression syntax for the NAPTR draft. This should
*not* be construed as indicating that the group has reached
consensus on the issue Leslie brought up yesterday. (For one
thing, Karen is on travel and will probably not be able to
respond to Leslie's message until next week). I am sending this
to the list becase I want to solicit input on a detail of
implementation if we do go down the regexp path. 

A few days back, Terry Allen said that he couldn't get sed to
work with the regexp examples in the NAPTR draft. He had to
put backslashes in front of the '(' and ')' characters which
are used for remembering subexpressions. He also said that his
implementation of sed didn't support the /i flag we assumed for
case insensitivity.

After a bit of research I have found out why. POSIX defines
two forms of regular expressions -Basic (BRE) and Extended (ERE).
sed is defined to work on BREs, while our implementation set the
flag to use EREs. Among the differences between the two is the
handling of parens. In BREs, we can create subexpressions by
using "\(" and "\)" to delimit the subexpression. Parens are not
special in BREs. Second, we can use the \DIGIT "backref" inside the
regexp itself. For example, the BRE   a\(b\)\1   would match the
string  "abb"   but not the string   "abc".

In contrast, EREs define parens as special characters. We put a backslash
in front of the paren if we want to match a literal paren. Second,
EREs do not have the notion of backrefs within the regexp themself.

My question to the list is which way shall we go? Extracting
subexpressions is a vital part of the rewrite, so we have to
go one way or the other.

In either case, we probably have to go on and provide a grammar
for just what we mean by a "substitution expression. "sed-style"
is not enough. I think the /i flag is very important for us to
provide.  Other flags, such as /p and /w don't seem to have a lot
of meaning in ou application.

I don't know that it makes a lot of difference which way we go.
BREs are closer to sed, but they will require more backslashes.
Since we have to escape the backslash with another backslash
in zone files, this doesn't seem like a large advantage.

Opinions?  Mine is to go with EREs, but it is not strongly held.

Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545    obscure_term: "hypernym"

Date: Tue, 27 Aug 1996 16:41:37 -0600
To: [email-masked] (Leslie Daigle), [email-masked]
From: Ron Daniel <[email-masked]>
Subject: Re: Need Input -- was Re: [URN] Concerns
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Leslie Daigle (at least at 04:33 PM 8/27/96 -0400)
...
>Let me try to summarize how I read the two sides of 
>the point, and draw out of it one issue that I would like to hear responses 
>on -- both affirming and dissenting, so that we can see what the overall 
>feeling is.
>
>	. We can build a URN system that requires some syntactic structuring
>	  of _all_ components of a URN, with a view to providing more 
>	  rigourous (how many u's do we use in that word in Canada?!)
>	  maintenance structures, resolution operations, etc.
>
>	. We can build a URN (resolution) system that relies on federating
>	  heterogeneous name spaces -- both existing and future.  In this
>	  case, the primordial need is to build a system that can accommodate
>	  the needs of a broad spectrum of namespaces with widely divergent
>	  aims and goals.  The goal is to provide some set of standardized
>	  mechanisms for handling URNs of all namespaces, while minimizing
>	  the requirements placed on the structure of the name.
[...]
>The real issue on which there needs to be traffic is which of the two
>paths we are going to follow here -- or creative suggestions for an
>alternate position.  

Well, I don't know how creative this is, but there is a middle ground
to consider. Let me point it out for compleness, then say why I still
like the second of your two items.

Your first point talks about the structuring of "all" components of a
name. I take that to mean that the developer of a namespace could write
some expression that would not only check that the syntax of the namespace
is being followed, but could yank out pieces of the URN and
put them into some canonical order for resolution. This was the gist
of Lewis' proposal. The problem with this is that the designer's knowledge
stops at the boundries of any "opaque string" in the name. Therefore,
the ability to delegate resolution also stops at that point, because we
can't detect components in the opaque string. However, lots of publishers
may want to put structure into their opaque strings and use that
structure in the resolution process for delegation. With the first model,
they don't get to, which is why I find it pretty much unacceptable.

As a middle ground between the two positions, we could have one where
the designer writes an expression that validates the URN and puts its
components into the canonical order up to the opaque string. Then,
when the resolution process has gotten us to a server that corresponds
with that opaque string boundary, we can get a new expression to
check the syntax and put it into the canonical order. (This model should
also allow recursive opaque strings).

I could live with this middle ground if it were the consensus of the
group that it was the correct model, but I still like it less than the
other position that Leslie mentioned, for at least three reasons:
  1) I think that the model where we can do a rewrite at any stage of
     the resolution process gives us enough power to get ourselves out
     of unforseeable jams down the line. I know that I am not smart
     enough to see the future clearly. I would rather approach it with
     a big toolkit than a small one.
  2) I think that the resolution system du jour should have no impact
     whatsoever on the allowed structure of names. What we do to make
     names easier to resolve for today's URN resolution system stands
     an excellent chance of making things more difficult to resolve
     for tomorrow's resolution system. We also stand a chance of not being
     able to handle some namespace because we didn't put the right
     wrench into our toolkit.
  3) I think that it will be more difficult to construct the expressions
     that check syntax, reorder the components, eat components off of the
     canonical order, etc. than it will be to construct the regular
     expression alternatives.  This difficulty stems from 3 areas:
     a) The conceptual model, which I think is much more difficult to
        understand than rewrites of the URN.
     b) The need to manage multiple inputs and outputs. Lewis' proposal
        has programs, rules, URNs, domain names, and the canonical string.
        Programs are what convert the URN into the canonical string. Rules
        are what eat components from the front of the canonical string and
        give the next domain name to look at. In addition to that task,
        the rules leave the canonical string shorter for the next rule.
        In contrast, the model in the NAPTR draft has URNs, domain names,
        and regexps. The next domain name to query can either be supplied
        directly (the replacement field) or can be obtained by a rewrite
        of the original domain name.
     c) The bit about the canonical string is important. Using the
        canonical string puts us into the situation where the input to
        a rule is the output of another. This is something that makes
        rewrite rules confusing, because you can't understand a rule
        without knowing how all preceding rules have affected the input
        string. Also crucial is that if someone upstream in the resolution
        process changes their rule, and changes the handling of a delimiter,
        all of a sudden your rules can be broken. In contrast, the rewrite
        rules in the NAPTR proposal can be understood in isolation, because
        they are applied to the URN and have a single output - a domain name.

        I should note that the canonical string offers a powerful advantage
        in addition to the disadvantages I have just mentioned. Since
        the canonical string shrinks at each stage of the resolution
        process, we have a gurantee of termination. The NAPTR stuff can
        detect cycles, but has no way of avoiding a long slow walk through
        a large "funhouse",  if someone can construct such a space without
        being detected. I believe that this possibility is more remote
        than the one of upstream changes.

All my humble opinion of course,  :-)

Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

From: Michael Mealling <[email-masked]>
Subject: Re: Need Input -- was Re: [URN] Concerns
To: [email-masked]
Date: Tue, 27 Aug 1996 17:25:11 -0400 (EDT)
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Michael Mealling <[email-masked]>

Leslie Daigle said this:
> 	. We can build a URN system that requires some syntactic structuring
> 	  of _all_ components of a URN, with a view to providing more 
> 	  rigourous (how many u's do we use in that word in Canada?!)
> 	  maintenance structures, resolution operations, etc.
> 
> 	. We can build a URN (resolution) system that relies on federating
> 	  heterogeneous name spaces -- both existing and future.  In this
> 	  case, the primordial need is to build a system that can accommodate
> 	  the needs of a broad spectrum of namespaces with widely divergent
> 	  aims and goals.  The goal is to provide some set of standardized
> 	  mechanisms for handling URNs of all namespaces, while minimizing
> 	  the requirements placed on the structure of the name.   
> 

While my answer might be obvious I wanted to iterate on why for a second.
I specifically want the second option for two reasons:

1) It facilitates buy in on a large scale due to the ability of current
publishers to get into the system with low up front cost. If its one thing
we've learned on the net its that if you make it cheap in the beginning they
don't mind getting expensive latter.

2) It allows some of the larger players such as OCLC, Handle, D&B to play
with their namespaces that have a HELL of a lot of stuff already in them.

-MM

-- 
------------------------------------------------------------------------------
Life is a game. Someone wins and someone loses. Get used to it.
<BR>
<HR><A HREF="http://www.gatech.edu/michael.html">Michael Mealling</A>

From: [email-masked] (Leslie Daigle)
Date: Tue, 27 Aug 1996 16:33:28 -0400
To: [email-masked]
Subject: Need Input -- was Re: [URN] Concerns
Sender: [email-masked]
Reply-To: [email-masked] (Leslie Daigle)

I think there is a single point that needs to get resolved before
we can proceed with the work here.  It is illustrated in the exchange
of views between Karen and Ron earlier this week, and to some extent between
Lewis and Ron earlier.  Let me try to summarize how I read the two sides of 
the point, and draw out of it one issue that I would like to hear responses 
on -- both affirming and dissenting, so that we can see what the overall 
feeling is.

	. We can build a URN system that requires some syntactic structuring
	  of _all_ components of a URN, with a view to providing more 
	  rigourous (how many u's do we use in that word in Canada?!)
	  maintenance structures, resolution operations, etc.

	. We can build a URN (resolution) system that relies on federating
	  heterogeneous name spaces -- both existing and future.  In this
	  case, the primordial need is to build a system that can accommodate
	  the needs of a broad spectrum of namespaces with widely divergent
	  aims and goals.  The goal is to provide some set of standardized
	  mechanisms for handling URNs of all namespaces, while minimizing
	  the requirements placed on the structure of the name.   

>From the sounds of discussion, we cannot do both -- the former requires
structuring in the namespace-specific strings, and the latter seeks to include
the many namespaces that cannot be so addressed (existing ones, proprietary
ones, etc).  The latter is too loose in structure to leave hooks for 
sophisticated maintenance, etc.

To regex or not to regex becomes an implementation detail.

The real issue on which there needs to be traffic is which of the two
paths we are going to follow here -- or creative suggestions for an alternate
position.  

I reserve my own comments, including what I _though_ we'd agreed on
to date, for a later posting.

Leslie.

-- 

------------------------------------------------------------------------------

                                                      Leslie Daigle
   "Learn and live."                                  Vice President, Research
                                                      Bunyip Information Systems
                          -- ThinkingCat              (514) 875-8611
                                                      [email-masked]
------------------------------------------------------------------------------

Date: Tue, 27 Aug 1996 14:16:29 -0600
To: Martin Hamilton <[email-masked]>, [email-masked]
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] use of info gleaned from DNS
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Martin Hamilton (at least at 08:33 PM 8/27/96 +0100)
>I don't think there's any mention of this in the current NAPTR
>draft, but it might be worth popping a note into the security
>considerations section about the potential danger of using info
>gotten from the DNS without any checking/santizing ?
>
>e.g. don't just invoke a program using info from an NAPTR RR as
>command line arguments, because there could be some nasty
>characters lurking in there

Right, good point. I'll do that.

As a clarification, the regexp field is not intended
to be given to programs on the command line. It is intended to be
given to a regular expression library that the client has linked to.
However, client implementors will do what they will do, so I will
warn them not to pass the string off to Perl or something similar
without checking it for sanity.

Regards,
 
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

To: [email-masked]
Subject: [URN] use of info gleaned from DNS
Date: Tue, 27 Aug 1996 20:33:00 +0100
From: Martin Hamilton <[email-masked]>
Sender: [email-masked]
Reply-To: Martin Hamilton <[email-masked]>

I don't think there's any mention of this in the current NAPTR
draft, but it might be worth popping a note into the security
considerations section about the potential danger of using info
gotten from the DNS without any checking/santizing ?

e.g. don't just invoke a program using info from an NAPTR RR as
command line arguments, because there could be some nasty
characters lurking in there

Martin

Date: Tue, 27 Aug 1996 13:31:30 -0600
To: [email-masked] (Daniel LaLiberte)
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] NAPTR "wizards"
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Daniel LaLiberte (at least at 11:21 AM 8/27/96 CDT)
>Ron Daniel writes:
> > Actually, at the BOF in Montreal, we explicitly said that URNs are not
> > going to solve the friendly naming problem.
>
>I wasn't there, so I don't know, but I would guess this is a bit
>strong.  Rather, URNs are not *necessarily* going to solve the
>friendly naming problem.  In other words, the system supported by
>NAPTR-based resolution could be used to support friendly names, but
>doing so is beyond the scope of the URN problem, which is unique
>names.

Right, I was imprecise. We are going after the unique names problem.
If they can also be friendly, so much the better. However, we are not going
specifically after the friendly names problem.

>  I would be opposed to a system which made it more difficult
>than necessary to allow unique names to also be friendly.

Agreed.

Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Tue, 27 Aug 1996 13:31:29 -0600
To: [email-masked] (Stu Weibel), [email-masked]
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] NAPTR "wizards"
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Stu Weibel (at least at 02:17 PM 8/27/96 -0400)
>Ron Daniel writes:
>
> > Actually, at the BOF in Montreal, we explicitly said that URNs are not
> > going to solve the friendly naming problem.
>
>Dan LaLiberte writes:
>
>> I would be opposed to a system which made it more difficult
>> than necessary to allow unique names to also be friendly.
>
>I strongly agree with Dan on this point.
>
>We're back to the groundhog day problem... we don't agree what the
>problem is, so we can't agree on the solution.

Whoa, wait a sec. I also agree with what Dan said. There is a big
difference between saying "URNs are intended to solve the unique
name problem, not the friendly naming problem" and saying "URNs must make
friendly names impossible". Dan says that he doesn't want us to mandate
that unique names also be unfriendly. I agree. I want us to mandate as
little as possible about the names themselves.

Where we start having a conflict is that what is normally meant by
"friendly names" are normally not unique, the whole thing with Apple
was to illustrate that point. First-come, first-serve is not the
best algorithm for assiging friendly names, but that is why "apple.com"
gets you Apple computer and not Apple music. If someone has a naming scheme
for assigning unique friendly names we should be able to use it in
deterministic resolution schemes. Non-unique names need additional
information from the user, which is what kicks them out of the
deterministic world into directory service world.

Regards,
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

From: [email-masked] (Leslie Daigle)
Date: Tue, 27 Aug 1996 15:05:22 -0400
To: [email-masked] (Stu Weibel), [email-masked]
Subject: Re: [URN] NAPTR "wizards"
Sender: [email-masked]
Reply-To: [email-masked] (Leslie Daigle)

Well, Stu, there may have been areas in which there was less consensus
than others in Montreal, but I think this actually need not be one of them.

What I understood out of the Montreal meeting is that we agreed to work
on a general uniform resolution system -- i.e., those things, and only
those things, necessary to get heterogeneous naming systems to hang
together so that they could be used in a broad variety of contexts, with
some common expectations about longevity, accessibility, and maintenance.

[Stu wrote:]
> NAPTR-based resolution is solving a RESOLUTION problem, not a unique naming
> problem (the solution to which is far simpler, and for which numerous 
> solutions already exist).

Not just "NAPTR-based", but the point of the framework presented in
Montreal.  Yes, they do tackle the resolution problem, not the naming
problem, _because_ those problems are dealt with elsewhere.  Let PURLs,
FPIs, ISBNs, Handles, etc play in the URN arena, and time and market space
will determine which work best.

So, 

[Ron wrote:]
> Actually, at the BOF in Montreal, we explicitly said that URNs are not
> going to solve the friendly naming problem.

and

[Dan wrote:]
> strong.  Rather, URNs are not *necessarily* going to solve the
> friendly naming problem.  In other words, the system supported by
> NAPTR-based resolution could be used to support friendly names, but
> doing so is beyond the scope of the URN problem, which is unique
> names.  [...]

and both are right.

If we set out with user friendly naming as a goal of the WG, then every
thing we did would have to ensure that it was true for all potential
namespaces (and my user-friendly is your user-abhorrent).  Rather, we
focus on the mechanics, and let individual namespace builders worry about
how user-friendly their users want names to be.

Leslie.

-- 

------------------------------------------------------------------------------

                                                      Leslie Daigle
   "Learn and live."                                  Vice President, Research
                                                      Bunyip Information Systems
                          -- ThinkingCat              (514) 875-8611
                                                      [email-masked]
------------------------------------------------------------------------------

Date: Tue, 27 Aug 1996 12:00:57 -0700 (PDT)
From: "Gregory J. Woodhouse" <[email-masked]>
To: Stu Weibel <[email-masked]>
Cc: [email-masked]
Subject: Re: [URN] NAPTR "wizards"
Sender: [email-masked]
Reply-To: "Gregory J. Woodhouse" <[email-masked]>

On Tue, 27 Aug 1996, Stu Weibel wrote:
[...]
> We're back to the groundhog day problem... we don't agree what the
> problem is, so we can't agree on the solution.
> 
> NAPTR-based resolution is solving a RESOLUTION problem, not a unique naming
> problem (the solution to which is far simpler, and for which numerous 
> solutions already exist).
> 
Good point. Perhaps I could rephrase my initial comment as saying that I
think regexps serve to minimize the extent to which URN schemes depend
upon the specifics of the NAPTR proposal. Ideally, of course, they should
be independent of one another, but from a practical point of view I agree
that regexps provide a high degree of independence without introducing too
much (computational) complexity.

That being said, I only ment to suggest that means may be available to help
alleviate the difficulty of maintaining a regexp based system and making it
less error prone. From the perspective of this group, I only think it is
significant that such means are available. The details are another matter
altogether.

[...]

---
Gregory Woodhouse     [email-masked]
home page:            http://www.wnetc.com/
resource page:        http://www.wnetc.com/resource/

From: Michael Mealling <[email-masked]>
Subject: Re: [URN] NAPTR "wizards"
To: [email-masked]
Date: Tue, 27 Aug 1996 14:56:14 -0400 (EDT)
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Michael Mealling <[email-masked]>

Stu Weibel said this:
> Ron Daniel writes:
>  > Actually, at the BOF in Montreal, we explicitly said that URNs are not
>  > going to solve the friendly naming problem.
> Dan LaLiberte writes:
> > I would be opposed to a system which made it more difficult
> > than necessary to allow unique names to also be friendly.
> 
> I strongly agree with Dan on this point.  
> 
> We're back to the groundhog day problem... we don't agree what the
> problem is, so we can't agree on the solution.
> 
> NAPTR-based resolution is solving a RESOLUTION problem, not a unique naming
> problem (the solution to which is far simpler, and for which numerous 
> solutions already exist).
> 
> I'm not sure who the "we" was in Montreal that decided on the scope and
> specific nature of the problem.  Attempts to clarify this in the BoF itself
> were sloughed off as obstructionist.  So, here we are again.

What we decided was that we weren't going to SPECIFICALLY try and solve
the "the user wants an easy term or phrase to remember so he can reliably
find Apples homepage". That's a directory services problem. If you 
personally want to try and design a URN namespace that tries to solve
this problem then go right ahead. Many of use just don't want to solve
that particular problem as the only problem.

-- 
------------------------------------------------------------------------------
Life is a game. Someone wins and someone loses. Get used to it.
<BR>
<HR><A HREF="http://www.gatech.edu/michael.html">Michael Mealling</A>

From: [email-masked] (Stu Weibel)
Date: Tue, 27 Aug 1996 14:17:31 -0400
To: [email-masked]
Subject: Re: [URN] NAPTR "wizards"
Sender: [email-masked]
Reply-To: [email-masked] (Stu Weibel)

Ron Daniel writes:

 > Actually, at the BOF in Montreal, we explicitly said that URNs are not
 > going to solve the friendly naming problem.

Dan LaLiberte writes:

> I would be opposed to a system which made it more difficult
> than necessary to allow unique names to also be friendly.

I strongly agree with Dan on this point.  

We're back to the groundhog day problem... we don't agree what the
problem is, so we can't agree on the solution.

NAPTR-based resolution is solving a RESOLUTION problem, not a unique naming
problem (the solution to which is far simpler, and for which numerous 
solutions already exist).

I'm not sure who the "we" was in Montreal that decided on the scope and
specific nature of the problem.  Attempts to clarify this in the BoF itself
were sloughed off as obstructionist.  So, here we are again.

stu

Date: Tue, 27 Aug 96 11:21:34 CDT
From: [email-masked] (Daniel LaLiberte)
To: Ron Daniel <[email-masked]>
Cc: [email-masked]
Subject: Re: [URN] NAPTR "wizards"
Sender: [email-masked]
Reply-To: [email-masked] (Daniel LaLiberte)

Ron Daniel writes:
 > Actually, at the BOF in Montreal, we explicitly said that URNs are not
 > going to solve the friendly naming problem.

I wasn't there, so I don't know, but I would guess this is a bit
strong.  Rather, URNs are not *necessarily* going to solve the
friendly naming problem.  In other words, the system supported by
NAPTR-based resolution could be used to support friendly names, but
doing so is beyond the scope of the URN problem, which is unique
names.  I would be opposed to a system which made it more difficult
than necessary to allow unique names to also be friendly.

--
Daniel LaLiberte ([email-masked])
National Center for Supercomputing Applications
http://union.ncsa.uiuc.edu/~liberte/

Date: Tue, 27 Aug 1996 10:00:03 -0600
To: "Gregory J. Woodhouse" <[email-masked]>, [email-masked]
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] NAPTR "wizards"
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Gregory J. Woodhouse (at least at 07:05 PM 8/26/96 -0700)

>people will want to design user friendly
>URN schemes, and I think they should. URNs really serve two purposes: one
>is persistence and the other is to allow resources to have more intuitive
>names. Instead of designing our URN schemes primarily to facilitate
>resolution, we should be willing to make the resolution schemes
>(meta-schemes?) flexible enough to handle user friendly URN schemes.

Actually, at the BOF in Montreal, we explicitly said that URNs are not
going to solve the friendly naming problem. The desire for short, mnemonic,
names conflicts with the desire for unique names. Apple Music vs. Apple
Computer is a classic example. (My town has an elecrical contractor that
goes by the name of Apple Electric). Many of us regard friendly names as a
directory service sort of problem. The user puts in a friendly sort of name,
and gets back a list of possible candidates. They pick which of those they
are interested in, and away they go.

This is not to say that friendly names are unimportant, just that we are
trying to solve a different problem - that of unique names.

Regards,
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Mon, 26 Aug 1996 19:05:40 -0700 (PDT)
From: "Gregory J. Woodhouse" <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: [URN] NAPTR "wizards"
Sender: [email-masked]
Reply-To: "Gregory J. Woodhouse" <[email-masked]>

I can readily sympathize with the concern that regexps are difficult to
design and manage correctly. Doubtless, it's the rewrite rules in sendmail
that gives it such a reputation for being difficult to administer!
Nevertheless, I think the use of regexps in the NAPTR proposal is an
excellent idea. My reason is that people will want to design user friendly
URN schemes, and I think they should. URNs really serve two purposes: one
is persistence and the other is to allow resources to have more intuitive
names. Instead of designing our URN schemes primarily to facilitate
resolution, we should be willing to make the resolution schemes
(meta-schemes?) flexible enough to handle user friendly URN schemes.

Okay, enough polemics. My point was only to argue that the flexibility of
regexps really is needed functionality, even if maintenance will become
much more difficult and error prone. On the other hand, I'd guess that many
URN schemes will be quite straightforward and it should be feasible to
generate the needed regexps (or even entire records) using simple
interactive "wizards" (or "experts" if you prefer Borland). What I propose
is that NAPTR wizards could either be made available as BIND add ons, or
even included in future releases of BIND. This would both make the NAPTR
scheme easier to administer and improve the robustness of the URN schemes
that rely on it.

---
Gregory Woodhouse     [email-masked]
home page:            http://www.wnetc.com/
resource page:        http://www.wnetc.com/resource/

Date: Mon, 26 Aug 1996 01:57:13 -0600
To: "Karen R. Sollins" <[email-masked]>, [email-masked]
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] concerns
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Karen R. Sollins (at least at 07:12 PM 8/23/96 -0400)

>Perhaps we would all be better
>off and have a higher probability of success, if we identified a
>smaller set of namespaces to which we were addressing our attention.
>By limiting the set of namespace schemes we might find that two
>additional concerns might be easier to address.

Sure, but I don't think it will be as much help as you hope.
The primary namespaces I am going after are:

1) CID - MIME ContentIDs.
2) ISO Formal Public Identifiers
   Best known in conjunction with SGML. These little gems have registered
   and unregistered naming authorities, as well as opaque strings.

Both of these have the notion of "opaque strings". Almost any
sort of structural info can be buried in there. That is why I
think we don't achieve much restriction on allowed syntax and
structure by focussing on a small number of name spaces.

In addition to those two main namespaces, there are a few others
that I would like to see us tackle once the first two are
working reasonably well. They are:

3) SICI - Serial Item Contribution Identifier
   Used for identifying journal articles - ISSNs only identify the journal,
   not articles in a journal issue.
4) ISBN - International Standard Book Number
5) ISAN - International Standard Audiovisual Number
6) ISWC - International Standard Work Code  (Music IDs)
   There is a lot of good content already identified with these schemes.
7) OIDs - ISO Object Identifiers
   Lots of things use OIDs as the naming authority part. Constructed OIDs
   allow an opaque ASN.1 "thing" to be appended to the hierarchical
   identifier.
8) http, ...
   NAPTR rewriting also provides a way for us to retrofit replication of
   resources and resolvers onto the existing URL pseudo-namespaces. It is
   less than perfect because there is no good way to ask for the NAPTR
   and/or A records for a domain name, but it is there.

>1) With foreknowledge of some syntactic constraints on URN schemes, we
>could reduce the probability of chaos from the otherwise uncontrolled
>regular expressions needed to parse and transform aspects of schemes.
>Although only a very small number of people have been commenting on
>the mailing list, I heard a great deal of concern about this in
>Montreal.

As mentioned above, there are no syntactic constraints to exploit
in "opaque strings".

What I heard in Montreal were expressions of concern about the
use of rewriting rules in general, not just regular-expression
based ones. I also had the sense that people were of the
opinion that, worrisome as it might be, this level of power seemed
to be needed to meet the requirements.

>2) With less complexity in the overall namespace structuring,
[...]

Again, I doubt that we will see any significant reductions in
complexity after we take a look at the things that people will do
in opaque strings. Those strings will undoubtably have lots of
structure in them, and it will not necessarily follow a simple
hierarchy. As an example, my lab already uses technical report
identifers like:
                   LA-UR-96-0023

LA = Los Alamos
UR = Unclassified Report. We have a number of different report series,
so those characters will change depending on the report series. That is
one place where delegation might occur.
96 = year of publication. I assume we will change to 4 digits RSN. :-)
     This year is another bit of significant info that might be used.
0023 = sequentially assigned number

These are existing identifiers that we at LANL would like to continue
to use, and I happen to think that it is a very reasonable thing for us
to do. I would oppose any effort in this group to limit what people might
be able to assign in their opaque strings.

[With simpler names,]
>migrating to a future set of mechanisms that do not involve regular
>expressions and the DNS might be manageable.

I don't think we can refuse to consider the class of namespaces that
use opaque strings. If we can't eliminate that class of namespaces,
I don't think we can reasonably achive significant simplification in
the names we have to resolve. Preventing people from assigning the
names of their choice pushes expense and effort onto others.

>Again, there was
>significant concern expressed about this in Montreal, although not on
>the mailing list since then.  This is a problem that is not addressed
>in the NAPTR related papers, but should be central to any plan that is
>proposed.

I don't recall much discussion of the migration issue during the BOF. What
I recall is that people were concerned about rewrite rules because they
get hairy over time. The topic of being locked into a rewrite-based
system because of its power was raised by Lewis, I think, in his
presentation but I can't recall it being discussed extensively.
(I could be wrong on this - Lewis came up shortly after I finished
and it takes me awhile to switch from speaker-mode to audience-mode).

You are correct that migration to future resolution systems is not addressed
in the NAPTR draft. I think that document should be a tight specification
of the NAPTR record format and usage. Migration should be (IMHO) handled
in the framework draft. Even there, I don't know that I would regard it
as central. There are other areas of concern, such as security,
replication, and maybe even rights management, that are equally important.
By definition, these concerns have a more pressing time frame than
migration.

We certainly don't want to do things now that are going to come back to
haunt us. Neither do we want to start with such a restricted system that
it can't accomplish its purpose. If we want URNs to start taking over
for URLs in order to achive their benefits, we have to offer a significant
advantage over URLs for J. Random Publisher. I think we have two shots at
that. The first is that Netscape, Microsoft, ... would really like a way to
transparently direct users to mirror sites instead of having the current
hack of listing 10 sites. Our second
chance is by accomodating namespaces such as ISBN that have a lot of
existing, high-quality, content. If we shut ourselves out of either of
those markets, URNs will probably not succeed.

Ugh. Its late - I'm going to bed.

Best regards,
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Fri, 23 Aug 1996 19:12:20 -0400
From: "Karen R. Sollins" <[email-masked]>
To: [email-masked]
Subject: [URN] concerns
Sender: [email-masked]
Reply-To: "Karen R. Sollins" <[email-masked]>

I have a couple of concerns here having to do with scope of the
problem and manageability of the solution.  It seems that in
discussion, whether or not in the text of the documents, the authors
of the NAPTR framework and proposal papers want to provide resolution
for all possible namespace schemes.  Perhaps we would all be better
off and have a higher probability of success, if we identified a
smaller set of namespaces to which we were addressing our attention.
By limiting the set of namespace schemes we might find that two
additional concerns might be easier to address.

1) With foreknowledge of some syntactic constraints on URN schemes, we
could reduce the probability of chaos from the otherwise uncontrolled
regular expressions needed to parse and transform aspects of schemes.
Although only a very small number of people have been commenting on
the mailing list, I heard a great deal of concern about this in
Montreal.

2) With less complexity in the overall namespace structuring,
migrating to a future set of mechanisms that do not involve regular
expressions and the DNS might be manageable.  Again, there was
significant concern expressed about this in Montreal, although not on
the mailing list since then.  This is a problem that is not addressed
in the NAPTR related papers, but should be central to any plan that is
proposed.

I realize that these statements are not concrete.  That is
intentional, in order that we begin to set out the scope of what we
want to do as an alternative to discussing mechanism with some assumed
but perhaps not well stated scope in mind.

			Karen Sollins

___________________________________________________________________
Karen R. Sollins				[email-masked]
Research Scientist				Phone: 617/253-6006
M.I.T. Laboratory for Computer Science		Fax:   617/253-2673
545 Technology Square
Cambridge, MA 02139

From: [email-masked] (Leslie Daigle)
Date: Fri, 23 Aug 1996 13:48:18 -0400
To: Lewis Girod <[email-masked]>, [email-masked]
Subject: Re: [URN] re URN rewriting
Cc: [email-masked]
Sender: [email-masked]
Reply-To: [email-masked] (Leslie Daigle)

[Lewis Girod wrote:]
> OK, that makes three disparate interpretations of these
> requirements...  perhaps a clarification by the authors is in order?

Indeed -- and my apologies for the delay in replying.  I've been out of
town.

First, some comments on the draft itself, and then some opinions.

[The section of the draft in question:]
>   ----------------------------------------
>   Namespace Identifier Requirements and 
>   Administration
>   ----------------------------------------
> 
>   Some set of criteria must be established to govern what can and cannot
>   be used as a (top-level) namespace (NID assignment).  This can include:
> 
> 	. demonstration of an established namespace management system
> 	  (including an overview of mechanisms for preventing the
> 	  duplication/reassignation of name identifiers.  These mechanisms
> 	  are particular to the individual namespaces).
> 	. multi-organization participation in the naming system
> 	. rules on how names are assigned, name assignment authority
> 	  delegation
> 	. provision of escape clause 
> 
>                   (from draft-daigle-urnframework-00.txt)

Ron is right in saying that these were proposed as more political level
requirements than technically-implementable ones.  They are rather
rough comments at this point, flagged as an area requiring more work.
The reason they came up is out of a discussion about registering namespaces,
what organizations would have to do in order to demonstrate viability,
etc.  This is why things like "multi-organization participation in the
naming system" was put there.

For similar requirements, see the Postel draft on new International Top Level
Domains.

[From Lewis' message:]
> 	. rules on how names are assigned, name assignment authority
> 	  delegation
>I interpret this as requiring a
>specification of the ways in which collections can be delegated within
>the syntax of the URNs.  The only way I can see to make this work is
>to somehow embed the rules of delegation into the URN syntax so that
>they can be verified syntactically.

The point has been made before that not every namespace is going to want
to publish how it assigns names.  At the time a namespace is declared, 
it should be possible to demonstrate that names will be assigned in
some reasonable fashion (that will ensure no name collision, etc).

[Ron wrote:]
> Hmm. That is not how I interpreted it. I was thinking that it would
> be more of a definition of the business rules that would be followed
> when one organization
>   1) Authorized a sub-naming-authority
> or
>   2) Sold off some of their assigned names to some other
>      organization.

That was certainly part of the original intention.

[Lewis wrote:]
> Perhaps my interpretations are outside of the sense of the
> requirements, or perhaps the requirements simply haven't been
> specified clearly... in any case, I think we need to make sure that

Agreed -- this does need to be made more clear, and the exact requirements
have to be determined.

And now, I will slip gently into opinion...

[Lewis also wrote:]
> I interpreted point one to imply something concrete in the way of a
> ``management system''.  For example, demonstration of administrative
> software that name assignment authorities would be given to maintain
> the databases is a requirement I had in mind.  That was why I was
> assuming that people would have a rule compiler (although it really is
> not that hard to implement!).  A decompiler would be nice for
> emergencies, but I was thinking that the rules would be kept in and
> installed from the source format, so to debug things you would go back
> to the source file.

I may have fallen off the conversational thread here, but are you suggesting
that because name assignment is done by algorithm, we should be able to 
have (naming) rule compilers and decompilers?  

In my recollection of how we've gotten to where we are, the purpose of
the framework was to build as little common structure as necessary to
hook together a very wide collection of naming systems into something
that could behave in a roughly unified and uniform fashion -- at least
to the extent of letting software handle URNs in a reasonable fashion to
get to the space that does final resolution.

I do not believe we can be in the business of defining _any_ set of rules
that will encompass all the ways in which names will be assigned.  Dot.  Period.

[Ron wrote:]
> authorities what to do. I am not opposed to a modification of your
> proposal that essentially says "once you get to the opaque string,
> the rules of hierarchy may change, and naming authorities are free to
> give you a new set of rules by which you can follow the new hierarchy".

A sort of "algorithmic redirect"?

[Terry Allen wrote:]
> And these requirements:
	[see quote from framework draft]
>
> are, I believe, out of scope.  The point is to find ways to resolve those
> naming schemes that humanity creates.  It is up to the name space owner
> to provide resolution, and not up to the top-level registry to limit how
> that resolution may be provided.

While I agree that we must do our best to handle as many naming systems
as possible, I do think it is reasonable to put some practical bounds on
what it means to be URN-compliant.    

Cheers!
Leslie.

-- 

------------------------------------------------------------------------------

                                                      Leslie Daigle
   "Learn and live."                                  Vice President, Research
                                                      Bunyip Information Systems
                          -- ThinkingCat              (514) 875-8611
                                                      [email-masked]
------------------------------------------------------------------------------

From: Terry Allen <[email-masked]>
Date: Tue, 20 Aug 1996 14:44:14 PDT
To: "Ronald E. Daniel" <[email-masked]>, [email-masked]
Subject: Re: [URN] revision 1 of the NAPTR draft
Sender: [email-masked]
Reply-To: Terry Allen <[email-masked]>

| From [email-masked] Tue Aug 20 14:27:44 1996
| Received: from ruby.ora.com (ruby.ora.com [198.112.208.25]) by rock.west.ora.com (8.6.13/8.6.11) with ESMTP id OAA11790 for <[email-masked]>; Tue, 20 Aug 1996 14:27:42 -0700
| Received: from acl.lanl.gov (acl.lanl.gov [128.165.147.1]) by ruby.ora.com (8.6.13/8.6.11) with ESMTP id RAA04724 for <[email-masked]>; Tue, 20 Aug 1996 17:25:52 -0400
| Received: from legiron.acl.lanl.gov (legiron.acl.lanl.gov [128.165.147.188]) by acl.lanl.gov (8.7.3/8.7.3) with SMTP id PAA26186 for <[email-masked]>; Tue, 20 Aug 1996 15:27:33 -0600 (MDT)
| Message-Id: <[email-masked]>
| X-Sender: [email-masked]
| X-Mailer: Windows Eudora Pro Version 2.2 (32)
| Mime-Version: 1.0
| Content-Type: text/plain; charset="us-ascii"
| Date: Tue, 20 Aug 1996 15:32:57 -0600
| To: Terry Allen <[email-masked]>
| From: Ron Daniel <[email-masked]>
| Subject: Re: comments on draft
| Status: OS
| 
| Here are you original comments - I've flagged the ones
| I have tried to address.
| 
| 
| At 12:34 PM 8/20/96 PDT, you wrote:
| >This is pretty clean and clear.  A few assorted comments:
| >
| >| "Must" or "Shall" - Software that does not behave in the mannar that this
| >|   namespace to another agency. Evaluating the NAPTR recors in the correct
| >
| >typos
| 
| Got em.
| 
| >
| >All quite clear to here,
| >
| >| * The replacement field is one of two fields used for the rewrite rules.
| >|   If the rewrite is a simple substitution of a domain name, that name is
| >|   given in the replacement field. The replacement field is a domain name
| >|   (subject to compression if a DNS sender knows that a given recipient is
| >|   able to decompress names in this RR type's RDATA field). If the rewrite
| >|   is more complex than a simple substitution of a domain name, the
| >|   replacement field should be set to . and the regexp field used.
| >
| >You might by way of emphasis say "simple substitution of a domain name
| >for the entire domain name submitted".  If that's what you mean.
| 
| I've tried something a little different on that point.
| >
| >| * The regexp field is the other field that may be used for rewrite rules.
| >|   The regexp field is a String containing sed-style substitution expressions.
| >
| >Expressions in the plural suggests a script with multiple regexps.  Not that
| >I'd read this carefully when I wrote you earlier today.
| 
| now is "a sed-style regular expression".
| 
| >|   These are applied to the original URN and yield the next domain name
| >|   to be queried.
| >
| >So that the server could supply (as a terminal NAPTR?) regexps that extracted 
| >info from the opaque part of the URN, if it knows they would work right
| with all 
| >opaque parts of all its URNs.  But this is not what you want for
| size/performance
| >reasons.  
| >
| >Now suppose I can redirect all NAPTR inquiries to my name space with a
| >script of 64 lines (64 regexps).  Will it actually be more efficient
| >to return 64 NAPTR records or one long one?  Or does it come to the
| >same thing?  (I dunno, just a thought; you guys are the architects.)
| >
| >| The first step in the resolution process is to find out about the DUNS
| >| namespace. The namespace identifier, duns, is extracted from the URN,
| >| prepended to urn.net, and the NAPTRs for duns.urn.net looked up. It might
| >
| >For us know-nothings, you might say where this first query goes.  As a child, 
| >I thought that Station Identification was some central studio that individual
| >stations invoked from time to time.  But there isn't any place called urn.net,
| >right?  Where is duns.urn.net?
| >
| 
| No action on any of the preceding points.
| 
| >| Since our example RR specified the "s" flag, it was terminal. Our
| >| next action is to lookup SRV RRs for _rcds._udp.isi.dandb.com, which
| >
| >You need to say "as we can't use dunslink, our next action ..." ?
| 
| Got it.
| 
| >
| >| conjunction with a long TTL for *.urn.net records, the average number
| >| of probes to DNS for resolving DUNS URNs would approach one.
| >
| >| ;;                  order pref flags service replacement       regexp
| >| cid.urn.net IN NAPTR 100  10   ""     ""       .        "/.+@([^@]+)/\1/i"
| >
| >I tried that regexp with sed:
| >
| >% sed 's/.+@([^@]+)/\1/i' < foo
| >sed: ``\digit'' out of range: s/.+@([^@]+)/\1/i
| >
| >but:
| >
| >% sed 's/.*@\([^@]*\)/\1/' < foo
| >mordred.gatech.edu
| >
| >where foo is urn:cid:[email-masked]
| >
| >My sed(1), under Sun OS 5.4 and 4.1 (where it's sed (1v))
| >doesn't have an /i anyway, useful though that would be.
| 
| Added additional caveats about '(', '*', etc. But point about
| your sed not having /i is still valid. Please raise it and we
| can see if anyone on the list has the POSIX standard handy.
| 
| >
| >| The flags field tells us that this is the last NAPTR patterns we
| >| should see, and after the rewrite (a simple replacement in this case) we
| >| should look up SRV records to get information on the hosts 
| >| that can provide the necessary service.  
| >
| >If that fails, what do I do, ask for NAPTR records from the same
| >places I asked for SRV records?
| >
| 
| No action.
| 
| >| Flags
| >|        A String giving flags to control aspects of the rewriting. Flags
| >|        are single characters from the alphabet [A-Z0-9]. The case of
| >
| >"are single alphanumeric characters"
| >
| 
| You might suggest "from the set of characters [A-Z0-9]".
| 
| >|        the alphabetic characters is not significant.
| >| 
| >|        At this time only two flags, "S" and "A", are defined. "S" means
| >|        that the next lookup should be for SRV records instead of NAPTR
| >|        records. "A" means that the next lookup should be for A records.
| >|        The S and A flags are mutually exclusive, and resolution libraries
| >|        SHOULD signal and error so that bad NAPTRs can be quickly detected.
| >
| >"signal an error to the server"?  does the client care?  
| >
| >| shells, there is a good chance that when you think you are saying "\\" you
| >| are actually saying "\".
| >
| >And you may need to say \\\ or \\\\ to get things through your shell.
| 
| Here is where the additional caveats on '('et. al went.
| >
| >| The "a" flag allows the next lookup to be for A records rather than
| >| SRV records. Since there is no place for a port specification in the
| >| NAPTR record, when the "A" flag is used the specified protocol must
| >| be running on its default port. 
| >
| >That seems suboptimal.  Why not make a place for it?  (Any good
| >reason will suffice.)
| >
| >|     Once a NAPTR has "matched" a URN, the client MUST NOT consider
| >|      any NAPTRs with a higer value of order than that of the matching NAPTR,
| >
| >typo.  And NAPTRs don't match URNs but rather domain names (right?).
| 
| Got the typo.
| 
| >|      (A match is defined as:
| >|       1) The NAPTR provides a replacement domain name
| >|       2) The regular expression matches the URN
| >|      )
| >
| >"The result of applying the regular expression matches the URN."
| 
| Ugh. And I disagree with you on "match" as I said earlier. I still
| think I am right on this, but we can let the list judge.
| 
| >
| >|   -  If a record at a particular order matches the URI, but the client
| >|      doesn't know the specified protocol and service, the client should
| >|      continue to examine records that have the same order. The client
| >|      MUST NOT consider records with a higher value of order. This is
| >|      necesary to make delegation of portions of the namespace work.
| >
| >typo.
| 
| Got "necessary".
| 
| >  And, this rationale could well be inserted above, at "Once a NAPTR".
| >Client writers will do what they want so long as 
| >they don't feel the need to claim conformance, so supplying the 
| >rationale is a good way to persuade them to do it right.  Hey,
| >it worked with Netscape, right?
| 
| I added a bit to "Once a NAPTR...".
| >
| >|   -  When multiple RRs have the same "order", the client should use
| >|      the value of the preference field to select the next NAPTR to
| >|      consider. However, because of preferred protocols or services,
| >|      estimates of network distance and bandwidth, etc. clients
| >|      may use different criteria to sort the records.
| >
| >That's unclear.  What does "sort" mean here?  It's the first time
| >in the document that the word appears in text, rather than code.
| 
| Changed "sort" to "select the next record to process".
| 
| >|   -  If the lookup after a rewrite fails, clients are strongly encouraged
| >|      to report a failure, rather than backing up to pursue other rewrite
| >|      paths.
| >
| >Answer to my earlier question; might say this earlier.
| 
| Not real clear on this comment.
| 
| >
| >
| >Regards,
| >
| >-- 
| >       Terry Allen      O'Reilly & Associates, Inc.   [email-masked]
| > "In going on with these experiments, how many pretty systems do we build,
| >  which we soon find ourselves obliged to destroy?"  -  Benjamin Franklin
| >        A Davenport Group sponsor     http://www.ora.com/davenport/
| >
| >
| Ron Daniel Jr.                       email: [email-masked]
| Advanced Computing Lab               voice: +1 505 665 0597
| MS B287                                fax: +1 505 665 4939
| Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
| Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"
| 
| 

Regards,

-- 
       Terry Allen      O'Reilly & Associates, Inc.   [email-masked]
 "In going on with these experiments, how many pretty systems do we build,
  which we soon find ourselves obliged to destroy?"  -  Benjamin Franklin
        A Davenport Group sponsor     http://www.ora.com/davenport/

Date: Tue, 20 Aug 1996 14:46:04 -0400
From: Lewis Girod <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: [URN] re URN rewriting
Sender: [email-masked]
Reply-To: Lewis Girod <[email-masked]>

   And these requirements:

   |   ----------------------------------------
   |   Namespace Identifier Requirements and 
   |   Administration
   |   ----------------------------------------
   |   Some set of criteria must be established to govern what can and cannot
   |   be used as a (top-level) namespace (NID assignment).  This can include:
   | 	. demonstration of an established namespace management system
   | 	  (including an overview of mechanisms for preventing the
   | 	  duplication/reassignation of name identifiers.  These mechanisms
   | 	  are particular to the individual namespaces).
   | 	. multi-organization participation in the naming system
   | 	. rules on how names are assigned, name assignment authority
   | 	  delegation
   | 	. provision of escape clause 

   are, I believe, out of scope.  The point is to find ways to resolve those
   naming schemes that humanity creates.  It is up to the name space owner
   to provide resolution, and not up to the top-level registry to limit how
   that resolution may be provided.

OK, that makes three disparate interpretations of these
requirements...  perhaps a clarification by the authors is in order?

	- Lewis

From: Terry Allen <[email-masked]>
Date: Mon, 19 Aug 1996 08:22:26 PDT
To: [email-masked]
Subject: [URN] re URN rewriting
Sender: [email-masked]
Reply-To: Terry Allen <[email-masked]>

Lewis wrote:

| There is some suggestion in the framework document that there be a set
| of requirements of new name schemes; any ``enforcement'' is limited to
| that sort of thing.  We have not yet figured out what requirements are
| to be applied, but clearly names will need to be verified somewhere to
| ensure that they are satisying them -- otherwise there might as well
| be no requirements.  So the question becomes, what requirements do we
| want to set?  One possible requirement for NAPTR classic might be that
| the syntax be defined and checked.  What I am suggesting boils down to
| specifying the hierarchy scheme and ensuring that the NAPTR system
| escape names to alternate systems as soon as the hierarchy spec is
| invalidated.

This seems to limit unnecessarily what the NAPTR system can be made to
do, and thus is not an attractive alternative to regexps.  

And these requirements:

|   ----------------------------------------
|   Namespace Identifier Requirements and 
|   Administration
|   ----------------------------------------
|   Some set of criteria must be established to govern what can and cannot
|   be used as a (top-level) namespace (NID assignment).  This can include:
| 	. demonstration of an established namespace management system
| 	  (including an overview of mechanisms for preventing the
| 	  duplication/reassignation of name identifiers.  These mechanisms
| 	  are particular to the individual namespaces).
| 	. multi-organization participation in the naming system
| 	. rules on how names are assigned, name assignment authority
| 	  delegation
| 	. provision of escape clause 

are, I believe, out of scope.  The point is to find ways to resolve those
naming schemes that humanity creates.  It is up to the name space owner
to provide resolution, and not up to the top-level registry to limit how
that resolution may be provided.

Were the top-level registry to decline to include a naming scheme,
people who want to resolve URNs employing that naming scheme would have to find
alternative methods, and "top-level" would cease to be the top level.

| For example, consider the third point; I interpret this as requiring a
| specification of the ways in which collections can be delegated within
| the syntax of the URNs.  The only way I can see to make this work is
| to somehow embed the rules of delegation into the URN syntax so that
| they can be verified syntactically. 

Your requirements might be made into a document that recommends ways to 
construct naming schemes that are easy to resolve by methods presently 
in use or contemplated, but to the extent that your alternative
to regexps is intended to exclude schemes that don't meet these or other
criteria from ever being used, it's not useful.  

Two further points:

- it's unwise to overload the syntax of URNs, period.

- delegation and all other matters of URN management policy will be
  decided in the real world (lawyers included), rather than by anyone
  now aware of this discussion.
  
Ergo, there is no need to, and it is not a good idea to, attempt to
enforce policy within the framework of URN resolution.

I agree with Ron's remark:

| I am opposed to any proposal which allows only the hierarchy set forth
| by the namespace designer. Lots of namespaces will be of the
| form <hierarchical-naming-authority><delimiter><opaque-string>. The
| string is not opaque if the namespace designer can tell the naming
| authorities what to do. I am not opposed to a modification of your
| proposal that essentially says "once you get to the opaque string,
| the rules of hierarchy may change, and naming authorities are free to
| give you a new set of rules by which you can follow the new hierarchy".

which is in accord with what the real world will do.

Regards,

-- 
       Terry Allen      O'Reilly & Associates, Inc.   [email-masked]
 "In going on with these experiments, how many pretty systems do we build,
  which we soon find ourselves obliged to destroy?"  -  Benjamin Franklin
        A Davenport Group sponsor     http://www.ora.com/davenport/

Date: Sun, 18 Aug 1996 21:13:20 -0600
To: [email-masked] (Daniel LaLiberte)
From: Ron Daniel <[email-masked]>
Subject: [URN] Re: Iterative lookup
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Daniel LaLiberte (at least at 12:33 AM 8/17/96 CDT)

>Thanks for the example, Ron.  That makes it abundantly clear.

Glad to do it.

> > I think it is pretty rare that we will have a namespace that
> > has such a mapping to DNS.
>
>The path scheme requires exactly this kind of iteration and reversal
>of names to get DNS names. 

I didn't say we would *never* have such a namespace.  :-)

> > [...]  I think most namespaces
> > will very quickly point the user to a database that handles
> > very large chunks of the namespace.
>
>Depends on what you mean by this.  If you mean that after the NAPTR
>resolution algorithm is done, the opaque string is handled
>internally to a server, then I think this is the wrong approach
>because it is unscalable for really large name spaces and large
>numbers of users.

That is what I mean, and I agree that it has some problems for some
really large namespaces. However, some of the namespaces I have
been investigating (the Intl. Std. Audiovisual Number in particular)
are centrally assigned and they want them centrally resolved. Name
spaces like the handle names space will probably also want to stop
using NAPTRs very quickly and start talking the handle protocol.

(I'm not so sure that there are problems for large numbers of users,
we can replicate the resolver because of the SRV record).

>It's OK with me that some name spaces work this way, but we should
>not require it of all name spaces.  

I wouldn't dream of making everyone do it. I just think there will
be a number of namespaces that are resolved that way. I also think
that the more money is at stake, the more centralized the name
assignment and resolution will be.

Regards,
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Sat, 17 Aug 96 00:33:18 CDT
From: [email-masked] (Daniel LaLiberte)
To: Ron Daniel <[email-masked]>
Cc: [email-masked]
Subject: [URN] Iterative lookup
Sender: [email-masked]
Reply-To: [email-masked] (Daniel LaLiberte)

Thanks for the example, Ron.  That makes it abundantly clear.

Ron Daniel writes:
 > So, that is one way it could be done. I am not wild about the
 > number of lookups that are needed, but I like it better than
 > adding a non-standard flag to regexps.

Most of the info will be cached locally if you are resolving a URN
that is similar to previous URNs.  This is proportional to the
computational complexity and network/server load of gethostbyname for
every single DNS lookup.  So this should not be considered an argument
against iteration.  Yes, servers and DNS requests are optimized for
gethostbyname lookups, but I think the difference is relatively
minor.

 > I think it is pretty rare that we will have a namespace that
 > has such a mapping to DNS.

The path scheme requires exactly this kind of iteration and reversal
of names to get DNS names. 

 > [...]  I think most namespaces
 > will very quickly point the user to a database that handles
 > very large chunks of the namespace.

Depends on what you mean by this.  If you mean that after the NAPTR
resolution algorithm is done, the opaque string is handled
internally to a server, then I think this is the wrong approach
because it is unscalable for really large name spaces and large
numbers of users.  The basic problem is that there is no way for
clients to know about a subdivision of the name space, should
subdivision become necessary in the future, without asking the server
for every single name.  What the client needs is a rule it can cache
and use later to subdivide the space when resolving another name.
This means that the previously opaque string is no longer opaque
because clients must be responsible for resolving it using the rule.

It's OK with me that some name spaces work this way, but we should
not require it of all name spaces.  

dan

Date: Fri, 16 Aug 1996 16:59:47 -0600
To: Lewis Girod <[email-masked]>
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] nasty rewriting rules
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Lewis Girod (at least at 06:12 PM 8/16/96 -0400)

>I think the question of what sorts of requirements are desirable has a
>significant impact on the likelihood that new name schemes would be
>servable by future systems.  I had inferred some stronger requirements
>than prevention of name reassignment from the framework document:
[...]
>	. rules on how names are assigned, name assignment authority
>	  delegation
>                  (from draft-daigle-urnframework-00.txt)
[...]
>I interpret this as requiring a
>specification of the ways in which collections can be delegated within
>the syntax of the URNs.  The only way I can see to make this work is
>to somehow embed the rules of delegation into the URN syntax so that
>they can be verified syntactically.

Hmm. That is not how I interpreted it. I was thinking that it would
be more of a definition of the business rules that would be followed
when one organization
  1) Authorized a sub-naming-authority
or
  2) Sold off some of their assigned names to some other
     organization.
The case of (2) is the most problematic. Publishers sell off particular
titles all the time, they also sell off whole blocks of titles (the works
of an author, all the titles in one of their business units, ..). So,
it seems important to be able to say things like
  urn:cid:[email-masked] is resolvable at foo.bar.com
  any other title from classics.penguin.com is resolvable at here.there.org
  and all other penguin titles are resolvable at catalog.penguin.com.

Some sites (mine perhaps) might have titles like
  urn:cid:[email-masked]
(la-ur is Los Alamos Unclassified Report. 96- is the year.) We might
want to put the resolver for all reports before 1980 on one machine,
all reports after 1980 on another. So, with regexps we can say
  pref = 10;  s/^(urn:)?cid:la-ur-[4-7]*/old.lanl.gov/i
  pref = 100  new.lanl.gov
The point of this example is that we have some hierarchy encoded in
our "opaque string". That hierarchy can not be known to the namespace
designer. However, being able to switch on it is (I think) important.

I am opposed to any proposal which allows only the hierarchy set forth
by the namespace designer. Lots of namespaces will be of the
form <hierarchical-naming-authority><delimiter><opaque-string>. The
string is not opaque if the namespace designer can tell the naming
authorities what to do. I am not opposed to a modification of your
proposal that essentially says "once you get to the opaque string,
the rules of hierarchy may change, and naming authorities are free to
give you a new set of rules by which you can follow the new hierarchy".

>As I see it, the primary function
>of my proposal is that the translation step and the limited nature of
>the rules after that enforces a delegation structure based on a
>definition at the top level.  At the top level the delegation
>hierarchy is ordered canonically and fixed; below that it cannot be
>processed out of order [...]

Right. That's just what I object to I'm afraid.

> [...] except by escaping to a separate resolution
>mechanism.

OK, so what do you mean by this? Is a marker that allows a
new program to canonicalize the opaque string OK, or do
you just want to say that all opaque strings from a naming
authority will remain opaque for all time?

>I interpreted point one to imply something concrete in the way of a
>``management system''.  For example, demonstration of administrative
>software that name assignment authorities would be given to maintain
>the databases is a requirement I had in mind.

Right, I was assuming something different - a legal contract that
would say naming authorities would not re-issue names and that
they would put the same requirement on any sub-naming authorities
they authorized.

>Because regexps allow a lot of flexibility downstream it is
>hard to make requirements, and thus the potential exists for dowstream
>administrators to make messes.

Well, I think we have an instance of conservation of nightmares here.
Being able to apply the regexp at every stage of the lookup can
lead to messes. Not being able to change the rules at the opaque string
boundaries leads to a ball-and-chain namespace that can't accomodate
the reasonable desires of a lot of users. (The LA-UR example above
is real. We have such a report series - actually we have several
report series - and most of them put a year into the report number).

But this is more than enough for a Friday afternoon.

Best regards,
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Fri, 16 Aug 1996 18:12:45 -0400
From: Lewis Girod <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: [URN] nasty rewriting rules
Sender: [email-masked]
Reply-To: Lewis Girod <[email-masked]>

   on Date: Thu, 15 Aug 1996 08:29:34 -0600 Ron Daniel wrote:

   Thus spoke Lewis Girod (at least at 03:54 PM 8/13/96 -0400):

   >Fair enough, although it is important to have agreement as to what
   >flavor of regexps are being used.  The original plan was to just
   >execute sed, which solves both issues on UNIX platforms, but requires
   >a little more work otherwise.

   Um, no, that was not the plan. The plan was to use regexp libraries, the
   syntax of the regexps was "sed-style" rather than "perl-style" because
   we believe that sed-style is actually the version most regexp libraries
   support.

OK.

   >There is some suggestion in the framework document that there be a set
   >of requirements of new name schemes

   Well, I thought that those requirements were going to be things like
   no reassigning of naming authorities, making sure that any naming
   authorities in a naming scheme require any sub-naming authorities to follow
   rules at lesat as restrictive as the ones they operate under, ...

   So, these really are requirements on naming schemes and the operations
   behing them. Such requiremnts can't be tested by validating the syntax of
   a URN - useful thought that might be.

	[...]

   >Humans are supposed to use the simple user-friendly
   >syntax that compiles into the terse form.  With decent management
   >software the underlying terse form should be transparent to the
   >administrator -- in fact, if we can assume that everyone is using a
   >compiler it might be a good idea to make the terse form even more
   >compact.

   Those are pretty strong assumptions Lewis. Furthermore, this software
   also has to decompile the terse form into the friendly form so that
   people can debug existing records. This sort of friendly software
   rarely gets written and maintained.

I think the question of what sorts of requirements are desirable has a
significant impact on the likelihood that new name schemes would be
servable by future systems.  I had inferred some stronger requirements
than prevention of name reassignment from the framework document:

  ----------------------------------------
  Namespace Identifier Requirements and 
  Administration
  ----------------------------------------

  Some set of criteria must be established to govern what can and cannot
  be used as a (top-level) namespace (NID assignment).  This can include:

	. demonstration of an established namespace management system
	  (including an overview of mechanisms for preventing the
	  duplication/reassignation of name identifiers.  These mechanisms
	  are particular to the individual namespaces).
	. multi-organization participation in the naming system
	. rules on how names are assigned, name assignment authority
	  delegation
	. provision of escape clause 

                  (from draft-daigle-urnframework-00.txt)

For example, consider the third point; I interpret this as requiring a
specification of the ways in which collections can be delegated within
the syntax of the URNs.  The only way I can see to make this work is
to somehow embed the rules of delegation into the URN syntax so that
they can be verified syntactically.  As I see it, the primary function
of my proposal is that the translation step and the limited nature of
the rules after that enforces a delegation structure based on a
definition at the top level.  At the top level the delegation
hierarchy is ordered canonically and fixed; below that it cannot be
processed out of order except by escaping to a separate resolution
mechanism.  An interesting property of my proposal is that there are
no real syntactic requirements beyond those needed to re-order the
URN.

As you point out, this particular effect cannot be obtained using the
regexp rules, even with an initial syntactic check at the beginning.

I interpreted point one to imply something concrete in the way of a
``management system''.  For example, demonstration of administrative
software that name assignment authorities would be given to maintain
the databases is a requirement I had in mind.  That was why I was
assuming that people would have a rule compiler (although it really is
not that hard to implement!).  A decompiler would be nice for
emergencies, but I was thinking that the rules would be kept in and
installed from the source format, so to debug things you would go back
to the source file.

Perhaps my interpretations are outside of the sense of the
requirements, or perhaps the requirements simply haven't been
specified clearly... in any case, I think we need to make sure that
the requirements that we do want are at least partially enforced in a
technical way by the system.  (We may need to figure out what they are
first?)  Because regexps allow a lot of flexibility downstream it is
hard to make requirements, and thus the potential exists for dowstream
administrators to make messes.

-lewis

Date: Fri, 16 Aug 1996 16:00:43 -0600
To: [email-masked] (Daniel LaLiberte)
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] revision of NAPTR draft
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Daniel LaLiberte (at least at 03:26 PM 8/16/96 CDT)

[Stuff on inability of regexps to revese order of components
 in an arbitrary length list deleted]

>In previous discussion with Ron, he suggested that it might be
>possible to use the resolution algorithm, as is but iteratively, to
>bite off a component at a time and construct the reversed DNS name.
>This sounds similar to what we recommend for the path scheme
>resolution in the first place, so I would be happy to go with it - it
>wouldnt require a list reversal extension.  But I don't see how it can
>be done.  Could you give an example of how this works?

Sure. Lets look at a semi-hypothetical name like
  janet:uk.ac.cam.eng:techrep-96-23

First, we extract the namespace identifier, janet, and
lookup(NAPTR, janet.urn.net) => a regexp like
  "s/(urn:)?janet:(.[^\.])*\.*/\2/i"
which matches everything after "janet:" but before the next "."

Now we lookup(NAPTR, "uk") => a regexp like
  "s/(urn:)?janet:uk\.(.[\.])*\..*/\2.uk/"
which matches everything after "janet.uk." but before the next ".".

Now we lookup(NAPTR, "uk.ac") => a regexp like
  "s/(urn:)?janet:uk\.ac\.(.[^\.])*\..*/\2.ac.uk/"

lookup(NAPTR, cam.ac.uk) =>
  "s/(urn:)?janet:uk\.ac\.cam\.(.[^\.])*\..*/\2.cam.ac.uk/"

lookup(NAPTR, eng.cam.ac.uk) =>
  terminal naptr telling us to fetch the SRV record for
  something like _ftp._tcp.eng.cam.ac.uk, with the SRV
  and A records included as additional info.

We pick one of those servers and ask it about techrep-96-23.

So, that is one way it could be done. I am not wild about the
number of lookups that are needed, but I like it better than
adding a non-standard flag to regexps.

I think it is pretty rare that we will have a namespace that
has such a mapping to DNS. For instance, an OID namespace
has its most significant components at the front of the name.
However, there is no good way to map 1.3.6.1.4.1.2 to ibm.com
and map 1.3.6.1.4.1.13 to utk.edu.  I think most namespaces
will very quickly point the user to a database that handles
very large chunks of the namespace. This is certainly how
names from schemes such as ISAN - Intl. Standard Audio-visual
Number - would work.

Handles are a namespace that comes close to this, but the mapping
between handle naming authorities and domain names is far from
exact, and we are likely to get instantly directed to G or to
a local handle server whenever we encounter a name in the hdl:
system.

Well, that's enough for a Friday afternoon.
Regards,

Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Fri, 16 Aug 96 15:26:41 CDT
From: [email-masked] (Daniel LaLiberte)
To: Ron Daniel <[email-masked]>
Cc: [email-masked]
Subject: [URN] revision of NAPTR draft
Sender: [email-masked]
Reply-To: [email-masked] (Daniel LaLiberte)

Ron Daniel writes:
 > This draft does not incorporate Lewis Girod's suggested replacement
 > for regexps. I would like to see some more discussion on the list
 > of that topic.

I don't have much to say about Lewis Girod's suggestion - I rather
like regular expressions and don't think they will be a problem for
what the do offer.

However, I am concerned because of features I know regular expression
lack.  In particular, there is a need to be able to reverse a list of
component names in the construction of a DNS name, because DNS uses
one order for components (i.e. most specific component to most
general, e.g. small.big.bigger.biggest.) whereas some names systems
might order components in the other order (e.g. the path scheme).
Regular expressions do not support the idea of reversing an arbitrary
number of elements.

One way around this obstacle is to add some features beyond (or
outside of) normal regular expression.  Three things are needed here.
A way to specify the components of the list, a way to specify that the
list should be reversed, and a way to concatenate the elements of the
list with the "." separator.  The list of elements might be specified
by parentheses.  E.g.  #(/([^/])*)*# matches a path such as /a/b/c
with the inner parentheses matching each of the components.  To
reverse the list, an "r" might be appended to the regular expression.
The replacement might assume that the constructed DNS name takes the
elements of the list, with the option of reversing them, and joins
with the "." separator.  There are some problems with this scheme that
I'll let others point out - maybe they have solutions too.

In previous discussion with Ron, he suggested that it might be
possible to use the resolution algorithm, as is but iteratively, to
bite off a component at a time and construct the reversed DNS name.
This sounds similar to what we recommend for the path scheme
resolution in the first place, so I would be happy to go with it - it
wouldnt require a list reversal extension.  But I don't see how it can
be done.  Could you give an example of how this works?

--
Daniel LaLiberte ([email-masked])
National Center for Supercomputing Applications
http://union.ncsa.uiuc.edu/~liberte/

From: "Ronald E. Daniel" <[email-masked]>
Date: Fri, 16 Aug 1996 13:49:33 -0600
To: [email-masked]
Subject: [URN] revision 1 of the NAPTR draft
Sender: [email-masked]
Reply-To: "Ronald E. Daniel" <[email-masked]>

INTERNET DRAFT                                                  Ron Daniel
draft-daniel-naptr-01.txt                   Los Alamos National Laboratory
                                                          Michael Mealling
                                           Georgia Institute of Technology
                                                              16 Aug, 1996

              Resolution of Uniform Resource Identifiers
                     using the Domain Name System

Status of this Memo
===================

    This document is an Internet-Draft.  Internet-Drafts are working
    documents of the Internet Engineering Task Force (IETF), its
    areas, and its working groups.  Note that other groups may also
    distribute working documents as Internet-Drafts.
  
    Internet-Drafts are draft documents valid for a maximum of six
    months and may be updated, replaced, or obsoleted by other
    documents at any time.  It is inappropriate to use Internet-
    Drafts as reference material or to cite them other than as
    ``work in progress.''
  
    To learn the current status of any Internet-Draft, please check
    the ``1id-abstracts.txt'' listing contained in the Internet-
    Drafts Shadow Directories on ftp.is.co.za (Africa),
    nic.nordu.net (Europe), munnari.oz.au (Pacific Rim),
    ds.internic.net (US East Coast), or ftp.isi.edu (US West Coast).

    This draft expires 16 Feb., 1997.

Abstract:
=========

Uniform Resource Locators (URLs) are the foundation of the World Wide
Web, and are a vital Internet technology. However, they have proven to
be brittle in practice. The basic problem is that URLs typically
identify a particular path to a file on a particular host. There is no
graceful way of changing the path or host once the URL has been
assigned. Neither is there a graceful way of replicating the resource
located by the URL to achieve better network utilization and/or fault
tolerance. Uniform Resource Names (URNs) have been hypothesized as a
replacement for URLs that would overcome such problems. URNs and URLs
are both instances of a broader class of identifiers known as Uniform
Resource Identifiers (URIs).

This document describes a new DNS Resource Record, NAPTR (Naming
Authority PoinTeR), that provides rules for mapping parts of URIs to
domain names.  By changing the mapping rules, we can change the host
that is contacted to resolve a URI. This will allow a more graceful
handling of URLs over long time periods, and forms the foundation for a
new proposal for Uniform Resource Names.

In addition to locating resolvers, the NAPTR provides for other naming
systems to be grandfathered into the URN world, provides independence
between the name assignment system and the resolution protocol system,
and allows multiple services (Name to Location, Name to Description,
Name to Resource, ...) to be offered.  In conjunction with the SRV RR
proposal [3], the NAPTR record allows those services to be replicated
for the purposes of fault tolerance and load balancing.

Introduction:
=============

Uniform Resource Locators have been a significant advance in locating
resources on the Internet. However, their  brittle nature over time
has been recognized for several years. The Uniform Resource Identifer
working group proposed the development of Uniform Resource Names to serve
as persistent, location-independent identifiers for Internet resources
in order to overcome most of the problems with URLs. RFC-1737 [1] sets
forth requirements on URNs.

Even if URN systems were in place now, there would still be a
tremendous number of URLs.  It should be possible to develop a URN
resolution system that can also provide location independance for those
URLs.  This is related to the requirement in [1] to be able to
grandfather in names from other naming systems, such as ISO Formal
Public Identifiers, Library of Congress Call Numbers, ISBNs, ISSNs,
etc.

The developers of various URN proposals have held a series of meetings,
resulting in a compromise known as the Knoxville framework. The
major principle behind the Knoxville framework is that any name
assignment hierarchy should be seperate from the resolution hierarchy.
This is in marked contrast to the Domain Name System, where the two are
identical.  Readers are referred to [2] for background on the Knoxville
framework for additional information on the context and purpose of this
proposal.

For the reasons mentioned earlier, we want to be able to resolve URNs
and URLs within the same framework. For the short term, DNS is the
obvious candidate for the resolution framework, since it is widely
deployed and understood. However, it is not appropriate to use DNS to
maintain information on a per-resource basis. First of all, DNS was
never intended to handle that many records.  Second, the limited record
size is inappropriate for catalog information.  Third, we have the
requirement mentioned above about grandfathering of other name
systems.

Therefore our approach is to use DNS to locate "resolvers" that can
provide information on individual resources, potentially including the
resource itself. To accomplish this, we "rewrite" the URI into a domain
name following the rules provided in NAPTR records. Rewrite rules
provide considerable power, which is important when trying to seperate
the name assignment and resolution procedures. However, collections of
rules can become difficult to understand. To lessen this problem, the
NAPTR rules are *always* applied to the original URI, *never* to the
output of previous rules.

Locating a resolver through the rewrite procedure may take multiple
steps, but the beginning is always the same. Every URI has a
colon-delimited prefix.  NAPTR resolution begins by taking this prefix,
appending the well-known suffix ".urn.net", and querying DNS for NAPTR
records at that domain name.  Based on the results of this query, zero
or more additional DNS queries may be needed to locate resolvers for the
URI. Three brief examples of this procedure are given in the next
section.

The NAPTR RR provides the level of indirection needed to keep the naming
system independent of the resolution system, its protocols, and services.
Coupled with the new SRV resource record proposal[3] there is also the
potential for replicating the resolver on multiple hosts, overcoming some
of the most significant problems of URLs. This is an important and subtle
point. Not only do the NAPTR and SRV records allow us to replicate
the resource, we can replicate the resolvers that know about the replicated
resource. Preventing a single point of failure at the resolver level
is a significant benefit. 

Terminology
===========

"Must" or "Shall" - Software that does not behave in the mannar that this
           document says it must is not conformant to this document.
"Should" - Software that does not follow the behavior that this document
           says it should may still be conformant, but is probably broken
           in some fundamental way.
"May" -    Implementations may or may not provide the described behavior,
           while still remaining conformant to this document.

Brief overview and examples of the NAPTR RR:
============================================

A detailed description of the NAPTR RR will be given later, but to give
a flavor for the proposal we first give a simple description of the
record and three examples of its use.

The key fields in the NAPTR RR are order, preference, service, flags,
replacement, and regexp:
* The order field specifies the order in which records MUST
  be processed when multiple  NAPTR records are returned in response to
  a single query. A naming authority may have delegated a portion of its
  namespace to another agency. Evaluating the NAPTR recors in the correct
  order is necessary for delegation to work properly.
* The preference field specifies the order in which records SHOULD
  be processed when multiple NAPTR records have the same value of "order".
  This field lets a service provider specify the order in which resolvers
  are contacted, so that more capable machines are contacted in preference
  to less capable ones.
* The service field specifies the resolution protocol and resolution
  service(s) that will be available if the rewrite specified by the
  regexp or replacement fields is applied. Resolution protocols are
  the protocols used to talk with a resolver, and will be specified in
  other documents. Resolution services are operations such as
  N2R (URN to Resource), N2L (URN to URL), N2C (URN to URC), etc.
  These are discussed in the URN Framework document[2], and
  their behavior in a particular resolution protocol will be given in
  the specification for that protocol.
* The flags field contains modifiers that affect what happens in the
  next DNS lookup, typically for optimizing the process.
* The replacement field is one of two fields used for the rewrite rules.
  If the rewrite is a simple substitution of a domain name, that name is
  given in the replacement field. The replacement field is a domain name
  (subject to compression if a DNS sender knows that a given recipient is
  able to decompress names in this RR type's RDATA field). If the rewrite
  is more complex than a simple substitution of a domain name, the
  replacement field should be set to . and the regexp field used.
* The regexp field is the other field that may be used for rewrite rules.
  The regexp field is a String containing sed-style substitution expressions.
  These are applied to the original URN and yield the next domain name
  to be queried.

Note that the client applies all the substitutions and performs all
lookups - this is not handled in DNS servers. Note also that there is no
reason to provide values in both the replacement and regexp field. Only
one should be specified. If a value is specified in both, the
replacement name MUST be used and the regexp string MUST be ignored. It
is the belief of the developers of this document that regexps should
rarely be used. The replacement field seems adequate for the vast majority
of situations.

Example 1
---------

Consider a URN that uses the DUNS namespace. DUNS numbers are
identifiers for approximately 30 million registered businesses around
the world, assigned and maintained by Dunn and Bradstreet. The URN
might look like:

                 urn:duns:002372413:annual-report-1997

The first step in the resolution process is to find out about the DUNS
namespace. The namespace identifier, duns, is extracted from the URN,
prepended to urn.net, and the NAPTRs for duns.urn.net looked up. It might
return records of the form:

duns.urn.net
;;      order pref flags service            replacement                regexp
 IN NAPTR 100  10  "s"  "dunslink+N2L+N2C" _dunslink._udp.isi.dandb.com   ""
 IN NAPTR 100  20  "s"  "rcds+N2C"         _rcds._udp.isi.dandb.com       ""
 IN NAPTR 100  30  "s"  "http+N2L+N2C+N2R" _http._tcp.isi.dandb.com       ""

The order field contains equal values, indicating that no name delegation
order has to be followed. The preference field indicates that the provider
would like clients to use the special dunslink protocol, followed by
the RCDS protocol, and that HTTP is offered as a last resort. All the
records specify the "s" flag, which will be explained momentarily.
The service fields say that if we speak dunslink, we will be able to
issue either the N2L or N2C requests to obtain a URL or a URC (description)
of the resource. The Resource Cataloging and Distribution Service (RCDS) could
be used to get a URC for the resource, while HTTP could be used to get
a URL, URC, or the resource itself.  All the records supply the next
domain name to query, none of them need to be rewritten with the aid of
regular expressions.

The general case might require multiple NAPTR rewrites to locate a
resolver, but eventually we will come to the "terminal NAPTR". Once we
have the terminal NAPTR, our next probe into the DNS will be for a SRV 
or A record instead of another NAPTR. Rather than probing for a non-existant
NAPTR record to terminate the loop, the flags field is used to indicate
a terminal lookup. If it has a value of "s", the next lookup should
be for SRV RRs, "a" denotes that A records should sought.

Since our example RR specified the "s" flag, it was terminal. Our
next action is to lookup SRV RRs for _rcds._udp.isi.dandb.com, which
will tell us hosts that can provide the necessary resolution service. 
That lookup might return:

;;                              Pref Weight Port Target
  rcds._udp.isi.dandb.com IN SRV 0    0    1000 defduns.isi.dandb.com
                          IN SRV 0    0    1000 dbmirror.com.au
                          IN SRV 0    0    1000 ukmirror.com.uk

telling us three hosts that could actually do the resolution, and
giving us the port we should use to talk to their RCDS server.
(The reader is referred to the SRV proposal [3] for the interpretation
of the fields above).

Since a domain administrator knows certain NAPTRs are terminal,  (s)he
might include the corresponding SRV RRs as additional info. Further,
the SRV RRs could include A records as additional info.  While this
recursive provision of additional information is not explicitly blessed
in the DNS specifications, it is not forbidden, and BIND does take
advantage of it [4].  This is a significant optimization. In
conjunction with a long TTL for *.urn.net records, the average number
of probes to DNS for resolving DUNS URNs would approach one.

Records MUST be processed in the order specified by the order field.
This allows administrators to say that "all records matching this
pattern go to server1, all others go to server2". Records SHOULD be
processed in the order specified in the preference field. However,
clients may have a preference for particular protocols and resolution
services that causes them to ignore the preference field, so resolver
administrators cannot count on the preference field always being
respected.

Example 2 
---------

Consider a URI namespace based on MIME Content-Ids.
The URN might look like this:
  
        urn:cid:[email-masked]
  
The first step in the resolution process is to find out about the CID
namespace. The namespace identifier, cid, is extracted from the URN,
prepended to urn.net, and the NAPTR for cid.urn.net looked up. It might
return records of the form:
 
;;                  order pref flags service replacement       regexp
cid.urn.net IN NAPTR 100  10   ""     ""       .        "/.+@([^@]+)/\1/i"
 
We have only one NAPTR response, so ordering the responses is not a
problem.  The replacement field is empty, so we check the regexp field
and use the pattern provided there. We apply that regexp to the entire
URN to see if it matches, which it does.  The \1 part of the regexp
returns the string "mordred.gatech.edu". Since the flags field is
empty, the lookup is not terminal, and our next probe to DNS is for
NAPTR records. The result might look something like:

*.gatech.edu IN NAPTR
;;     order pref flags service           replacement             regexp
IN NAPTR 100  50  "s"  "z39.50+N2L+N2C"   _z3950._tcp.gatech.edu    ""
IN NAPTR 100  50  "s"  "rcds+N2C"         _rcds._udp.gatech.edu     ""
IN NAPTR 100  50  "s"  "http+N2L+N2C+N2R" _http._tcp.gatech.edu     ""

Note that a wildcard has been used so that the domain administrator
at Georgia Tech has only a single NAPTR record to maintain. The
values of the order and preference fields are equal in all records, so
we are free to pick any record.
The flags field tells us that this is the last NAPTR patterns we
should see, and after the rewrite (a simple replacement in this case) we
should look up SRV records to get information on the hosts 
that can provide the necessary service.  Assuming we prefer the Z39.50
protocol, our lookup might return:
 
;;                            Pref Weight Port Target
_z3950._tcp.gatech.edu IN SRV 0    0      1000 z3950.gatech.edu
                       IN SRV 0    0      1000 z3950.cc.gatech.edu
                       IN SRV 0    0      1000 z3950.uga.edu

telling us three hosts that could actually do the resolution, and
giving us the port we should use to talk to their Z39.50 server.

Users of regular expressions should note that there is a significant
caveat about the use of backslashes in DNS zone files. DNS treats
backslashes as the escape character so that '.' can be escaped when
necessary. This means that when a regular expression is entered into
the zone file, the backslashes must be escaped by another backslash.
For the case of the cid.urn.net record above, the regular expression
entered into the zone file should be "/.+@([^@]+)/\\1/i".  When the
client code actually recieves the record, the pattern will have been
converted to "/.+@([^@]+)/\1/i".

Example 3
---------

As mentioned above, the NAPTR RR can also be used for URLs that have
already been assigned. Assume we have the URL for a very popular piece
of software that the publisher wishes to mirror at multiple sites around
the world:

     http://www.foo.com/software/latest-beta.exe

We extract the prefix, "http", and lookup NAPTR records for
http.urn.net. This might return a record of the form

http.urn.net IN NAPTR
;;  order   pref flags service replacement          regexp
     100     90   ""      ""        .        "/.*\/\/([^\/:]+)/\1/i"

This expression returns everything after the first double slash and
before the next slash or colon. Backslashes are needed to escape the
forward slash since the forward slash character is what seperates the
components of the substitution pattern. (Recall from the previous
example that in the zone file, this pattern actually needs to be
entered as "/.*\\/\\/([^\\/:]+)/\\1/i").  Applying this pattern to the
URL extracts "www.foo.com". Looking up NAPTR records for that might
return:

www.foo.com
;;       order pref flags   service     replacement      regexp
 IN NAPTR 100  100  "s"   "http+L2R" _http._tcp.foo.com   ""
 IN NAPTR 100  100  "s"   "ftp+L2R"  _ftp._tcp.foo.com    ""

Looking up SRV records for _http._tcp.foo.com would return information
on the hosts that foo.com has designated to be its mirror sites. The
client can then pick one for the user. 

NAPTR RR Format
===============

The format of the NAPTR RR is given below. The DNS type code for
NAPTR is 104.

    Domain TTL Class Order Preference Flags Service Replacement Regexp

where:

Domain
       The domain name this resource record refers to.
TTL
       Standard DNS Time To Live field
Class
       Standard DNS meaning
Order
       A 16-bit integer specifying the order in which the NAPTR records
       MUST be processed to ensure correct delegation of portions
       of the namespace over time. Low numbers are processed before
       high numbers, and once a NAPTR is found that "matches" a URN,
       the client MUST NOT consider any NAPTRs with a higher value
       for order.

Preference
       A 16-bit integer which specifies the order in which NAPTR records
       with equal "order" values SHOULD be processed, low numbers
       being processed before high numbers.  This is similar to the
       preference field in an MX record, and is used so domain
       administrators can direct clients towards more capable hosts
       or lighter weight protocols. 

Flags
       A String giving flags to control aspects of the rewriting. Flags
       are single characters from the alphabet [A-Z0-9]. The case of
       the alphabetic characters is not significant.

       At this time only two flags, "S" and "A", are defined. "S" means
       that the next lookup should be for SRV records instead of NAPTR
       records. "A" means that the next lookup should be for A records.
       The S and A flags are mutually exclusive, and resolution libraries
       SHOULD signal and error so that bad NAPTRs can be quickly detected.

       The remaining alphabetic flags are reserved. The numeric flags
       may be used for local experimentation.

Service 
       Specifies the resolution service(s) available down this rewrite
       path. May also specify the particular protocol that is used to
       talk with a resolver. If a protocol is specified in the service
       field, the rewrite will almost certainly be terminal, although the
       flags field MUST still be considered the authoritative source of
       information on that subject.

       The service field may take any of the values below (using the
       Augmented BNF of RFC 822[5]):

           service_field = [ [protocol] *("+" rs)]
           protocol      = "RCDS" / "HTTP"
           rs            = "N2L" / "N2Ls" / "N2R" / "N2Rs" / "N2C"
                         / "L2R" / "L2C"

       i.e. an optional protocol specification followed by 0 or more
       resolution services. Each resolution service is indicated by
       an initial '+' character.

       Note that the empty string is also a valid service field. This
       will typically be seen at the top levels of a namespace, when it
       is impossible to know what services and protocols will be offered
       by a particular publisher within that name space.

       At this time the known protocols are RCDS (a binary, UDP-based
       protocol) and HTTP (a textual, TCP-based protocol). More will be
       allowed later. At this time the allowed service requests are:
             N2L  - Given a URN, return a URL
             N2Ls - Given a URN, return a set of URLs
             N2R  - Given a URN, return an instance of the resource.
             N2Rs - Given a URN, return multiple instances of the resouce,
                    typically encoded using multipart/alternative.
             N2C  - Given a URN, return a collection of meta-information on
                    the named resource. The format of this response is the
                    subject of another document.
             L2R  - Given a URL, return the resource.
             L2C  - Given a URL, return a description of the resource.
       The actual format of the service request and response will be
       determined by the resolution protocol, and is the subject for other
       documents.

Replacement
       The next NAME to query for NAPTR, SRV, or A records depending on
       the value of the flags field. 

Regexp
       A STRING containing a sed(1)-style substitution expression that is
       applied to the original URI in order to construct the next name to
       lookup. For example:
         "/^.*inet:([^:]+):*/\\1/i"
               Regular expression replacement. The URN will be searched
               from the beginning for the pattern "inet:". All characters
               after that colon, up to but not including the next colon,
               will form the next domain name to be looked up.  The /i
               flag says that the match will be case-insensitive, so
               IneT, Inet, INET, ... would all be matched. The new
               search key would also be normalized to lower case.

Advice to domain administrators:
================================

Beware of regular expressions. Not only are they a pain to get
correct on their own, but there is the previously mentioned interaction
with DNS. Any backslashes in a regexp must be entered twice in a zone
file in order to appear once in a query response. More seriously, the
need for double backslashes has probably not been tested by all
implementors of DNS servers. We anticipate that urn.net will be the
heaviest user of regexps. Only when delegating portions of namespaces
should the typical domain administrator need to use regexps.

On a related note, beware of interactions with the shell when manipulating
regexps from the command line. Since '\' is a common escape character in
shells, there is a good chance that when you think you are saying "\\" you
are actually saying "\".

Administrators are strongly encouraged to provide SRV records or A
records as additional information in terminal NAPTR records.

The "a" flag allows the next lookup to be for A records rather than
SRV records. Since there is no place for a port specification in the
NAPTR record, when the "A" flag is used the specified protocol must
be running on its default port. 

Usage
=====

Pseudocode for a client routine using NAPTRs is given below:

    //
    // findResolver(URN)
    // Given a URN, find a host that can resolve it.
    // 
    findResolver(string URN) {
      sprintf(key, "%s.urn.net", extractNS(URN));  // prepend prefix to urn.net
      do {
        rewrite_flag = false;
        terminal = false;
	if (key has been seen) {
	  quit with a loop detected error
	}
	add key to list of "seens"
	records = lookup(type=NAPTR, key); // get all NAPTR RRs for 'key'

        sort NAPTR records by "order" field and "preference" field.
        n_naptrs = number of NAPTR records in response.
        curr_order = records[j].order;
        max_order = records[n_naptrs-1].order;
        
        // Process current batch of NAPTRs according to "order" field.
        for (j=0; j < n_naptrs && records[j].order <= max_order; j++) {
          newkey = rewrite(URN, naptr[j].replacement, naptr[j].regexp);
          if (!newkey) // Skip to next record if the rewrite didn't match
             continue;  
          // We did do a rewrite, shrink max_order to current value
          // so that delegation works properly
          max_order = naptr[j].order;
          // Will we know what to do with the protocol and services
          // specified in the NAPTR? If not, try next record.
          if(!isKnownProto(naptr[j].services)) {
            continue;
          }
          if(!isKnownService(naptr[j].services)) {
            continue;
          }

          // At this point we have a sucessful rewrite and we will know
          // how to speak the protocol and request a known resolution
          // service. Before we do the next lookup, check some
          // optimization possibilities.
          
          if (strcasecmp(flags, "S") || strcasecmp(flags, "A")) {
             terminal = true;
             services = naptr[j].services;
             addnl = any SRV and/or A records returned as additional info
                     for naptr[j].
          }
          key = newkey;
          rewriteflag = true;
          break;
        }
      } while (rewriteflag && !terminal);

      if (!addnl) { // No SRVs came in as additional info, look them up
        srvs = lookup(type=SRV, key);
      }  

      sort SRV records by preference, weight, ...
      foreach (SRV record) { // in order of preference
        try contacting srv[j].target using the protocol and one of the
            resolution service requests from the "services" field of the
            last NAPTR record.
        if (successful)
          return (target, protocol, service);
          // Actually we would probably return a result, but this
          // code was supposed to just tell us a good host to talk to.
      }
      die with an "unable to find a host" error;
    }

Notes:
======
  -  The "urn:" prefix is a matter of religious controversy. Client
     code should handle the cases when it is and is not present.
     Similarly, if regular expressions are used in NAPTR records, they
     should be immune to the presence or absence of "urn:".
  -  A client MUST examine all the RRs in a reply, not just the first.
     The client MUST process them according to the value of the order field.
     Once a NAPTR has "matched" a URN, the client MUST NOT consider
     any NAPTRs with a higer value of order than that of the matching NAPTR,
     even if it does not know the protocols necessary to talk with the
     matching resolver.
     (A match is defined as:
      1) The NAPTR provides a replacement domain name
      2) The regular expression matches the URN
     )
  -  If a record at a particular order matches the URI, but the client
     doesn't know the specified protocol and service, the client should
     continue to examine records that have the same order. The client
     MUST NOT consider records with a higher value of order. This is
     necesary to make delegation of portions of the namespace work.
  -  When multiple RRs have the same "order", the client should use
     the value of the preference field to select the next NAPTR to
     consider. However, because of preferred protocols or services,
     estimates of network distance and bandwidth, etc. clients
     may use different criteria to sort the records.
  -  If the lookup after a rewrite fails, clients are strongly encouraged
     to report a failure, rather than backing up to pursue other rewrite
     paths.
  -  Note that SRV RRs impose additional requirements on clients.

Acknowledgements:
=================

The authors would like to thank Keith Moore for all his consultations
during the development of this draft. We would also like to thank Paul
Vixie for his assistance in debugging our implementation, and his answers
on our questions.

References:
===========

[1] RFC-1737 "Functional Requirements for Uniform Resource Names", Karen
    Sollins and Larry Masinter, Dec. 1994.

[2] draft-daigle-urn-framework-00.txt "A Uniform Resource Naming
    Framework", Leslie Daigle and Patrik Faltstrom, June, 1996.

[3] draft-gulbrandsen-dns-rr-srvcs-03.txt  " A DNS RR for specifying the
    location of services (DNS SRV)",  Arnt Gulbrandsen and Paul Vixie,
    March 1996.

[4] Paul Vixie, personal communication.

[5] RFC-822 "Standard for the Format of ARPA Internet Text Messages",
    Dave H. Crocker, August 1982.  

[6] Keith Moore, personal communication.

Security Considerations
=======================
  The use of "urn.net" as the registry for URN namespaces is subject to
  denial of service attacks, as well as other DNS spoofing attacks. The
  interactions with DNSSEC are currently being studied. It is expected
  that NAPTR records will be signed with SIG records once the DNSSEC
  work is deployed. 

  The rewrite rules make identifiers from other namespaces subject to
  the same attacks as normal domain names. Since they have not been
  easily resolvable before, this may or may not be considered a problem.

Author Contact Information:
===========================

Ron Daniel
Los Alamos National Laboratory
MS B287
Los Alamos, NM, USA, 87545
voice:  +1 505 665 0597
fax:    +1 505 665 4939
email:  [email-masked]

Michael Mealling
Office of Information Technology, Network Services
Georgia Institute of Technology
Atlanta, GA, USA 30332-0730
voice:  (404) 894-1712
fax:  (404) 894 9548
[email-masked]

    This draft expires 16 Feb., 1997.
Ron Daniel Jr.                email: [email-masked]
Advanced Computing Lab        voice: (505) 665-0597
MS B-287  TA-3  Bldg. 2011      fax: (505) 665-4939
Los Alamos National Lab        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM,  87545       slogan: "Dump Navigator Now!"

Date: Fri, 16 Aug 1996 13:54:56 -0600
To: [email-masked]
From: Ron Daniel <[email-masked]>
Subject: [URN] revision of NAPTR draft
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Hi all,

The next message from me will be the text of the revised NAPTR
draft. The major changes are:

1) The pattern field has been split into two fields - replacement
   and regexp. If one particular domain name is to be the output
   of a rewrite, it is given in the replacement field. This allows
   domain name compression to be used. The change was made at
   the request of Paul Vixie.

2) The preference field has been split into "order" and "preference"
   fields. I was afraid that domain administrators would abuse the
   original preference field to state preferred resolvers, rather
   than follow the spec which stated that it should be used for
   evaluating NAPTRs in the correct order for name delegation to
   work. (The choice of names was probably not a big help either :-)
   Since there was no other way for resolver administrators to
   say "please contact this service if you can, only use this
   other one if you must" the change seems justified.

This draft does not incorporate Lewis Girod's suggested replacement
for regexps. I would like to see some more discussion on the list
of that topic.

Next week I will submit the revised draft to the Internet-drafts
editor, but wanted to give people on this list some advance
notice.

Regards,
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Thu, 15 Aug 1996 12:13:55 -0600
To: Jon Knight <[email-masked]>
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] nasty rewriting rules
Cc: [email-masked], [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Jon Knight (at least at 05:04 PM 8/15/96 +0100)
>On Thu, 15 Aug 1996, Ron Daniel wrote:
>> One difference is that sed-style regexps use \1 \2 etc. for inserting
>> matches in substitution operations, while perl uses $1 $2 etc. 
>
>Perl uses \1, \2, etc for substitutions within the regexp.  The $1, $2, 
>etc is so that you can use the matched parts of regexp in the following 
>code.  See page 104 of "Programming in Perl" (the Camel book) for details.

Actually, that page says that \1 is only good in the pattern, and
that for the replacement part of substitution operations, $1 should
be used.

Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Thu, 15 Aug 1996 17:04:19 +0100 (BST)
From: Jon Knight <[email-masked]>
To: Ron Daniel <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: [URN] nasty rewriting rules
Sender: [email-masked]
Reply-To: Jon Knight <[email-masked]>

On Thu, 15 Aug 1996, Ron Daniel wrote:
> One difference is that sed-style regexps use \1 \2 etc. for inserting
> matches in substitution operations, while perl uses $1 $2 etc. 

Perl uses \1, \2, etc for substitutions within the regexp.  The $1, $2, 
etc is so that you can use the matched parts of regexp in the following 
code.  See page 104 of "Programming in Perl" (the Camel book) for details.

> Perl
> also provides things like \w for matching words, \W for matching non-words,
> etc. that I do not recall being in sed-style regexps. There are, no
> doubt, many smaller differences. For example, in Perl, '.' does not
> match a newline character.

This is quite true.  Perl regexp's are a superset of egrep's regexp's and 
so have lots of "nice" extra features (one of the reasons I love Perl so 
much!)

Tatty bye,

Jim'll

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Jon "Jim'll" Knight, Researcher, Sysop and General Dogsbody, Dept. Computer
Studies, Loughborough University of Technology, Leics., ENGLAND.  LE11 3TU.
* I've found I now dream in Perl.  More worryingly, I enjoy those dreams. *

Date: Thu, 15 Aug 1996 09:04:31 -0600
To: [email-masked]
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] nasty rewriting rules
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke [email-masked] (at least at 12:28 PM 8/15/96 +0200)

>There's a regexp description in Appendix E of RFC 1801.
>Note that this does NOT define substitution syntax, which URNs need,
>and for which there are 2 main variants: PERL and SED; one uses
>\( \) and the other uses ( ) as delimiters, I believe.

One difference is that sed-style regexps use \1 \2 etc. for inserting
matches in substitution operations, while perl uses $1 $2 etc. Perl
also provides things like \w for matching words, \W for matching non-words,
etc. that I do not recall being in sed-style regexps. There are, no
doubt, many smaller differences. For example, in Perl, '.' does not
match a newline character.

Regards,

Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Thu, 15 Aug 1996 08:29:34 -0600
To: Lewis Girod <[email-masked]>, [email-masked]
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] nasty rewriting rules
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Lewis Girod (at least at 03:54 PM 8/13/96 -0400):

>   On Tue, 13 Aug 1996 15:02:24 +0100 (BST) Jon Knight wrote
>
>   Regexp interpreters are already available in source code format that you 
>   can just plug and play on many current platforms and which will ease 
>   porting to new architectures.
>
>Fair enough, although it is important to have agreement as to what
>flavor of regexps are being used.  The original plan was to just
>execute sed, which solves both issues on UNIX platforms, but requires
>a little more work otherwise.

Um, no, that was not the plan. The plan was to use regexp libraries, the
syntax of the regexps was "sed-style" rather than "perl-style" because
we believe that sed-style is actually the version most regexp libraries
support.

>The fact remains that unless such
>syntax-checking regexps are actually implemented at the top level,
>there is no guarantee that a namespace will retain a set structure.

We have not had any plans to use regexps that also validate the form
of the URN and report errors. 

>There is some suggestion in the framework document that there be a set
>of requirements of new name schemes

Well, I thought that those requirements were going to be things like
no reassigning of naming authorities, making sure that any naming
authorities in a naming scheme require any sub-naming authorities to follow
rules at lesat as restrictive as the ones they operate under, ...

So, these really are requirements on naming schemes and the operations
behing them. Such requiremnts can't be tested by validating the syntax of
a URN - useful thought that might be.

>   Of course those of us that do 
>   want regexp are still free to have a "NAPTRclassic" (:-) ) namespace 
>   hived off using SRV records that does do regexp replacement, so maybe we 
>   can all have our cake and eat it?

No, the syntax and capabilitites of the rewrite rules are a property of
the NAPTR record, not of individual namespaces. The question before the
group is if NAPTR rewrite rules should be regexps or something else.
Lewis' proposal is, so far at least, the only alternative that has been
advanced. I don't think we can have both, we have to decide.

>I am the first to say that the terse syntax is wretched, but it was
>really easy to implement and it is terse enough to easily fit into a
>DNS response.

Being short enough to fit into a DNS response is quite important for
efficiency considerations. Exceeding 512 bytes in a response shifts us
from UDP to TCP. Also, we want to have plenty f room so that we can include
SRV and A records as additional information to reduce the number of
queries to DNS.

>Humans are supposed to use the simple user-friendly
>syntax that compiles into the terse form.  With decent management
>software the underlying terse form should be transparent to the
>administrator -- in fact, if we can assume that everyone is using a
>compiler it might be a good idea to make the terse form even more
>compact.

Those are pretty strong assumptions Lewis. Furthermore, this software
also has to decompile the terse form into the friendly form so that
people can debug existing records. This sort of friendly software
rarely gets written and maintained.

Regards,
 
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

From: [email-masked]
To: Lewis Girod <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: [URN] nasty rewriting rules
Date: Thu, 15 Aug 1996 12:28:27 +0200
Sender: [email-masked]
Reply-To: [email-masked]

There's a regexp description in Appendix E of RFC 1801.
Note that this does NOT define substitution syntax, which URNs need,
and for which there are 2 main variants: PERL and SED; one uses
\( \) and the other uses ( ) as delimiters, I believe.

                 Harald A

Date: Tue, 13 Aug 1996 15:54:34 -0400
From: Lewis Girod <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: [URN] nasty rewriting rules
Sender: [email-masked]
Reply-To: Lewis Girod <[email-masked]>

   On Tue, 13 Aug 1996 15:02:24 +0100 (BST) Jon Knight wrote

   Regexp interpreters are already available in source code format that you 
   can just plug and play on many current platforms and which will ease 
   porting to new architectures.

Fair enough, although it is important to have agreement as to what
flavor of regexps are being used.  The original plan was to just
execute sed, which solves both issues on UNIX platforms, but requires
a little more work otherwise.

   > There are two questions here.  First, this statement is only true
   > given that the structure of the name scheme actually remains
   > consistent with this set stucture in the context of NAPTR (this gets
   > back to point (1) above).  If they were resolved with NAPTR using
   > regexps little would prevent someone from changing the format of an
   > ISBN in various ways; there is no technical impediment to this, and if
   > the top level ISBN naming authority doesn't want it to happen their
   > only recourse is legal.

   Surely though as long as the end users are using "legal" ISBNs in their
   URNs, the rewriting that happens is just the business of the people
   "owning" that ISBN and their resolution agents (might be one and same
   thing in some cases).  If the resolution agents at the top level of a
   naming authority reject "illegal" constructs in that scheme that they
   might get given, people will soon get the idea and stop using them.

Oops, thanks for catching that bug.  Yes, it is sufficient to reject
malformed URNs at the top level, and as you say that fixes the
problem.  As a rule, this can be done using regexps (although I have
heard of no plans to do so).  The fact remains that unless such
syntax-checking regexps are actually implemented at the top level,
there is no guarantee that a namespace will retain a set structure.
                                                                       If
   the top level naming authority never see the "illegal" rewritten versions
   then that's fine as it means that the rewriting is happening in private
   lower down the resolution tree.  

Remember, the rewrite rules are rewriting URNs into domain names, not
into other URNs -- so a URN is either legal or illegal from the start.
The concern is that names could be added to the namespace which don't
conform to the name scheme as defined, but which would still be
resolved by the system.  If unpleasant contortions are required to
perform the resolution downstream it may make the name scheme harder
to migrate as a whole.  The question is whether the flexibility that
makes this possible is neccessary.

                                    I'd say that "enforcement" is just a
   private thing within a particular naming scheme resolution tree and isn't
   something we should be dipping our toes into (not if we want URNs in this
   lifetime anyway).

There is some suggestion in the framework document that there be a set
of requirements of new name schemes; any ``enforcement'' is limited to
that sort of thing.  We have not yet figured out what requirements are
to be applied, but clearly names will need to be verified somewhere to
ensure that they are satisying them -- otherwise there might as well
be no requirements.  So the question becomes, what requirements do we
want to set?  One possible requirement for NAPTR classic might be that
the syntax be defined and checked.  What I am suggesting boils down to
specifying the hierarchy scheme and ensuring that the NAPTR system
escape names to alternate systems as soon as the hierarchy spec is
invalidated.

   > This should make it easier for
   > people to learn and program in the ``language'' (if such a simple
   > thing can be called a language!!) while at the same time making syntax
   > errors less likely.  For example, we could use S-expressions (pardon
   > my rough adherence to standard BNF form..):

   I can feel an Emacs mode coming on... :-) :-)

But of course... :-)

   > So for example, given a canonical form email address mail:edu.mit.lcs@girod
   > 
   >   ((eat-including ":" &replace-with "")
   >    (eat-until "@" &copy)
   >    (eat-including "@" &replace-with ".mail-urn"))
   > 
   > Would ``compile'' to ``x":"+p"";x"@"-v;x"@"+p".mail-urn";'', and would
   > generate the pair ("mail-urn.lcs.mit.edu", "girod").  Note that the
   > translation took care of removing the ``urn:'' if it was there.  This
   > or something like it should be simple enough to learn.

   And this is simpler that regexps?  As a sysadmin I'd much rather use 
   something I'm already used to (ie: regexps) rather than learning (and 
   making mistakes in) another new syntax.  Of course those of us that do 
   want regexp are still free to have a "NAPTRclassic" (:-) ) namespace 
   hived off using SRV records that does do regexp replacement, so maybe we 
   can all have our cake and eat it?

I am the first to say that the terse syntax is wretched, but it was
really easy to implement and it is terse enough to easily fit into a
DNS response.  Humans are supposed to use the simple user-friendly
syntax that compiles into the terse form.  With decent management
software the underlying terse form should be transparent to the
administrator -- in fact, if we can assume that everyone is using a
compiler it might be a good idea to make the terse form even more
compact.  As you say, you can always escape to some kind of proxy that
does regexp stuff.

The explanation in that message didn't really explain the language,
mainly because the terse form is explained in more detail with
examples in the proposal document.  But conceptually it is pretty
simple; each ``statement'' is executed in sequence, and each chops 0
or more chars off the front of the string and then produces a string
to catenate into a domain name.  The following specify what to chop
off:

* eat-including: chops right after the first character that appears
  in the specified delimiter set.

* eat-until: chops off right before the first char that appears in
  the specified delimiter set.  

* eat-x-chars: eats a specified number of chars.  

* match-prefix: fails unless the specified string is a prefix of the
  string, and if it matches eats the prefix.  (We may want a case-
  insensitive version of this as well..)

* match-rest: fails unless the rest of the string is an exact
  match, if it matches eats the rest of the string.  

Once a prefix is chopped off, it can either be replaced with a
specified string (&replace-with) or copied verbatim (&copy).  The
result strings is catenated in order and then the result is reversed
to produce the domain name to look up.  For example, continuing this
lisp metaphor:

  The input is eaten in the following parts:

	mail: edu.mit.lcs @ girod
             ^           ^ ^
          
  --> (dns-reverse (catenate "" "edu.mit.lcs" ".mail-urn"))
  --> (dns-reverse "edu.mit.lcs.mail-urn")
  --> "mail-urn.lcs.mit.edu"

Hope this clarifies things, and thanks a lot for the comments,

	- Lewis

Date: Tue, 13 Aug 1996 15:02:24 +0100 (BST)
From: Jon Knight <[email-masked]>
To: Lewis Girod <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: [URN] nasty rewriting rules
Sender: [email-masked]
Reply-To: Jon Knight <[email-masked]>

On Tue, 13 Aug 1996, Lewis Girod wrote:
> (4) Regexp interpreters are hard to implement if they are not already
> available on the target system, whereas the languages in my proposal
> have been implemented quite easily.

Regexp interpreters are already available in source code format that you 
can just plug and play on many current platforms and which will ease 
porting to new architectures.

> There are two questions here.  First, this statement is only true
> given that the structure of the name scheme actually remains
> consistent with this set stucture in the context of NAPTR (this gets
> back to point (1) above).  If they were resolved with NAPTR using
> regexps little would prevent someone from changing the format of an
> ISBN in various ways; there is no technical impediment to this, and if
> the top level ISBN naming authority doesn't want it to happen their
> only recourse is legal.

Surely though as long as the end users are using "legal" ISBNs in their
URNs, the rewriting that happens is just the business of the people
"owning" that ISBN and their resolution agents (might be one and same
thing in some cases).  If the resolution agents at the top level of a
naming authority reject "illegal" constructs in that scheme that they
might get given, people will soon get the idea and stop using them.  If
the top level naming authority never see the "illegal" rewritten versions
then that's fine as it means that the rewriting is happening in private
lower down the resolution tree.  I'd say that "enforcement" is just a
private thing within a particular naming scheme resolution tree and isn't
something we should be dipping our toes into (not if we want URNs in this
lifetime anyway). 

> This should make it easier for
> people to learn and program in the ``language'' (if such a simple
> thing can be called a language!!) while at the same time making syntax
> errors less likely.  For example, we could use S-expressions (pardon
> my rough adherence to standard BNF form..):

I can feel an Emacs mode coming on... :-) :-)

> So for example, given a canonical form email address mail:edu.mit.lcs@girod
> 
>   ((eat-including ":" &replace-with "")
>    (eat-until "@" &copy)
>    (eat-including "@" &replace-with ".mail-urn"))
> 
> Would ``compile'' to ``x":"+p"";x"@"-v;x"@"+p".mail-urn";'', and would
> generate the pair ("mail-urn.lcs.mit.edu", "girod").  Note that the
> translation took care of removing the ``urn:'' if it was there.  This
> or something like it should be simple enough to learn.

And this is simpler that regexps?  As a sysadmin I'd much rather use 
something I'm already used to (ie: regexps) rather than learning (and 
making mistakes in) another new syntax.  Of course those of us that do 
want regexp are still free to have a "NAPTRclassic" (:-) ) namespace 
hived off using SRV records that does do regexp replacement, so maybe we 
can all have our cake and eat it?

Tatty bye,

Jim'll

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Jon "Jim'll" Knight, Researcher, Sysop and General Dogsbody, Dept. Computer
Studies, Loughborough University of Technology, Leics., ENGLAND.  LE11 3TU.
* I've found I now dream in Perl.  More worryingly, I enjoy those dreams. *

Date: Tue, 13 Aug 1996 02:28:54 -0400
From: Lewis Girod <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: [URN] nasty rewriting rules
Sender: [email-masked]
Reply-To: Lewis Girod <[email-masked]>

******** NOTE: ********
*
*  This reply is written in reference to the document and code at
*
*    http://ana-www.lcs.mit.edu/people/girod/translator.c  (~40K)
*
*  The file is mostly document, and explains in great detail with examples the
*  proposal discussed in this reply.
*
***********************

   On Fri, 02 Aug 1996 15:29:58 -0600, Ron Daniel <[email-masked]> wrote:

   So, what *do* we want to avoid about regexps? [...] You said that your 
   primary objection to them was that adopting them now might prevent future
   resolution systems that are not also based on general rewrite rules. With
   respect, I think that reason is a red herring. 

A1. What do we need to fix about Regexps?

I didn't spend a lot of time explaining this clearly in the original
proposal document, so I will summarize what I thought should be fixed:

(1) We should enable specific namespace schemes to be enforceable.  I
think this is a better way of putting what I had said originally,
i.e. preventing future name schemes from being contorted.  While I
agree that it would be a patently bad idea to _specify_ a contorted
name scheme, with regexp rules at the basis of a resolution system the
whole idea of a name scheme specification no longer applies.  At any
level of delegation some part of a URN is likely to be essentially
opaque, and lower levels really can't be stopped from varying from the
intended scheme.  This can be seen as a bug or a feature; I consider
it a bug.  To me it is better to explicitly define a scheme and
relegate things that do not fit to other schemes (see discussion in
below in sections A4.1 and A3).  You mentioned that one difficulty
might be ``constructed OIDs'', which I am not familiar with...  I
would suspect that these could be just as easily constructed into a
different name scheme (perhaps by prepending URN:oid-xyz: or
something) rather than changing the previously defined format of OIDs.
Further, part of this problem is caused by the present lack of a
resolution system; once one exists people will make namespaces that
fit it.  How are constructed OIDs currently used and resolved?

(2) We should simplify the construction of maintenance interfaces.
Maintainers need to understand what is going on with the system and
need to keep the rules working as new functionality is added to them.
It would be nice if this can be done with a simple interface which
could display the data in an easy to follow manner and which provides
static and end-to-end checks to ensure that the system is working.
Such an interface is sketched out below in section A2.  The regexps
are hard to follow and at each stage the whole string has to be
processed to figure out what part is relevant.  This makes them harder
to fit into an interface.

(3) We should try to keep the bulk of the data in a format that is
easily convertible in the future.  This might be a feature of the
maintainance interface, since the maintenance interface could keep the
data in a canonical form which is then compiled into rules (see
section A2.3).  It seems to me that this sort of processing is much more
difficult to do with regexps than with a simpler rule language.  But
to the degree that name scheme hierarchy can be specified (depending
on the details of each specific scheme) it may be possible to have a
single canonical data format for an entire name scheme.

(4) Regexp interpreters are hard to implement if they are not already
available on the target system, whereas the languages in my proposal
have been implemented quite easily.

   Currently, there are a variety of naming schemes in existence that I
   would like to be able to resolve as URNs. (ISBNs, OIDs, FPIs, etc). 
   The structure of those names is already set, and the ability of future
   systems to crack that structure is not affected by what we allow in NAPTRs.

There are two questions here.  First, this statement is only true
given that the structure of the name scheme actually remains
consistent with this set stucture in the context of NAPTR (this gets
back to point (1) above).  If they were resolved with NAPTR using
regexps little would prevent someone from changing the format of an
ISBN in various ways; there is no technical impediment to this, and if
the top level ISBN naming authority doesn't want it to happen their
only recourse is legal.

Second, I am curious to know how NAPTR would resolve these three name
schemes.  I can see how to do ISBN (if there are -'s in it), and OIDs
shouldn't be a problem, even constructed ones as long as the
constructions are in the set of things regexp rules can handle.
Similarly, both of these can be dealt with by my proposal, although
the set of interpretable constructions is more restricted.  Since FPIs
are the other space we want to handle, has anyone figured out how
regexps would be used to resolve them?  From what I can tell it seems
tricky.

   I certainly don't want us to do something now with NAPTRs that is going to
   come back to haunt us.  [...]
   1) The distinction between the "program" that is used to canonicalize
      the order of any hierarchy in the URN, and the "rules" that are used
      to yank things off the front and make the next domain name, needs to
      be more explicit. Only the people at urn.net need to know a lot
      about the program. Site administrators only need to write the rules.
      Putting them both into the same RR may be confusing.

This is true.  I grafted them into the same record only to show how it
fit into the original NAPTR framework.  In fact, in section A2.1 I
suggest using three records.

   2) What I thought was one of the ameliorating influences in the NAPTR
      proposal was that the regexps would be applied to the original URN
      in its entirety. I think this is a much simpler conceptual model than
      your two-part approach, especially when the second part is rules that
      operate on the output of other rules.
   3) One of the nasty bits about iterative rewrite rules is that upstream
      changes can affect the correctness of my rules. I forsee a class of
      errors arising where delimiters are not handled uniformly in upstream
      rules, making life a pain in the butt for downstream administrators.
      (Can you get rid of the + and - rules for handling delimiters and
       just say that if a character is matched, it is eaten?)
   4) While error-prone, regexps are nonetheless familiar to a large number
      of people. The syntax of your proposal (which I understand you can
      change) does not seem to be an improvement on regexps, and is familiar
      to no-one.
   5) "rules" in your model have two outputs - the domain name to be
      looked up and the tail of the canonicalized string to be left for
      subsequent rules. This is a bit warty, but seems unavoidable in
      rules that are meant to be applied to the output of other rules.

I think that all four of these points can be addressed at the level of
management software.  Because of the simplification of the rules
involved, the management software is easier to specify, and with a few
added features I think it can ameliorate the issues with invisible
breakage downstream.

A2.  A General Framework for Management Software

A2.1 Records

The management software described here is used to create and update a
set of DNS records on a given server.  There are four types of record
involved here:

(1) SRV records, providing pointers to terminal servers

(2) NAPTR records, providing an order parameter and a ruleset decribing
how to generate the next domain name

(3) NIDSPEC records, which provide for a given name scheme a
translation program for rendering URNs canonical and an initial
ruleset that generates the next domain name

(4) NADOC records, which contain documentation corresponding to a
given NIDSPEC or NAPTR, as described below.

The description of this management software is very sketchy right now
but you should be able to see the intent and how it might work.  I
think there are some clever simplifications that can be made to the
design (especially with respect to the storage and signing of
``agreements'') but I haven't thought it through enough to see them.
I offer this mainly as a proof of concept, not as a final solution.
To explain how this management software works, let us consider an
example.

A2.2 Contracts Between Clients and Administrators

At a site below the top level, management is taking place over a
collection of data that is referenced by domain name.  Suppose our
site is foo.com, and they are in the business of being a ``name
authority'' (or at any rate providing one step of resolution for a
name authority); then xxx.foo.com retrieves data (NAPTR, NADOC, SRV
records, etc.) from foo.com's name server.  The purpose of the
management software is to generate foo.com's name database based on
their clients' needs.

The source data contains two logical types of record:

(1) client-instance descriptors, which describe rules that have a
fixed set of valid operations.  For example, a rule might apply only
to two specific clients who can be listed explicitly:  the rule
``m"girod/"*p"edu.mit.lcs.skadhwe";'' would only apply to my URNs
and would direct clients only to my machine.  Other rules might 
apply to two or a hundred clients.

(2) client-class descriptors, which describe a rule that is applied to
an unspecified set of clients.  For example, for a rule that locates a
domain name in the text of the URN and appends ``.urn.net'', it might
not be convenient to list all possible clients.

The format of a client-instance descriptor is as follows:

client-instance {
	client agreement list = list of client-agreement
	order = integer
	ruleset = string  // in user-freindly form (see below)
}

client-agreement {
   // specification of expected input
	other info about referer (parent) = (implementation specific)
	domain name reference came from = string
	sample expected tail of canonical URN from above = string

   // specification of promised output
	other info about client = (implementation specific)
	domain name of client referring to = string
	tail expected by client at above domain name = string
}

Client-class descriptors need not be implemented separately; they can
be implemented as trivial instances of client-instance descriptors by
simply listing any relevant examples.

These descriptors would be set up as a result of an agreement between
the naming authority and the clients, and would be fixed.  The source
data is stored in NADOC records and can be retrieved as such by the
clients.  In the NADOC record the agreement (excepting the rule
specification) can be signed by the agreeing parties so that the
agreement used to check the rule can be verified.  The site
administrator can update the rules, etc, but before the database is
actually updated, the software runs a static check against the
input/output specifications to make sure they work.  If so the
database is modified and an end-to-end test is made.

The explicit specification of agreements provides safety against the
breakage problem and the data involved can be kept in a fairly
canonical form (i.e. sets of client-instance records) which should be
easier to migrate to new systems.

A2.3 Canonical Data Storage for Automated Generation of Contracts

For some name schemes, it will be possible to have an even more foolproof
system, one which generates the rules directly from a lexical list of 
namespaces.  This is easiest in name schemes with simple syntax.  A 
list of namespaces (using the model of delegation specified in section 4
of our I-D, which is similar to that of the ``path'' scheme) such as

       urn:bar:foo/a/* --> a.com
       urn:bar:foo/c/* --> c.com
       urn:bar:foo/c/fg/* --> fg.com
       urn:bar:foo/x/* --> x.com
       urn:bar:foo/x/y/z/* --> z.com
       urn:bar:foo/x/yz/* --> yz.com
      
could be used to automatically generate all of the rules and
agreements involved, perhaps assuming that the tail string coming in
begins right after ``urn:bar:foo/'' has been eaten (because at the
previous level there was a namespace definition ``urn:bar:foo/* -->
foo.com'').  The various clients involved sign the agreements, and
when changes are made most of the agreements stay the same, and any
that change will need to be signed again.  As a rule the agreements
tend to remain pretty constant; mostly rules are added over time, not
changed.

A2.4 A User Friendly Rule Syntax

Another important feature of this software should be a small rule
compiler that translates an easy-to-understand language into the terse
format transmitted to the browsers.  This should make it easier for
people to learn and program in the ``language'' (if such a simple
thing can be called a language!!) while at the same time making syntax
errors less likely.  For example, we could use S-expressions (pardon
my rough adherence to standard BNF form..):

<RULESET> = 
	( *<STMT> )
<STMT> = 
	( eat-until <DELSET> <DISPOSITION> ) |
	( eat-including <DELSET> <DISPOSITION> ) |
	( eat-x-chars <INTEGER> <DISPOSITION> ) |
	( match-prefix <STRING> <DISPOSITION> ) |
	( match-rest <STRING> <DISPOSITION> )
<DELSET> = <STRING>
<DISPOSITION> = 
	&replace-with <STRING> |
	&copy

So for example, given a canonical form email address mail:edu.mit.lcs@girod

  ((eat-including ":" &replace-with "")
   (eat-until "@" &copy)
   (eat-including "@" &replace-with ".mail-urn"))

Would ``compile'' to ``x":"+p"";x"@"-v;x"@"+p".mail-urn";'', and would
generate the pair ("mail-urn.lcs.mit.edu", "girod").  Note that the
translation took care of removing the ``urn:'' if it was there.  This
or something like it should be simple enough to learn.

   6) I am not so sure that your matching rules will continue to be simpler
      than the major ideas in regexps. Point 3 showed one case where optionality
      would be extremely helpful, so that upstream changes in matching on a
      delimiter would not kill us. The religious debate on "urn:" is another.
      FPIs are a namespace this group is supposed to try to accommodate. They
      use :: and // in broadly equivalent ways. So, alternation is another
      capability that probably needs to be in the rewriting language. If those
      go into your language, you are well on the way to regexps.

At this time I have chosen to attack the upstream breakage problem
differently; the URN: issue should be handled at the level of the
canonical translation and won't affect the rulesets.  In section 5 of
the proposal document I discuss the issues that I see surrounding
``URN:''; I think that we should make an effort to resolve this debate
sometime soon.

FPIs are as yet still an unknown here, and indeed they may very well
be difficult to handle.  I was hoping to figure out how to do so after
seeing a general plan for doing so with rewrite rules, mainly because
it seems possible to me that the NAPTR system might be used to handle
some sections of that namespace while relying on other mechanisms to
handle the remainder.

   Another concern that I have is the growth of the number of rewrite rules
   over time as portions of collections are delegated to other agencies.
   This is a concern about rewriting, and applies to either your scheme or
   to regexps.

This is true, and I see this as the central reason for making sure
that NAPTR can be replaced by something that will still be efficient
when partial delegations become more the rule than the exception.  It
is my opinion that the translation portion of this proposal will come
in handy when a new system is implemented.  

A3. How Canonical Tranlsation Fits into the Future

In the future, I think canonical translation will be a very useful
thing.  The main reason is that if the direction of hierarchy is
consistent, information about prefixes can be cached (i.e. classes of
URNs that share the same resolution information).  Without a canonical
hierarchy such cache systems would need to understand the hierarchy
format of any name scheme from which prefixes will be cached, and URNs
for which the hierarchy is not understood could only be cached
individually.  To my way of thinking the logical place to do this
transform is at the client (if that is in fact possible..) so that
URNs can be handled consistently within the resolution/caching
mechanism until the terminal resolver is reached (which should operate
on the original URN).  This is a URN framework issue.

Ignoring for the moment the question of whether name schemes should in
fact be specified (see discussion in A4.1), a given translation
language has a finite and _definite_ capacity to render URNs
canonical.  If we make a choice and specify a language it enables a
system to process a subset of possible URNs in a consistent manner.
Other schemes for URNs would need to be handled by a different system,
which could be implemented later with a more capable translation
language encompassing a different set of URNs, and so on.  In the
short term, the initial translation language would be coded into the
browser, and it might be wise for browser implementations to leave
room to plug new functionality into the translation language.  My
guess is that a translation language capable of dealing with the
existing namespaces will provide sufficient flexibility that new
namespaces will have no trouble fitting into the system.

   What I would like to see is some more analysis of what capabilities
   are needed (like the stuff above on why I think alternation and
   optionality are needed). 

A4.  What Capabilities are Needed and Why?

As we have seen, the basic theory in terms of what is needed is that
by folding stuff into the translation layer the rules layer doesn't
need much functionality at all.

A4.1 Specification of Hierarchy on a Per-Name-Scheme Basis

One potential problem with this proposal is its presupposition that
the hierarchy for a name scheme can be specified, which may or may not
be valid (i.e. constructed OIDs.)  I think we need to establish
whether this is a realistic problem.  Given that there are namespaces
that cannot be rendered canonical in a single preprocessing step, can
we estimate the negative impact of escaping to another mechanism at
the level at which the specification no longer holds (in the case of
OIDs many of them would be in canonical form already)?

One possible ameliorating factor here is that because the translation
layer represents a partial specification of name scheme syntax, there
may be a subsequent reduction in the bother associated with allowing
new name schemes to be developed (the framework document suggested
that there should be standards for admission, etc), does this then
imply that names (such as constructed OIDs) that violate their
``default'' specification might instead be created in a (possibly new)
name scheme for which they are structured according to spec?

I am fairly convinced that the benefit gained by the canonical
hierarchy well outweighs the problems it causes.  The central
benefits, which I have touched on elsewhere in this message, are

(1) Simpler rules in the NAPTR scheme

(2) Makes it easier for management software to store data canonically

(3) Makes caching easier for future systems

(4) Allows name schemes to specify a hierarchy and enables technical
solutions to enforce that specification to a much greater degree

The problems are:
 
(1) Some existing namespaces may be incompatible and will need to
escape to separate resolution technologies

(2) Some future namespaces may be incompatible or may be compelled to 
maintain compatibility

The question is: how many URNs and namespaces are we talking about,
and is it a problem for them to make the (possibly minor)
modifications required to be compatible?

Assuming for the moment that the name schemes can be sufficiently
specified, the next question is: what sorts of things will we need to
handle after canonicalization and how powerful does the rule language
have to be.  Ron brought up optionality and alternation as two things
we will need.  In many cases this can be handled with the language
currently specified.

A4.2 Support for Alternation

If we want to pull off a token delimited by either :: or // we can
determine which of these cases we have using the match construct.  For
example, if the following two rules are tried on "foo::bar" and
"foo//bar":

	((eat-until ":/" &copy)
	 (match-prefix "::" &replace-with ".")
	 (eat-until "" &copy)
	 (eat-x-chars 0 &replace-with ".colons"))

	((eat-until ":/" &copy)
	 (match-prefix "//" &replace-with ".")
	 (eat-until "" &copy)
	 (eat-x-chars 0 &replace-with ".slashes"))

the results will be ("colons.bar.foo", "") and ("slashes.bar.foo", "")
respectively.  This is because the match-prefix construct fails if it
doesn't match the specified string prefix exactly.  Another way of
handling this would be to replace the syntactic structures "::" and
"//" with special marker characters in the translation step (that way
the initial string can contain a colon or slash).

In short, alternation is possible if the point of alternation can be
recognised by either a set of possible characters or a pre-inserted
marker.

A4.3 Support for Optionality

If we want to detect a port number in a URL if it is there (I suggest
putting one in during the translation step however..), we can do
something like this:

	((eat-until ":/" &copy)
	 (match-prefix ":" &replace-with ".tcp_port_")
	 (eat-until "/" &copy)
	 (eat-including "/" &replace-with ""))

	((eat-until ":/" &copy)
	 (match-prefix "/" &replace-with ".tcp_port_80"))

so,
	com.foo:8001/path generates ("tcp_port_8001.foo.com", "path")
	com.foo/path generates ("tcp_port_80.foo.com", "path")

In short, optionality can be handled if the option can be recognised
by either a set of possible characters or a pre-inserted marker.

A4.4 Other Things?

What else do we need?  I'm not sure.  I have by no means proved that
this language is sufficient and don't know how I would go about doing
that.  However I believe the primary reason for needing functionality
here is grandfathering; barring radical new technologies (which would
probably come with their own radical new resolution mechanisms), new
namespaces will likely make do with what is there if it is reasonably
flexible.  Consequently I am more concerned with specific
counterexamples than I am with generalized ideas.  That is, we look at
the names we have to deal with and try to see how they would be handled
by rulesets and by regexps.  

                            I would also like to see some info on
   the general problems of rewrite rules and what, if anything, we
   can do to overcome them. Some of the problems with rules are:
   1) Confusion on how a result was obtained, typically due to
      successive applications of rules or to overly-capable rewrite
      languages.
   2) Accretion of rules over time making a total rats nest of delegations.
      This is related to 1.

I believe how a result is obtained can be determined by tracing the
lookup process, similarly to the original NAPTR process.  No doubt the
trace may get confusing, and worse, long.  As the number of servers in
the path grows this can get to be a real problem.  Also, the number of
rulesets (or, equivalently, NAPTR regexps) returned from a given
lookup may grow as delegations increase in number, meaning that more
data is being transferred back to the client at a given step.  Both of
these issues concern me and affect both proposals in similar ways
(however, as I indicated in section 3.0, the point of this proposal is
more to make it easier to migrate forward later than it is to fix the
long term problems with the resolution mechanism -- it is still really
NAPTR at heart.)

   3) Upstream changes invisibly breaking downstream results.
   4) My site getting no requests / somebody else's requests due to
      upstream errors.

As you point out, the regexp rules don't have that property of
invisible breakage.  Is the mgmt software solution indicated
in section 2.0 a good enough fix for the ruleset proposal?  

Also, note that the limitations on rulesets severely limit the ways in
which rulesets returned from the same lookup can interfere.  It also
makes it easier for mgmt software to be written to verify them.

   The point in starting this thread was to talk about rewrite rules and
   if they were a flaw or a feature. At the BOF, concern was raised about
   rewrite rules and someone said that if meeting the requirements
   requires the power of rewrite rules, and if we don't like that
   answer, do we want to change the requirements? Up to now I have not
   seen any convincing arguments that the requirements can be met without
   using rewriting. Nor have I seen any calls for us to change the
   requirements on URNs, at least not in any fashion that restricts the
   task to be accomplished. (If we were to simplify the requirements, it
   would be appropriate to reconsider the abilities of NAPTRs).

I think there is a third alternative, which more accurately describes
my proposal: _directly_ support all URN spaces with the exception of
those for which the implementation requires a system of regexp rules.
That is, balance the need to support URNs now and in the future with
the relevant technical capabilities.  For example, suppose sorting out
a some subset of the URNs in use required a turing-complete language
while the others could all be processed using a finite automaton; if
the set of URNs gained by adding this functionality (and at the same
time adding the potential to loop) is relatively insignificant, the
best solution may be to escape and resolve them another way.

   If we assume for the moment that rewriting is necessary, then we get
   into discussing what capabilities are needed and how they will be
   expressed in the language that is used for the rewrite rules. This is
   where I think your proposal fits. It represents a noticeable reduction in
   the capabilities of the NAPTR system, since major rewriting can only
   happen at the namespace level when we canonicalize the URN. But that was
   the purpose of the proposal.

True.

   I kind of like the notion of rules that eat off the front of the
   canonicalized name. However, I am not sure that only allowing that
   canonicalization to be performed at the namespace level is enough.
   Especially as collections of resources are delegated over time, I
   think we are going to see lots of URNs that are old URNs plus some
   new stuff. It seems impossible to canonicalize such things with the
   limited amount of information available at the namespace level.
   (See the stuff on "constructed OIDs" below for more info).
	[...]
   I am concerned about the "program"
   portion of the rewrite only being available at the namespace level.
   As a specific example, I came across "constructed OIDs" the day before
   yesterday. These seem to allow "other stuff" to be appended to an OID to
   make a new OID that does not follow the simple rules of integer name
   components. If we have an OID namespace, all it can possibly know
   about constructed OIDs is where the "other stuff" starts and ends. We can't
   provide a program at the namespace level that will canonicalize the
   order of all components. 

This gets at what me want to mean by name scheme specification (see
sections A3, A4.1).  Certainly the original specification of OIDs
doesn't allow these additions?  Do you have more information about
constructed OIDs?

   >* It is already clear that there are some name schemes that cannot be
   >efficiently resolved by this new scheme and will need to be escaped
   >(just as some cannot be conveniently handled by NAPTR).  To what
   >degree is it necessary to add flexibility beyond what is required to
   >grandfather in existing namespaces?

   Well, this sort of depends on what namespaces you want to grandfather in. :-)
   The regexps give us a set of tools to deal with new things as they arise,
   typically in the course of trying to grandfather in a new namespace.
                                                         ^^^^^^^^^^^^^

By ``new namespace'' do you mean one that is recently invented or an
existing namespace?  I suspect it would be more likely that new
namespaces be designed to fit into whatever structure is in place,
rather than go through a lot of trouble to build an alternate
resolution mechanism.  It's easy to invent a namespace that can't be
resolved effectively by NAPTR with regexps -- but is that our problem?

Apologies for the length of this reply, but there were a lot of
questions to answer.  Thank you for your comments on the proposal,

	Regards,

	   Lewis

Date: Mon, 5 Aug 1996 00:03:12 -0400
From: Lewis Girod <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: [URN] re NAPTR, URN-res-req
Sender: [email-masked]
Reply-To: Lewis Girod <[email-masked]>

Ron Daniel wrote on Fri, 02 Aug 1996 15:54:59 -0600:

>I guess I thought that one way that unofficial hints could
>get into the system was by direct injection by users. I think
>it is certainly outside the scope of NAPTRs, but is something to
>keep in mind for the future. There might be a ComMentor-style
>annotation set that was user-supplied resolution hints.

Yes, users could inject unofficial hints as well as publishers.  The
end user trying to locate the resource would probably try (in the
abscence of other knowledge about the resource) the official hint
first, then any hints supplied by the owner of the namespace, then any
hints that appear to be supplied by the author (if that person is
known..), then whatever else is there (users? etc.).  All of this is
definitely outside the scope of NAPTR resolution.. an idea for the
future.

I have not had a chance to address your comments re. the rewrite rule
stuff yet as I have been on vacation this past week and have had only
sporadic net access..

	- Lewis

Date: Mon, 5 Aug 1996 00:02:20 -0400
From: Lewis Girod <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: [URN] re NAPTR, URN-res-req
Sender: [email-masked]
Reply-To: Lewis Girod <[email-masked]>

On Sat, 3 Aug 1996 11:09:48 PDT, Terry Allen <[email-masked]> wrote:

>Would you sketch a sample lookup sequence for a URN for which official
>and unofficial hints have been injected into the system?  Is it a matter
>of user/client preferences as to which hints are heeded?  (See the
>Bobsnamespace example below.)

It would be a matter of end client preferences which hints would be
used first and which would be provided to the user at all.  I envision
the client requesting the official information as well as at most N
unofficial hints from the system for any particular URN lookup.  The
browser would use the official hint info by default while at the same
time pointing out that other information is available.  There might be
other types of preferences as well, I haven't thought about it too
much.

>I am imagining something like this:  Bobsnamespace consists of the
>names 1 through 8.  Bob sells off from this name space 2, 4, 5, and 7,
>rather than a block of names, and further he declines to resolve them
>after he sells them (maybe he goes out of business).  The result
>will be (I think) that no one resolver handles the entire name space,
>and that it is not possible to predict from the URN which of the
>several resolvers for Bobsnamespace should be queried first.  

This is at the crux of the namespace migration problem.  The
generalized scheme I mentioned in the document involves two different
methods of delegation.  When Bob ``sells off'' names from his space,
he has the option of ``officially'' delegating the names or of
handling the delegation himself.  The difference here is that an
``official'' delegation puts the delegation information up at the
``top'' level, in effect flattening the namespace, at least in terms
of layers of resolution mechanism.  For Bob's clients, this has
advantages: (1) Bob has nothing to do with their resolution, because
their URNs go direct to their specified resolution service, so they
are not dependent on him, only on the top level (2) Depending on how
the top level resolution system works, this may be faster since it may
involve fewer steps of resolution.  The disadvantages to official
delegation are (1) a little bit more of the structure of your
namespace is publicly known (2) the top level gets big as time goes on
(but for this to work at all the system has to be able to scale in
response!) (3) it will likely cost money to enter an official
delegation, which is likely passed on to the client.

In answer to Bob's conundrum, if the names are not easily generalized
according to whatever rules exist (for example, simple string prefixes
after a canonicalization process) it will cost a lot more to
officially delegate the names, because in some cases there may simply
be no generalization and official delegation would be commensurate
with listing each URN.  To the degree that the names are not
officially delegated all bets are off when Bob goes out of business.
Presumably, however, Bob or someone else could extort sufficient funds
from of his clients to continue resolving the URNs.. ;-)

In general, with such a system clients who intend to use their URNs
for a long time would be wise to be sure that if something goes wrong
with their service providers they can salvage their URNs in a
cost-effective manner.

>This will work for some, perhaps a great many, URN lookups, but not
>all.  

Yes.  Some URN lookups may want to use a secret or proprietary 
resolution process for example.  No generalized system would be
sufficient in all such cases

>Further, it won't be possible to prevent people from inventing name
>spaces that don't decompose nicely.  The top-level URN name space
>registry can't refuse to register such name spaces, or it ceases to
>be useful.  In addition to grandfathering, there's a requirement to
>be able to "cousin-in" name spaces invented by people who weren't
>aware they weren't following the rules preferred by the top-level
>registry.  These may have to be resolved by brute force (matching
>the entire URN string) or use of other info.

This is true; but if a certain type of service is provided by this
system, it increases the likelihood that new namespaces fit themselves
into it.  To the degree that they don't or can't, they will need to be
escaped to other systems from the top level.  I guess it is a question
of balancing the goal of keeping URNs alive with that of having a
buildable system.  The system I sketched in section 4 and the NAPTR
proposal represent different points on this spectrum.  With any
system, there may be cases where a namespace is a real mess and can't
be effectively split up; either it gets served in total by someone
else or by some other technique, or parts of it are forced into the
system and the remaining parts are lost.

What we are really talking about here is having a dependable top level
sevice, one that doesn't go out of business as we suspect Bob might.
Presumably this dependability would stem from some kind of cooperative
management in which a large number of interested parties run servers
that serve a large distributed database.  This database can escape to
other similar systems if the need arises (or a new version of the
software can be written which extends the capabilities for
generalization exhibited by the original.)  The controversy is over
what basic services should be provided by this top level, and what
services should be delegated to other (possibly less dependable)
levels, and how this in fact affects the goals of long-lived URNs, of
flexibility of namespace design, and of grandfathering in existing
namespaces.

Personally, I think that one simple way to fix a lot of these problems
is to offer each person his/her own namespace (for a very reasonable
fee), in a way that can be generalized so that their URNs are easily
mapped to the resolution service of their choice.  For example,
URN:PEOPLE:unambiguous_name/* would be one syntax (perhaps it would be
best to assign people numbers in order to cut down on lawsuits and
trademark issues).  Doing this guarantees a certain base level of
stability, since people currently tend to keep a constant identity for
say 50 years or so.  Doing this adds a lot of stuff to the database
initially, but the total amount is limited by other factors such as
the availability of food and the asking price for slots in the
database, so scaling issues should be controllable.  How each person
chooses to structure his namespace below this is not specified, but at
least any damage is constrained to that individual's personal space.

This sort of fix would occur after the system is in place (i.e. the
future), and I bring it up mostly as an example of the sorts of
namespace-design techniques that will likely help with these problems,
after the top level system is implemented and the framework is
essentially set in stone, at least for the short term.  I doubt that
any system can keep every URN alive, unless it is prepared to store
the whole list of URNs.  What is more important is getting most of
them (this includes deciding what meaning of ``most'' satisfies you),
and providing ways of setting things up that both satisfy the clients'
requirements and ensure that their URNs will stick around.  Those that
choose not to follow these guidelines may get into trouble.  I think
this is true of any system that works on a principle of generalizing
classes of URNs; for all systems, there is some generalization
strategy is not efficient (for example, if all of your URNs are
products of two large prime numbers... ;-) )

  Regards,

 	Lewis

From: Terry Allen <[email-masked]>
Date: Sat, 3 Aug 1996 11:09:48 PDT
To: Lewis Girod <[email-masked]>, [email-masked]
Subject: Re: [URN] re NAPTR, URN-res-req
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Terry Allen <[email-masked]>

(Slightly out of order.)  Lewis Girod:

| True.  Our omission of an explanation of hints has caused a great deal
| of confusion, which this reply has hopefully addressed to a degree.
| Since we did not have a clear technical plan at the time of writing we
| did not want to do into a lot of detail; we hoped to instead get out
| some ideas we were plaing with and see what sort of reactions were
| generated.  As for the idea of `hints in reverse', the URN community
| should decide if this is within the scope of our problem or whether we
| want to declare it outside.  My opinion is that it is outside; it
| widens the problem and makes the primary technique for resolution
| (i.e. precomputed lexical generalization of URNs) impossible.

Okay with me if it is to be considered out of scope, although I think
that for some classes of resources straight URN lookup will not be
the most common method of retrieval.

| We used the term ``hint'' to mean precisely the data formats used by
| the system to transmit to the client information that aids in locating
| a document given a URN.  Much of our document was intended to be a set
| of general issues and requirements that we thought were critical to
| producing a working URN system; for this reason we did not want to
| specify formats there.  I don't think coming up with formats is too
| difficult; it is more a question of what clients actually understand.
| At the time of writing we had envisioned such formats as the proposed
| NAPTR and SRV records as possible initial `hint' formats, since if the
| NAPTR system was in use this would be the type of data the clients
| would be expecting.  It should be stated, however, that we were not
| intending hints to solve the sorts of semantics-oriented search
| problems you describe; we see this as a higher level of the problem
| that should be kept separate from the straight URN->document(s)
| mapping problem.

Would you sketch a sample lookup sequence for a URN for which official
and unofficial hints have been injected into the system?  Is it a matter
of user/client preferences as to which hints are heeded?  (See the
Bobsnamespace example below.)

Regarding the "straight URN->document(s) mapping problem," I am suggesting
that the real problem is document identification and retrieval, and that
URNs are only one part of its solution.  But if that's out of scope, okay.

| These concerns are definitely something to think about.  In terms of
| naming authorities disappearing, I had envisioned the solution being
| for the naming authority to transfer or sell off its namespace to
| other entities before going away.  For example, if you have some names
| issued by shady.com, they might sell them to you before they go out of
| business, at which point you would be responsible for telling the
| resolution system where your authoritative server is.  In our model

I am imagining something like this:  Bobsnamespace consists of the
names 1 through 8.  Bob sells off from this name space 2, 4, 5, and 7,
rather than a block of names, and further he declines to resolve them
after he sells them (maybe he goes out of business).  The result
will be (I think) that no one resolver handles the entire name space,
and that it is not possible to predict from the URN which of the
several resolvers for Bobsnamespace should be queried first.  

| Another issue that is parallel to this is that the resolution models
| being discussed here all try to deal with URNs in groups in order to
| cut down on the size of higher levels of the database.  NAPTR does
| this in a similar way to the DNS.  The model I tend to think of tends
| to assume that the top level of the directory is pretty big, but still
| is based on the idea of sending all URNs starting with this prefix to
| this particular resolver.  In the end this is a good idea because it
| allows changes to be made at a local level rather than having to
| bother the top level every time.  This means that the system we are
| talking about does not deal with individual URNs under normal
| circumstances; they will always be generalized somehow, in my mind
| usually through the concept of a naming authority.  

This will work for some, perhaps a great many, URN lookups, but not
all.  

Further, it won't be possible to prevent people from inventing name
spaces that don't decompose nicely.  The top-level URN name space
registry can't refuse to register such name spaces, or it ceases to
be useful.  In addition to grandfathering, there's a requirement to
be able to "cousin-in" name spaces invented by people who weren't
aware they weren't following the rules preferred by the top-level
registry.  These may have to be resolved by brute force (matching
the entire URN string) or use of other info.

Regards,

-- 
       Terry Allen      O'Reilly & Associates, Inc.   [email-masked]
 "In going on with these experiments, how many pretty systems do we build,
  which we soon find ourselves obliged to destroy?"  -  Benjamin Franklin
        A Davenport Group sponsor     http://www.ora.com/davenport/

Date: Fri, 02 Aug 1996 15:54:59 -0600
To: Lewis Girod <[email-masked]>
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] re NAPTR, URN-res-req
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

Thus spoke Lewis Girod (at least at 07:22 PM 8/1/96 -0400)
>At 12:32 PM 7/28/96 PDT, Terry Allen wrote:
>
>>Much is made here of "hints," 
[...]
>It should be stated, however, that we were not
>intending hints to solve the sorts of semantics-oriented search
>problems you describe; we see this as a higher level of the problem
>that should be kept separate from the straight URN->document(s)
>mapping problem.

This is a useful clarification, thanks Lewis.

>>[...] ignores the power of supplying as input into the URN resolution
>>system ancillary information about the URN (hints in reverse, so to speak).
>
>[...]As for the idea of `hints in reverse', the URN community
>should decide if this is within the scope of our problem or whether we
>want to declare it outside.  My opinion is that it is outside; it
>widens the problem and makes the primary technique for resolution
>(i.e. precomputed lexical generalization of URNs) impossible.

I guess I thought that one way that unofficial hints could
get into the system was by direct injection by users. I think
it is certainly outside the scope of NAPTRs, but is something to
keep in mind for the future. There might be a ComMentor-style
annotation set that was user-supplied resolution hints.

>Under such a
>system, [the canonicalization proposol for an alternative to regexps]
>a name scheme designer can enforce certain policies about the
>format and semantics of the names under that scheme, whereas according
>to the original NAPTR model the top-level name scheme design had no
>impact on the structure underneath.

Something I left out of my previous message (hard to believe I left
anything out, I know :-)...
One of my concerns about the canonicalization proposal was that
a single program at the namespace level could not deal with constructed
names. How about allowing that program to insert a marker into the
canonical form of a name. When the client sees the marker, it means
that it would be legal for the incoming NAPTR to provide a program
for dealing with the rest of the URN. This way we get the restrictions
on permissable behavior that you want and can still handle constructed
names, which I think will become an important thing to do.

Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Fri, 02 Aug 1996 15:29:58 -0600
To: Lewis Girod <[email-masked]>
From: Ron Daniel <[email-masked]>
Subject: Re: [URN] nasty rewriting rules
Cc: [email-masked]
Sender: [email-masked]
Reply-To: Ron Daniel <[email-masked]>

At 07:25 PM 8/1/96 -0400, Lewis Girod wrote:

>This message is in response to a thread started a long time ago by
>Ron, namely, what about rewrite rules?

Thanks for picking up on that thread Lewis.

I have mixed opinions about your proposal. You are right that we chose
regexps in a very ad-hoc fashion. Once we decided that rewriting was the
solution to the problem of separating resolution procedures from name
assignment procedures, we needed to have a language for expressing
the rewriting rules. Regexps appeared to have the power to
do what was needed, we were all familiar with them, and we didn't have to
spend a lot of time trying to define and implement our own rewrite
language. The last point, of course, is a big part of why we settled on
regexps.

I'm happy to see that you have not flinched from the effort that we backed
away from. However, I don't think your proposal is ready to replace regexps,
for reasons I'll get into below. But it is a start and there are things I
like about it.

So, what *do* we want to avoid about regexps? You said that your primary
objection to them was that adopting them now might prevent future
resolution systems that are not also based on general rewrite rules. With
respect, I think that reason is a red herring. I understand the concern,
but don't think it holds up.
Currently, there are a variety of naming schemes in existence that I
would like to be able to resolve as URNs. (ISBNs, OIDs, FPIs, etc). 
The structure of those names is already set, and the ability of future
systems to crack that structure is not affected by what we allow in NAPTRs.
NAPTRs only interfere with future resolution systems if the
developers of future namespaces look at the capabilities of regexps and
create a contorted naming scheme to take advantage of them. I submit that
such contortions are extremely unlikely. Furthermore, they could be handled
in future systems through escape mechanisms similar to those that NAPTR
will utilize for several of the above namespaces. 

I certainly don't want us to do something now with NAPTRs that is going to
come back to haunt us. I am much more concerned about other modes of
failure in the future than contorted namespaces. My first concern is
that administrators creating NAPTRs will get them wrong. I'm the first
to admit that regexps are prone to errors. However, I think that your
proposal is no better on that front, and is quite possibly worse.
1) The distinction between the "program" that is used to canonicalize
   the order of any hierarchy in the URN, and the "rules" that are used
   to yank things off the front and make the next domain name, needs to
   be more explicit. Only the people at urn.net need to know a lot
   about the program. Site administrators only need to write the rules.
   Putting them both into the same RR may be confusing.
2) What I thought was one of the ameliorating influences in the NAPTR
   proposal was that the regexps would be applied to the original URN
   in its entirety. I think this is a much simpler conceptual model than
   your two-part approach, especially when the second part is rules that
   operate on the output of other rules.
3) One of the nasty bits about iterative rewrite rules is that upstream
   changes can affect the correctness of my rules. I forsee a class of
   errors arising where delimiters are not handled uniformly in upstream
   rules, making life a pain in the butt for downstream administrators.
   (Can you get rid of the + and - rules for handling delimiters and
    just say that if a character is matched, it is eaten?)
4) While error-prone, regexps are nonetheless familiar to a large number
   of people. The syntax of your proposal (which I understand you can
   change) does not seem to be an improvement on regexps, and is familiar
   to no-one.
5) "rules" in your model have two outputs - the domain name to be
   looked up and the tail of the canonicalized string to be left for
   subsequent rules. This is a bit warty, but seems unavoidable in
   rules that are meant to be applied to the output of other rules.
6) I am not so sure that your matching rules will continue to be simpler
   than the major ideas in regexps. Point 3 showed one case where optionality
   would be extremely helpful, so that upstream changes in matching on a
   delimiter would not kill us. The religious debate on "urn:" is another.
   FPIs are a namespace this group is supposed to try to accommodate. They
   use :: and // in broadly equivalent ways. So, alternation is another
   capability that probably needs to be in the rewriting language. If those
   go into your language, you are well on the way to regexps.

Another concern that I have is the growth of the number of rewrite rules
over time as portions of collections are delegated to other agencies.
This is a concern about rewriting, and applies to either your scheme or
to regexps.

What I would like to see is some more analysis of what capabilities
are needed (like the stuff above on why I think alternation and
optionality are needed). I would also like to see some info on
the general problems of rewrite rules and what, if anything, we
can do to overcome them. Some of the problems with rules are:
1) Confusion on how a result was obtained, typically due to
   successive applications of rules or to overly-capable rewrite
   languages.
2) Accretion of rules over time making a total rats nest of delegations.
   This is related to 1.
3) Upstream changes invisibly breaking downstream results.
4) My site getting no requests / somebody else's requests due to
   upstream errors.
What other problems with rewrites can we come up with? What tools can
we provide to ease those problems?

I would also like to know how regexps or alternative proposals such
as yours impact the problems above. Regexps may be an overly general
language to use to specify the rules. Your proposal may fall into the
trap of confusing results due to applying rules to the output of
other rules. Both are subject to errors such as 4.

>All of this
>can be found at
>
>   http://ana-www.lcs.mit.edu/people/girod/translator.c  (~40K)
>
>I am curious to see what you think.  

I will have some specific comments on that document later.

>* I think that this addresses a major design flaw of the original
>NAPTR system.  Do you?  Are the rewrite rules a flaw or a feature?  If
>they are a flaw does this address that flaw?

The point in starting this thread was to talk about rewrite rules and
if they were a flaw or a feature. At the BOF, concern was raised about
rewrite rules and someone said that if meeting the requirements
requires the power of rewrite rules, and if we don't like that
answer, do we want to change the requirements? Up to now I have not
seen any convincing arguments that the requirements can be met without
using rewriting. Nor have I seen any calls for us to change the
requirements on URNs, at least not in any fashion that restricts the
task to be accomplished. (If we were to simplify the requirements, it
would be appropriate to reconsider the abilities of NAPTRs).

If we assume for the moment that rewriting is necessary, then we get
into discussing what capabilities are needed and how they will be
expressed in the language that is used for the rewrite rules. This is
where I think your proposal fits. It represents a noticeable reduction in
the capabilities of the NAPTR system, since major rewriting can only
happen at the namespace level when we canonicalize the URN. But that was
the purpose of the proposal.

I kind of like the notion of rules that eat off the front of the
canonicalized name. However, I am not sure that only allowing that
canonicalization to be performed at the namespace level is enough.
Especially as collections of resources are delegated over time, I
think we are going to see lots of URNs that are old URNs plus some
new stuff. It seems impossible to canonicalize such things with the
limited amount of information available at the namespace level.
(See the stuff on "constructed OIDs" below for more info).

>* Can this two-step process handle everything we want it to handle?
>If not, are there modifications to the languages that will enable it
>to do so?  (In general, we can increase the complexity of the
>translation language as needed, but that makes it harder to implement
>and inreases the probability of bugs or loops)

I am concerned about the "program"
portion of the rewrite only being available at the namespace level.
As a specific example, I came across "constructed OIDs" the day before
yesterday. These seem to allow "other stuff" to be appended to an OID to
make a new OID that does not follow the simple rules of integer name
components. If we have an OID namespace, all it can possibly know
about constructed OIDs is where the "other stuff" starts and ends. We can't
provide a program at the namespace level that will canonicalize the
order of all components. 

I've already mentioned alternation and optionality as two capabilities
that will probably be needed.

>* It is already clear that there are some name schemes that cannot be
>efficiently resolved by this new scheme and will need to be escaped
>(just as some cannot be conveniently handled by NAPTR).  To what
>degree is it necessary to add flexibility beyond what is required to
>grandfather in existing namespaces?

Well, this sort of depends on what namespaces you want to grandfather in. :-)
The regexps give us a set of tools to deal with new things as they arise,
typically in the course of trying to grandfather in a new namespace.

>How much flexiblity is needed
>beyond what is provided by this system, and how much more does the
>original NAPTR proposal provide?

Optionality, alteration, and the ability to deal with constructed
names are the ones that I can think of now. 

Regards,
Ron Daniel Jr.                       email: [email-masked]
Advanced Computing Lab               voice: +1 505 665 0597
MS B287                                fax: +1 505 665 4939
Los Alamos National Laboratory        http://www.acl.lanl.gov/~rdaniel/
Los Alamos, NM, USA  87545       tautology:"Conformity is very popular"

Date: Thu, 1 Aug 1996 19:25:10 -0400
From: Lewis Girod <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: [URN] nasty rewriting rules
Sender: [email-masked]
Reply-To: Lewis Girod <[email-masked]>

Hi,

This message is in response to a thread started a long time ago by
Ron, namely, what about rewrite rules?  I have this idea that the
client side of the NAPTR system that currently uses the rewrite rules
can be replaced by a two-step process which gives a better modularity
and will make the data in the system easier to deal with (as well as
making the creation of name scheme security policies much simpler.)

The idea here is to divide the problem into two halves; first, put the
URN into a canonical form in which the direction of hierarchy is
fixed.  For example, the domain name in

	http://lcs.mit.edu/girod/paper.ps 

would need to be reversed so that the hierarchy goes from biggest on
the left to smallest on the right:

	http://edu.mit.lcs/girod/paper.ps

Once the URN is in canonical form, the actual resolution process can
begin.  Because the hard part of sorting out the URN has already been
done in the initial canonicalization phase, the rest of the resolution
can be done using some very simple rules.  These rules are very
constrained in comparison to the unconstrained rewrite rules; they
operate only by taking parts off the left hand side of the canonical
string, and can either copy that data verbatim or replace it with a
specific string.  However, it appears that most stuff we want to do
can be done this way.

If such a scheme can work, there are two main benefits to the new
modularity: (1) the vast majority of the data driving the resolution
process is in a simplified and standardized form, which is easier to
check and understand because there is less flexibility and because the
hard stuff is done once at the beginning, (2) the format of the name
scheme is specified at the top and deviation from this format is more
difficult because of the restrictions placed on rules below the
initial translation.

I have invented two small languages (a translation language and a rule
language) and written up a document that explains them and gives
examples of how they can be used to do everything I could think of.  I
have also provided C code that implements the interpreters for these
languages, and some pseudocode that shows how it can be used to
operate the NAPTR resolution process.  The syntax of the languages is
terse and ugly but really easy to implement and at bottom very simple.
The syntax can probably be cleaned up significantly...

In total, I think this modification requires little in the way of
modification to the server side of the existing NAPTR code, but it
does modify the client significantly.  The C code provided should be
pretty portable and has even been tested to a degree... All of this
can be found at

   http://ana-www.lcs.mit.edu/people/girod/translator.c  (~40K)

I am curious to see what you think.  

* I think that this addresses a major design flaw of the original
NAPTR system.  Do you?  Are the rewrite rules a flaw or a feature?  If
they are a flaw does this address that flaw?

* Can this two-step process handle everything we want it to handle?
If not, are there modifications to the languages that will enable it
to do so?  (In general, we can increase the complexity of the
translation language as needed, but that makes it harder to implement
and inreases the probability of bugs or loops)

* It is already clear that there are some name schemes that cannot be
efficiently resolved by this new scheme and will need to be escaped
(just as some cannot be conveniently handled by NAPTR).  To what
degree is it necessary to add flexibility beyond what is required to
grandfather in existing namespaces?  How much flexiblity is needed
beyond what is provided by this system, and how much more does the
original NAPTR proposal provide?

	- Lewis Girod
        
         ([email-masked])

Date: Thu, 1 Aug 1996 19:22:43 -0400
From: Lewis Girod <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: [URN] re NAPTR, URN-res-req
Sender: [email-masked]
Reply-To: Lewis Girod <[email-masked]>

At 12:32 PM 7/28/96 PDT, Terry Allen wrote:

>Much is made here of "hints," which I understand as heuristic instructions
>that the client or intermediary can use to determine how to start 
>the URN resolution process.  I think something like this is 
>essential, but it needs fleshing out.  Can a syntax or format 
>of hints be generalized?  (My guess is that it can't.)  
>Can the semantics of hints be generalized:  "for anything
>written by author={Jane Austen} see the Jane Austen Home Page"?  (I think
>this might be possible for well defined bibliographic data.)

This was an area in which we were not very specific.  In our private
discussions while writing this we developed a concept we mean by
``hints'' which we unfortunately forgot to explain clearly in the
document.  

We used the term ``hint'' to mean precisely the data formats used by
the system to transmit to the client information that aids in locating
a document given a URN.  Much of our document was intended to be a set
of general issues and requirements that we thought were critical to
producing a working URN system; for this reason we did not want to
specify formats there.  I don't think coming up with formats is too
difficult; it is more a question of what clients actually understand.
At the time of writing we had envisioned such formats as the proposed
NAPTR and SRV records as possible initial `hint' formats, since if the
NAPTR system was in use this would be the type of data the clients
would be expecting.  It should be stated, however, that we were not
intending hints to solve the sorts of semantics-oriented search
problems you describe; we see this as a higher level of the problem
that should be kept separate from the straight URN->document(s)
mapping problem.

>I may also wish to discover all resolvers for a given URN, because
>(especially after the publisher has gone away) some may return a 
>better quality object than others--remember that "sameness" is defined 
>by the publisher or namer, and an URN doesn't necessarily resolve
>to a unique object.  Has this task been considered in either draft?

In the models we are considering, this problem would typically be
solved using unofficial hints.  This is because there is an assumption
that the URN in question falls into the domain of some naming
authority, and they are providing (or not) official hints as to where
the URN can be resolved.  They may not provide or even be aware of all
relevant resolvers; for this reason collections of unofficial hints
may be provided by anyone that can direct people to alternate
resolvers.  In the long run unofficial hints are less efficient to
maintain and are in general less certain (the models we consider do
not guarantee that unofficial hints are provided with the accuracy
accorded to official hints), so they should be considered to be
temporary fixes for problematic situations.  In answer to your
question, in such a system all advertised resolvers would be
determinable.

>But in time, as NAs disappear, sometimes leaving large messes behind
>them, there will be URNs for which no single authoritative resolver
>exists; this problem is only compounded for hints, which offer fertile
>...
>Thus much of section 4.1 is moot.  One cannot require that there be
>authoritative URN resolution services; there will be simply services.
>Whether they work for you or not is something you'll have to do your
>homework on.  The hints you'll find most valuable may be those
>supplied by third parties, or discovered by your own desktop client
>based on experience; and there may be no correlation with their
>supposed authoritativeness.

These concerns are definitely something to think about.  In terms of
naming authorities disappearing, I had envisioned the solution being
for the naming authority to transfer or sell off its namespace to
other entities before going away.  For example, if you have some names
issued by shady.com, they might sell them to you before they go out of
business, at which point you would be responsible for telling the
resolution system where your authoritative server is.  In our model
``authoritative server'' does not imply much of anything in terms of
authenticity -- it merely is the server that the naming authority
owning the names says is authentic.  This is another place where
unofficial hints come into play; an unofficial hint may give
information that contradicts the ``official'' information from the
``official'' naming authority.  In the end it is up to the user to
decide who is telling the truth; it is quite possible that the
unofficial hints are written by the author of the eventual resource
(and authenticatable) but to know that you need to know who the author
is, etc.  Note that our ideas of having unofficial hints integrated
into the system do not preclude third party hint systems!  In fact
third party hint systems are a really good idea, but from my
perspective it seems best to also try to integrate them into the
official infrastructure in the interests of fairness (it may not
succeed, of course... :-) ).

Another issue that is parallel to this is that the resolution models
being discussed here all try to deal with URNs in groups in order to
cut down on the size of higher levels of the database.  NAPTR does
this in a similar way to the DNS.  The model I tend to think of tends
to assume that the top level of the directory is pretty big, but still
is based on the idea of sending all URNs starting with this prefix to
this particular resolver.  In the end this is a good idea because it
allows changes to be made at a local level rather than having to
bother the top level every time.  This means that the system we are
talking about does not deal with individual URNs under normal
circumstances; they will always be generalized somehow, in my mind
usually through the concept of a naming authority.  Note also that
systems set up this way cannot deal with the semantic content of URNs
because that radically changes the way they are hierarchicalized (For
example, sorting on author and on title produce two very different
search trees...)  Since the data they are storing is really just a
description of generalization, these systems always deal with just one
hierarchy and stick to it.

>4.1.2 describes as necessary a Hint Passing Interface, and says
>that hints lie within given naming schemes.  Gosh, this seems
>simplistic.  Why is this a requirement for URN resolution?  Has
>anything of the sort been attempted?  Why must hints be restricted
>to the NA vector ("anything written by author={Jane Austen}")?
>This is not the way we locate most bibliographic information today.  

It should be noted that section 4 describes some ideas we had
for a model or framework for URNs, more than it states requirements.
The requirements we wanted to set are in section 2.  

>BTW, who owns the URN?  the top-level NA, or the NA responsible for
>actually associating the URN with something?  If I offer resolution
>for your URNs, can you object?  especially if the URN denotes an
>object neither of us owns rights in or that cannot be retrieved?  
>Pseudoexamples: 
>FPI:ISO:+ISO/IEC 8070/RA::A00002/Athens/Acropolis/Erectheum
>PP:944940/Domesday Book/page 18
>USArchives:Nixon/tape/18.5-min-gap

Interesting question.. in the end I suppose this is a legal issue, but
in my opinion I would say no.  Just because a URN is resolved to a
location does not imply that (a) the location serves it to you, or (b)
the location admits that it exists.  I suppose that certain types of
resolution process might be proprietary, and in that case executing
the algorithm might be illegal even if it was independently derived.
I have never agreed with this, but that probably has little bearing on
reality.

>4.2.1 "There is a need for an effective model of name delegation."
>This form of words, "there is a need", is becoming all too common.
>What is the need?  if the need is not satisfied (and it never
>will be fully satisfied in practice), does URN resolution become 
>impossible or merely messy?

Well, to the degree that the need is not satisfied things may get
messy and/or people may get annoyed.  I don't think it is the end of
the world, nor is it entirely avoidable, but I do believe that certain
design choices can make a difference.  In this section I was trying to
analyse exactly what people want out of a name and how that interferes
with the desired property of longevity.  The essential problem is that
in the short term people dont generally care about longevity so they
do things that may screw them later.  The fact is that if what gets
built is pretty much good enough, people will use it even if it doesn't
really do quite what they want, especially if it is the first thing 
that came along and everybody's browser understands it.  For that reason
I think that setting a few design limitations that happen to prevent
certain types of predictable disaster might be a good thing overall.

>4.2.2 states some obvious things that will be under the control of
>NAs, which will do as they please.  The behavior of NAs is not
>within scope for the IETF, so long as they don't overlap each
>other's namespaces.

True, but these constraints only affect them if they choose to utilize
the system that is being described in section 4.  If they want to do
something else, they are free to escape to their own entirely separate
and unregulated resolution system.  (Note that this is a problem for
NAPTR as well, although it is not explicitly stated.  There are
namespaces which do not lend themselves to NAPTR-style decomposition
and thus would need to be escaped.)  The question is, how restrictive
is this really?  I have been thinking a lot about this and have found
little in the way of problematic counterexamples.

>But this admittedly "vague sketch" of strategies is simply too vague, 
>as it relies on the concept of hints but doesn't describe them, and 
>ignores the power of supplying as input into the URN resolution system 
>ancillary information about the URN (hints in reverse, so to speak).

True.  Our omission of an explanation of hints has caused a great deal
of confusion, which this reply has hopefully addressed to a degree.
Since we did not have a clear technical plan at the time of writing we
did not want to do into a lot of detail; we hoped to instead get out
some ideas we were plaing with and see what sort of reactions were
generated.  As for the idea of `hints in reverse', the URN community
should decide if this is within the scope of our problem or whether we
want to declare it outside.  My opinion is that it is outside; it
widens the problem and makes the primary technique for resolution
(i.e. precomputed lexical generalization of URNs) impossible.

In any case, thanks a lot for the comments.  Yours are the first to
address any of the ideas in section 4.  I have been continuing to work
on those ideas, in particular the stuff relating to canonical forms.
Following this message I am forwarding a pointer to a small
explanation of how various naespaces might be canonicalized on the
client side enough to be taken apart by very simple rules.  The
canonicalization is done by a simple translation program that takes a
little ``program'' as input.  The idea is to rearrange the URN at the
beginning so that the hierarchy goes in one consistent direction; the
theory is that after the name is canonicalized the rest of the
resolution can be done with very simple ``rules''.

Essentially, where NAPTR allows arbitrary rewrite patterns, the
simplified rules allow only a few very simple moves, removing
characters from the front of the canonicalized string and copying them
or replacing them with specified strings.  This way most of the data
driving the resolution are in this simplified format, which makes it
easier to handle and easier to specify from above.  Under such a
system, a name scheme designer can enforce certain policies about the
format and semantics of the names under that scheme, whereas according
to the original NAPTR model the top-level name scheme design had no
impact on the structure underneath.

Such a modified NAPTR resolution mechanism will be able to resolve a
very specific subset of all possible namespaces, and the rest will be
escaped to other resolution mechanisms.  The main difference here (as
far as I can tell) is that using the simplified rules it becomes more
obvious what naming schemes are possible and which are not.  It is not
clear that this modification results in any significant reduction in
the universe of resolvable name schemes.  In fact, NAPTR is not well
suited to many types of namespaces.  For example, a namespace of MD5
digests of documents could not efficiently be resolved by NAPTR, nor
can it be efficiently resolved by this modified system.  One name
space that the original NAPTR plan can resolve that the modified one
can't is a namespace in which there is no pre-defined hierarchy.  For
example, generalization can be based on flags:

  URN:flags:girod#paper.ps#about-translation#about-urns

contains several #-delimited flags that could be matched (for example,
all URNs of that type containing my name go to my resolver, etc.)
However, for a number of reasons, this is not currently a useful way
of structuring URNs, and if it becomes useful it seems reasonable to
build a separate resolution mechanism.