Subject: [ietf-provreg] a new core command...?

From: Eugenio Pinto <[email-masked]>
Date: Tue, 06 Jun 2006 12:02:36 +0100

Hi all,

I don't know if this subject was already discussed in the past, in the 
provreg-wg, but I need some opinions about it.

As a Provisioning Protocol I'm wondering if it shouldn't be there 
another query command besides <check>, <info> and <transfer> that would 
allow us to retrieve all the objects actually provisioned by a repository.

In DNS.PT we needed a command with such functionality in order to 
retrieve all the domains owned/managed by the current logged user and we 
needed to do an extension.

My question is:

Is it reasonable to include such a command in the core protocol?

We have two nearly candidates for that goal:

1 - The EPP <check> command which is used to determine if an object can 
be provisioned within a repository.

2 - The EPP <info> command which is used to retrieve information associated with an existing object.

The problem is that the commands above are designed to check well 
identified objects each time. If we are not looking for a specific 
object we get nothing...

As Andrew Sullivan and Frederico Neves already did, I'm working on an 
I-D detailing some of the DNS.PT experiences with implementation. This 
extension will be there.

Once I've completed this, I'll be sure to forward it here.

Waiting for feed-back,

Eugenio Pinto
FCCN - DNS.PT

---

From: James Gould <[email-masked]>
Date: Tue, 06 Jun 2006 08:24:24 -0400

Eugenio,

We have not had a need to support a bulk form of query in EPP.  This had
come up several times, but we kept the EPP protocol strictly a provisioning
protocol and left the bulk information up to reports.  I know that a bulk
query would be a real problem for a Registry like COM/NET.

JG 

-------------------------------------------------------
James F. Gould
Senior Software Engineer
VeriSign Information Services
[email-masked]
Direct: 703.948.3271
Mobile: 703.628.7063

21345 Ridgetop Circle
LS2-2-1
Dulles, VA 20166

Notice to Recipient:  This e-mail contains confidential, proprietary and/or
Registry  Sensitive information intended solely for the recipient and, thus
may not be  retransmitted, reproduced or disclosed without the prior written
consent of  VeriSign Naming and Directory Services.  If you have received
this e-mail message in error, please notify the sender immediately by
telephone or reply e-mail and destroy the original message without making a
copy.  Thank you.

> From: Eugenio Pinto <[email-masked]>
> Date: Tue, 06 Jun 2006 12:02:36 +0100
> To: <[email-masked]>
> Subject: [ietf-provreg] a new core command...?
> 
> Hi all,
>

---

From: "Michael Young" <[email-masked]>
Date: Tue, 6 Jun 2006 08:45:56 -0400

Eugenio I also have to support James. This doesn't seem scalable for the
core protocol, an extension seems the most appropriate place for it.

I could see how in an extension you could address a lot of concerns, i.e.
queueing a large request, applying policy rules to frequency of these types
of requests, etc. 

I am looking forward to your I-D.

Best Regards,

Michael Young 
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of James Gould
Sent: Tuesday, June 06, 2006 8:24 AM
To: Eugenio Pinto; [email-masked]
Subject: Re: [ietf-provreg] a new core command...?

Eugenio,

JG 

21345 Ridgetop Circle
LS2-2-1
Dulles, VA 20166

> another query command besides <check>, <info> and <transfer> that 
> would allow us to retrieve all the objects actually provisioned by a
repository.
> retrieve all the domains owned/managed by the current logged user and 
> we needed to do an extension.
> 
> My question is:
> 
> 1 - The EPP <check> command which is used to determine if an object 
> can be provisioned within a repository.
>

---

From: Francisco Obispo <[email-masked]>
Date: Tue, 06 Jun 2006 10:37:52 -0400

Dear Eugenio,

For our registration system, whatever was not accesible via de EPP 
protocol, was implemented
using WebServices (SOAP).

But, it would probably be helpful to have a list of objects given a 
search pattern :-/

regards

-francisco

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 6 Jun 2006 10:46:29 -0400

[re-sending, as this appears to have gotten caught on a misbehaving
system.  sorry for any dupes.]

> allow us to retrieve all the objects actually provisioned by a repository.

I believe the iris-dreg2 CRISP approach provides what you want.  This
seems to be a matter of querying, rather than of provisioning.  You can
find that at http://tools.ietf.org/wg/crisp/draft-ietf-crisp-iris-dreg2

A
-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Afilias Canada                        Toronto, Ontario Canada
<[email-masked]>                              M2P 2A8
                                        +1 416 646 3304 x4110

---

From: Eugenio Pinto <[email-masked]>
Date: Tue, 06 Jun 2006 15:52:29 +0100

Ok.

It's not an extension but an entire external service. Thanks for you 
feed-back.

Just a question: Are you using that SOAP service for anything else?

There is anyone else who would like to use (or is actually using, by 
means of extensions or external services) an EPP command for searching 
objects stored in a shared central repository?

Eugenio Pinto
FCCN - DNS.PT

Francisco Obispo escreveu:
>> Michael Young -----Original Message-----
>>> I don't know if this subject was already discussed in the past, in 
>>> the provreg-wg, but I need some opinions about it.
>>>
>>> retrieve all the domains owned/managed by the current logged user 
>>> and we needed to do an extension.
>>>
>>> My question is:
>>>
>>> As Andrew Sullivan and Frederico Neves already did, I'm working on 
>>> an I-D detailing some of the DNS.PT experiences with implementation. 
>>> This extension will be there.
>>>

---

From: Eugenio Pinto <[email-masked]>
Date: Tue, 06 Jun 2006 16:02:50 +0100

Thank you

I'll check it.

Eugenio

---

From: Eugenio Pinto <[email-masked]>
Date: Tue, 06 Jun 2006 15:04:37 +0100

Dear James, Michael, Scott

Thanks for the comments.

I understand EPP wasn't designed to be a search protocol because the 
idea was rejected since there was no space in a very big Registry like 
COM/NET to such a funtionality.

We all know that EPP isn't anymore just a communication protocol used in 
global and big Registries.
ccTLD's are beeing interested on it and some premisses that let EPP core 
developers to some decisions may now not be all so true...

Just for the records:

EPP <renew> Command

   Renewal semantics do not apply to host objects, so there is no
   mapping defined for the EPP <renew> command.

EPP <renew> Command

   Renewal semantics do not apply to contact objects, so there is no
   mapping defined for the EPP <renew> command.

So, nowadays we have a <renew> Transform Command used just for a type of 
objects that is the "domain name mapping" because of a specific object 
implementation that is the "domain name mapping"

I'm wondering why wasn't it defined an extension in the "domain name 
mapping" instead of the introduction of the <renew> command in the core 
protocol.

Let's see:

Suppose the development decision was the extension for the domain renew 
command, we would be using a bigger message for the same functionality! 
We would have a lost in performance.
That's the kind of thing I want to prevent with the addition of the new 
command.

Let's compare the <renew> command with a possible <search> command.

1 - The renew command is actually used only for the domain name mapping
2 - A <search> command can be applied to every future object mapping

I know. We can't apply it when we are in a very large Registry 
environment. But, who knows? Maybe if... Are you seeing what I mean?

There are a few commands that DNS.PT will not implement, like the 
<delete> command (policy aspects).

And it's not because we don't need it that the command was not defined.

I think that in a global provisioning protocol, used in a different 
variety of environments  a <search> command is much more useful then a 
<renew> command. But it's just my humble opinion...

Is anyone "out there" that would like to use (or is actually using, by 
means of extensions) an EPP command for searching objects stored in a 
shared central repository?

That's the question...

(Please, reply to this email if you do)

Eugenio Pinto
FCCN - DNS.PT

---

From: Klaus Malorny <[email-masked]>
Date: Tue, 06 Jun 2006 17:57:17 +0200

> (Please, reply to this email if you do)

Hi Eugenio,

the fact is that while EPP has some good ideas and tries to keep things 
symmetrical and simple, it also has some nasty flaws, like the fixed command 
set, the fixed set of status values in the contact, host and domain mappings, 
the whole error reporting mechanism (fixed set of error codes, the error value 
mechanism which is flawed and mostly not understood and correctly supported by 
the registries) and so on.

In my humble opinion, EPP is too good that there would be an urgent need for a 
replacement, but at the same time also too bad to base any further protocol 
development upon it. Extensions as described in the RFCs are the way to go, but 
keep them simple and use them conservatively in order to do the registrars a 
favour. Don't make the mistakes like those that have been made with the 
so-called EPP implementation for the .eu domains, which is more or less an 
implementor's nightmare because of the large deviations from the established 
procedures.

Regards,

Klaus

___________________________________________________________________________
      |       |
      | knipp |                   Knipp  Medien und Kommunikation GmbH
       -------                           Technologiepark
                                         Martin-Schmeißer-Weg 9
      Dipl. Inf. Klaus Malorny           44227 Dortmund
      [email-masked]             Tel. +49 231 9703 0

---

From: Francisco Obispo <[email-masked]>
Date: Tue, 06 Jun 2006 12:52:52 -0400

So far,

The list of domains managed by a certain email address, and
the list of hosts asociated to a domain name.

for example, the domain, test.com.ve  has dns1.test.com.ve y 
dns2.test.com.ve

regards

Eugenio Pinto wrote:
>>>> an I-D detailing some of the DNS.PT experiences with 
>>>> implementation. This extension will be there.
>>>>

---

From: Eugenio Pinto <[email-masked]>
Date: Wed, 07 Jun 2006 14:52:33 +0100

>Andrew Sullivan wrote:

Hi all,

I've already checked this out and I have some doubts I would like you to answer.

If IRIS is specifically designed for querying and if a bulk query would be a real problem for big Registries (like COM/NET) are big Registries implementing IRIS on server side?

Is anyone providing EPP and IRIS implementations to your Registrars/Resellers?

I've seen some interesting implementation extensions to EPP and some aspects of IRIS fits better in the whole variety of Registries then EPP definitions. Wouldn't it be better to provide a provisioning and querying protocol all-in-one to our Registrars, instead of two different API's with different schemas?...

I think so. If you don't, please explain me why.

Best Regards,

Eugenio Pinto
FCCN - DNS.PT

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 7 Jun 2006 10:18:38 -0400

On Wed, Jun 07, 2006 at 02:52:33PM +0100, Eugenio Pinto wrote:
> If IRIS is specifically designed for querying and if a bulk query would be 
> a real problem for big Registries (like COM/NET) are big Registries 
> implementing IRIS on server side?

I'm not sure what you mean "on server side". 

> Is anyone providing EPP and IRIS implementations to your 

We are not at the moment.  I know there have been internal
discussions about it, but that's about all I can say on the topic
just now.

> I've seen some interesting implementation extensions to EPP and some 
> aspects of IRIS fits better in the whole variety of Registries then EPP 
> definitions. 

The dreg2 stuff was explicitly aimed at supporting EPP-based systems
as well as other systems.  The author was very responsive to some
suggestions I sent; as nearly as I can tell, the approach provides an
extremely good query framework.

> Wouldn't it be better to provide a provisioning and querying 
> protocol all-in-one to our Registrars, instead of two different API's with 
> different schemas?...

I don't see the reason for this.  You have to be able to provide a
query interface anyway to non-registrar users, so you need such a
thing.  IRIS is intended to provide that.  It seems more likely that
one will provide a consistent experience for everyone by providing
bulk query access via the same interface to everyone.

A

---

From: Andrew Newton <[email-masked]>
Date: Wed, 07 Jun 2006 10:51:36 -0400

Eugenio Pinto wrote:
> definitions. Wouldn't it be better to provide a provisioning and 
> querying protocol all-in-one to our Registrars, instead of two different 
> API's with different schemas?...

Eugenio,

IRIS is meant to be a registry/registrar interface to the general public 
where as EPP is meant to be an interface between registries and 
registrars.  So they are intended to serve two different purposes, but 
there is nothing that precludes using IRIS as a query interface between 
registries and registrars.

As for doing bulk queries over EPP, I believe there are two very good 
reasons for wanting that separated out from normal EPP flows:

1) The underlying framing of EPP messages would require larger buffers 
on the server side.  IRIS's use of BEEP and/or XPC makes this less of a 
problem with trade-off of being less simple.

2) Given the number of transactions flowing over EPP in realtime, 
separating bulk queries into another interface makes managing time 
sensitive transactions easier.  Let the bulk queries hit another set of 
servers and even another back-end datastore.

You could probably get around #1, but #2 is a pretty good reason why 
some registries would not like to do this.

-andy

---

From: James Gould <[email-masked]>
Date: Wed, 07 Jun 2006 11:24:38 -0400

What I meant from a server side is that with defined OLTP response time
SLA's, it would be a problem handling search queries using the OLTP servers
and database.  For a Registry the size of COM/NET, returning every domain
sponsored by a specific Registrar via the provisioning API would be
extremely expensive and could impact the response times of existing
commands.  We currently provide reports for this type of information.

JG 

21345 Ridgetop Circle
LS2-2-1
Dulles, VA 20166

> From: Andrew Sullivan <[email-masked]>
> Reply-To: Andrew Sullivan <[email-masked]>
> Date: Wed, 7 Jun 2006 10:18:38 -0400

---

From: Eugenio Pinto <[email-masked]>
Date: Wed, 07 Jun 2006 16:54:50 +0100

A. Sullivan,

When you say "(...)You have to be able to provide a query interface 
anyway to non-registrar users(...)" are you talking about WHOIS?

A. Newton,

I understood your point.

So, if a Registrar wants to do a transform operation to a sort of 
domains, he needs first to query the Registry using IRIS and then, using 
the information given, apply the command to all domains (one by one) 
using EPP. That's it?

James,

The reports you provide are one per month/week/day?

If a Registrar wants to access the last-minute created domains are they 
able to do that?

Do you provide IRIS for your Registrars?

Eugenio

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 7 Jun 2006 12:16:10 -0400

> anyway to non-registrar users(...)" are you talking about WHOIS?

Sure.  But whois has a couple problems, not least of which is that,
by specification, it's ASCII only.  Moreover, there's no
specification of the data structure.  In the broadest strokes,
ameliorating that situation is (part of) what I understand IRIS to be
doing.

> the information given, apply the command to all domains (one by one) 

I think everyone has assumed, historically, that the sponsor of a
repository object in EPP would also keep their own data about their
sponsored objects.  They ought to know about those object already,
no?  In any case, since all the commands are per-object in EPP, I'm
having a hard time understanding the motivation for a bulk query. 
I'm likely just dim, though (other people have also observed that
about me!), so if you have a use case in mind, I'd be interested to
hear it.

A

---

From: Eugenio Pinto <[email-masked]>
Date: Wed, 07 Jun 2006 18:12:54 +0100

Andrew Sullivan escreveu:
>  In any case, since all the commands are per-object in EPP, I'm
> having a hard time understanding the motivation for a bulk query. So if you have a use case in mind, I'd be interested to
> hear it.
>
> A
Andrew,

I could tell you some different use cases where there are needed bulk 
queries. But, unfortunately for my arguments, they are applied only in 
the transition from non-EPP environment to EPP...

So, I think you're wright. We don't need bulk queries unless our 
Registrars don't have a coherent information system. But, that's their 
problem.

---------------

Another thing that confuses me is why there isn't a <domain:roid> 
element at the <domain:info> command?

A <domain:roid> element contains the Repository Unique Object IDentifier 
assigned to the domain object when the object was created. It makes all 
sense that we can retrieve the information we need from the <roid>.

I've noticed that DNS.PT is not the only Registry that has different 
domain objects with the same name. Even if in EPP everyone has assumed 
only ACTIVE domains were to be provisioned and that the name is a unique 
identifier on the set of ACTIVE domains, it's a very limiting approach 
using it as the identifier for the object. Just imagine two different 
objects with the same name on pendingCreate status. We almost don't know 
how to query such a domain yet.

I know that .br defined an extension for just the same problem. We were 
almost to do it too. But that's the kind of thing that must be changed 
in the core specification. Am I wrong?

Best Regards,

Eugenio

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Thu, 8 Jun 2006 10:10:36 +0200

On Wed, Jun 07, 2006 at 12:16:10PM -0400,
 Andrew Sullivan <[email-masked]> wrote 

> about me!), so if you have a use case in mind, I'd be interested to

I can mention a real use-case that we had recently: a registrar of
".fr" simply wanted to check that the two databases agreed, so they
sent whois requests for every domain they manage. And asked us if
something was missing from their list.

They told us that it was part of a new quality policy, SOX, corporate
governance and so on.

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Thu, 8 Jun 2006 10:17:03 +0200

On Wed, Jun 07, 2006 at 06:12:54PM +0100,
 Eugenio Pinto <[email-masked]> wrote 
 a message of 42 lines which said:

> We don't need bulk queries unless our Registrars don't have a
> coherent information system. But, that's their problem.

Do not underestimate the need for cross-checking, for instance after a
big reorganisation in the registrar or a merge with another registrar
or the acquisisition of a registrar with a poorly-managed base,
etc. In theory, you are right, in practice, you may be too strict.

---

From: Eugenio Pinto <[email-masked]>
Date: Thu, 08 Jun 2006 10:28:08 +0100

> etc. In theory, you are right, in practice, you may be too strict.

Stephane,

When I said "we don't need bulk queries" I was talking about EPP.

For all the cases you mentioned we can use reports, as James said,  
using IRIS or another external process.

Regards,
Eugenio

---

From: Werner Staub <[email-masked]>
Date: Thu, 08 Jun 2006 11:34:58 +0200

Hi Andrew,

I can only echo Stephane and Eugenio in this respect. Actually,
the problem is much bigger than it appears. And it is compounded in
case of longer value-added chains.

EPP is strong in terms of executing provisioning requests, but weak
in terms of managing the result. In the early days, it was justifiable
to neglect the management bit, but now, after 6 years, we really should
do something about it.

Regards,

Werner

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 8 Jun 2006 07:00:51 -0400

> to neglect the management bit, but now, after 6 years, we really should

Well, if someone wants to put together a draft outlining such
changes, I promise to read it (and comment, if I have anything to
say).  It seems to me that this is duplicating functionality already
available elsewhere, though.  I still don't see how a second
protocol, well designed for large result sets, is the wrong way to do
this.  Think UNIX: small tools that do one thing well, chained
together.  Indeed, that's one of the things I like about the desigh
of EPP -- the mappings are more or less independent of one another,
and the core protocol only defines what is needed for mappings.

A

---

From: Werner Staub <[email-masked]>
Date: Thu, 08 Jun 2006 17:53:29 +0200

Hi Andrew,

> and the core protocol only defines what is needed for mappings.

You are certainly right from a technical point of view. But functionality
is one thing, standardization is another. By analogy, inches, yards,
zoll, pulgadas, Ellen, coudées, Faden etc. do the same job as meters.
But their diversity and lack of coherence makes them problematic at best.

This is also the answer to whether a new protocol or an extension of the
existing protocol should be used. EPP is just a name. Techies can
live with many acronyms. Non-techies will only learn one. And the
non-techies are the ones who will make the final decisions. To them,
EPP means "*the* solution for domains". So most probably, we we should
do exactly what you say, choose a collection of small tools, but
specify how exactly they should be used in this context, and refer to
the whole as EPP.

Regards,

Werner