Subject: [provreg] Lists and EPP

From: Ulrich Wisser <[email-masked]>
Date: Wed, 08 Jun 2011 09:50:49 +0200

Hi,

one of the first question I get everytime I meet registrars is

   How can I get a list of all my domains?

And yes, I believe registrars should be able to keep such a list 
themself. Even if only for business (invoice) reasons. But what are you 
going to do if your customers request a specific service? At some point 
you will have to do it anyway, so why not do it properly.

I believe a list extension would be really helpful. Due to system load a 
registry might need to make some policies about usage but it would make 
live easier for registries and registrars.

/Ulrich

---

From: Jim Reid <[email-masked]>
Date: Wed, 8 Jun 2011 10:05:20 +0100

On 8 Jun 2011, at 08:50, Ulrich Wisser wrote:

> themself. Even if only for business (invoice) reasons. But what are  
> you going to do if your customers request a specific service? At  
> some point you will have to do it anyway, so why not do it properly.

IMO that means running some sort of SQL statement over the database  
and then sending the registrar a bill for the output. This would also  
encourage registrars to maintain proper records about the domain names  
they supposedly manage. :-)

It's not clear to me an EPP hook is necessary to solve this issue. I  
wonder too about the practicalities of this sort of EPP transaction if  
GoDaddy's asking Verisign (to pick a registrar and registry not quite  
at random).

---

From: Patrik Fältström <[email-masked]>
Date: Wed, 8 Jun 2011 11:12:10 +0200

On 8 jun 2011, at 11.05, Jim Reid wrote:

> It's not clear to me an EPP hook is necessary to solve this issue. I wonder too about the practicalities of this sort of EPP transaction if GoDaddy's asking Verisign (to pick a registrar and registry not quite at random).

The main reason this might be needed in some sort of automated fashion is that some registries do have a policy that imply domains might move easier between the registrars than at other registries. I.e. it is easier than one think to end up out-of-sync.

There are a number of other sync-mechanisms I would not mind seeing, but it is important the registries do that the same way... I do see registrars start to be irritated on the high number of different implementations of The Same Thing between registries.

   Patrik

---

From: Bernie Hoeneisen <[email-masked]>
Date: Wed, 8 Jun 2011 11:16:30 +0200 (CEST)

Hi Ulrich

As I understand this functionality is out-of-scope for EPP. Such 
functionality has been addressed and standardized by the CRISP working 
group as IRIS protocol.

For reference, see RFC 3981 ff.

   http://tools.ietf.org/html/rfc3981 ff.

cheers,
  Bernie

--

http://ucom.ch/
Tech Consulting for Internet Standardization

On Wed, 8 Jun 2011, Ulrich Wisser wrote:

> And yes, I believe registrars should be able to keep such a list themself. 
> Even if only for business (invoice) reasons. But what are you going to do if 
> your customers request a specific service? At some point you will have to do 
> it anyway, so why not do it properly.
>
> registry might need to make some policies about usage but it would make live 
> easier for registries and registrars.
>
> /Ulrich
> _______________________________________________
> provreg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/provreg
>

---

From: InterNetX - Volker Janzen <[email-masked]>
Date: Wed, 08 Jun 2011 11:19:31 +0200

Hi,

Am 08.06.2011 11:12, schrieb Patrik Fältström:
> I.e. it is easier than one think to end up out-of-sync.

I agree with this. It's very nice for registrars to have a list of all
domains in the account to find domains that are out of sync quick.

If every registry offers such a list in a different way, you'll always
have the pain of writing script for fetching. Would be nice if there is
a standard way to provide these lists.

Regarding the database load caused by list generation: this could be
done by a cron job and the registrar could fetch the last list that has
been generated (marked with date, when the list has been generated).

Volker Janzen
Team Entwicklung

-- 
InterNetX GmbH
Maximilianstr. 6
93047 Regensburg
Germany

Tel: +49 941 59559-0
Fax: +49 941 59579-050

www.internetx.com
www.facebook.com/InterNetX
www.twitter.com/InterNetX

Geschäftsführer/CEO: Thomas Mörz
Amtsgericht Regensburg, HRB 7142

GPG-Key: 0x186C5F77
GPG-Fingerprint: 392E 8730 FE23 DCE8 8878 8524 5361 BCCC 186C 5F77

---

From: Klaus Malorny <[email-masked]>
Date: Wed, 08 Jun 2011 12:00:25 +0200

On 08/06/11 11:16, Bernie Hoeneisen wrote:
> Hi Ulrich
>
> As I understand this functionality is out-of-scope for EPP. Such functionality
> has been addressed and standardized by the CRISP working group as IRIS protocol.
>

Hi,

hmm, for some reasons, CRISP never really took off. And are there any 
alternative transport mechanisms besides BEEP, which does not seem to have 
widespread library support?

Besides this, an implementation with CRISP has probably the same issues as an 
EPP implementation, namely that you have potentially to deal with hundred 
thousands domains, which may eat up your resources server-side.

However, generally, a reporting mechanism could be done in an incremental way, 
so that it even could be implemented in EPP:

If one considers the list of domains being lexicographically sorted, one can 
give a starting point from which the names shall be reported. The server returns 
only the next n names according the given sort order (n ranging from a few tens 
to a few hundreds). To continue the list, one specifies the last name of the 
last response as the new starting point. While this seems to be less efficient, 
it has the benefit that the returned list is nearly up-to-date (compared to 
generated reports which may be up to 24 hours old), and one can do one's 
synchronization stuff in a continuous mode if need be. If this is too slow, the 
client can even use multiple starting points in parallel.

We have implemented such methods in different places and it works sufficiently 
enough.

regards,

Klaus

---

From: Jim Reid <[email-masked]>
Date: Wed, 8 Jun 2011 11:13:35 +0100

On 8 Jun 2011, at 10:12, Patrik Fältström wrote:

> There are a number of other sync-mechanisms I would not mind seeing,  
> but it is important the registries do that the same way... I do see  
> registrars start to be irritated on the high number of different  
> implementations of The Same Thing between registries.

Yes, an industry-wide standarised way to do this kind of thing would  
of course be good Patrik. I'm not convinced EPP is the tool to use  
though. Not every registry and registrar speaks EPP. There could be  
icky implementation issues too: eg locking the database while  
something ploughs through it to generate the list. [We can't have the  
database changing while the list is generated, eh?] I fear this could  
be the start of a slippery slope as more SQL reporting type stuff gets  
shoe-horned into EPP: "gimme a list of all the domain names I  
registered for this contact that then got transferred to that  
registrar over the last N months", "which domains I manage reference  
this nameserver (or contact) object?".

Perhaps a BCP might be a better approach than an EPP extension?

---

From: Patrik Fältström <[email-masked]>
Date: Wed, 8 Jun 2011 13:32:43 +0200

On 8 jun 2011, at 12.13, Jim Reid wrote:

> Yes, an industry-wide standarised way to do this kind of thing would of course be good Patrik. I'm not convinced EPP is the tool to use though. Not every registry and registrar speaks EPP. There could be icky implementation issues too: eg locking the database while something ploughs through it to generate the list. [We can't have the database changing while the list is generated, eh?] I fear this could be the start of a slippery slope as more SQL reporting type stuff gets shoe-horned into EPP: "gimme a list of all the domain names I registered for this contact that then got transferred to that registrar over the last N months", "which domains I manage reference this nameserver (or contact) object?".
> 

I agree with this, but the day one of the registries do implement something like this in epp, I want other registries to follow.

And, if you want to know what has been shoe horned into epp, have a look at what the various registries actually have implemented.

I.e. I do not agree this would be the beginning of a slippery slope. We are already falling down that slippery slope in very high speed as no registry is compliant with other registries epp implementation.

People have, we see now, used the "e" part of epp a bit too much.

And I dare to say that as I as Area Director when epp was invented could have pushed back more, but I did not. So I blame myself.

   Patrik

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Wed, 8 Jun 2011 07:51:09 -0400

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Patrik Fältström
> Sent: Wednesday, June 08, 2011 7:33 AM
> To: Jim Reid
> Cc: Ulrich Wisser; [email-masked]
> Subject: Re: [provreg] Lists and EPP

[snip]

> And I dare to say that as I as Area Director when epp was invented
> could have pushed back more, but I did not. So I blame myself.

Don't, Patrik.  The hard part about what we were doing is that we were trying to produce a protocol to automate a multitude of existing disparate registration processes and policies.  That's not a good starting point for being able to reach agreement on one way and one way only.  Given the operating environment I don't think we could have ever produced anything at all without extensibility.

I still have "that's policy!" nightmares... :)

Scott

---

From: Patrik Fältström <[email-masked]>
Date: Wed, 8 Jun 2011 13:55:43 +0200

On 8 jun 2011, at 13.51, Hollenbeck, Scott wrote:

> The hard part about what we were doing is that we were trying to produce a protocol to automate a multitude of existing disparate registration processes and policies.  That's not a good starting point for being able to reach agreement on one way and one way only.  Given the operating environment I don't think we could have ever produced anything at all without extensibility.
> 

Yeah, you are the hero that really never gave up!!!

What irritates me is though that there are a number of extensions that are made slightly differently, with slightly different rules on data format and of course that registries use their own extensions even though they are very similar, or identical, to what other people are doing.

So yes, we needed extensions, I just think registries could have done better if they coordinated a bit more the standardization of the extensions, instead of creating one epp extension per registry/domain name combination.

To conclude: I agree that epp is probably not the most efficient protocol to do sync:ing, but I rather see one syncing extension than "one per registry". That was my (only) point.

   Patrik

---

From: InterNetX - Marco Schrieck <[email-masked]>
Date: Wed, 08 Jun 2011 14:29:26 +0200

Hi all,

Am 08.06.11 13:55, schrieb Patrik Fältström:
> So yes, we needed extensions, I just think registries could
> have done better if they coordinated a bit more the standardization
> of the extensions, instead of creating one epp extension per
> registry/domain name combination.

I agree with that, but most process are in some way the same, but from
EPP side there are not recommendations, how to handle this. So it is
clear that every one do his own stuff. For example Ownerchanges and
Transfers with Ownerchanges.

> To conclude: I agree that epp is probably not the most efficient 
> protocol to do sync:ing, but I rather see one syncing
> extension than "one per registry". That was my (only) point.

Most registry have a ftp/http download for such list, most lists are
quite indentical. So if there was a standard for the minimum
requirements it would be nice.

eg.

list creationdate with date and time

some basic fields:

domain,created,expire,updated,autorenew,phase (status: redemtionPeriod)

Regards

Marco Schrieck
Bereichsleiter Entwicklung

- -- 
InterNetX GmbH
Maximilianstr. 6
93047 Regensburg
Germany

Tel. +49 941 59559-0
Fax +49 941 59579-050

www.internetx.com

ICQ:     232016987

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 8 Jun 2011 09:31:34 -0400

On Wed, Jun 08, 2011 at 12:00:25PM +0200, Klaus Malorny wrote:

> However, generally, a reporting mechanism could be done in an
> incremental way, so that it even could be implemented in EPP:

Why do you want to Extend the protocol so that it's no longer even
about Provisioning?

I can see a possibly useful extension that requests a report be
generated.  That is, like other things in EPP, one can request that
something be provisioned (in this case, a report).  But the response
to that should not be a long list of domains in-protocol, I think.
EPP is poorly structured for that sort of bulk operation, on purpose.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Edward Lewis <[email-masked]>
Date: Wed, 8 Jun 2011 09:32:22 -0400

At 11:13 +0100 6/8/11, Jim Reid wrote:

>Yes, an industry-wide standarised way to do this kind of thing would of
>course be good Patrik.

First let's put the cart and the horse in the right sequence.  First 
solve the problem and then worry about making the solution generally 
applicable.  Before that (zeroth?) define and scope the problem.

If "one of the first question I get every time I meet registrars is 
'How can I get a list of all my domains?'" really is something common 
across registries, this sounds like a worthwhile topic to take on. 
Solving real problems is a lot more fun than making them up. ;)

In the "it seems to me" column, efforts with the goal of finding a 
standard solution usually die on the vine.  Efforts that attack the 
problem regardless of generalization yield better fruit from which we 
can extract a more general solution.

>I'm not convinced EPP is the tool to use though. Not every registry
>and registrar speaks EPP.

That's a valid point ("the tool to use") but the issue isn't that it 
isn't ubiquitous.  No one is prevented from adopting EPP or deploying 
it as an interface, but anyone is free to not adopt it.

>There could be icky implementation issues too: eg locking the database while
>something ploughs through it to generate the list. [We can't have the database
>changing while the list is generated, eh?] I fear this could be the start of a
>slippery slope as more SQL reporting type stuff gets shoe-horned into EPP:
>"gimme a list of all the domain names I registered for this contact that then
>got transferred to that registrar over the last N months", "which domains I
>manage reference this nameserver (or contact) object?".

What's to fear isn't the slippery slope.  There's no rules for what 
any protocol can or can't be made to do.

The analysis needed (and Jim begins it) is first to identify the 
strengths and weaknesses of EPP.  Besides its functional abstract, 
how well does it do the job in certain situations and how poorly does 
it do the job in others.  Then any alterations to the protocol had 
better not weaken its strong points.  Alterations should highlight 
the strong points.

EPP is a registrar-registry protocol.  Its role is entering 
provisioning data.  The two are not the same.  How well does it 
handle high latency operations?

Why has IRIS failed to bloom?  Why did it fail where EPP succeeded? 
Maybe there are clues there pointing out how EPP might be able to do 
this.

Maybe this is a fit for EPP.  But maybe it isn't.  Just because EPP 
was built for one use case (provisioning) doesn't mean it has to 
stick to that.  The "E" stands for Extensible after all.

>Perhaps a BCP might be a better approach than an EPP extension?

Remember that code trumps spec in the IETF.

(Die on the vine, yield fruit, IRIS blooming...hmm maybe I should 
take the rest of the day off and tend to a garden.)

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis
NeuStar                    You can leave a voice message at +1-571-434-5468

Now, don't say I'm always complaining.
Wait, that's a complaint, isn't it?

---

From: Edward Lewis <[email-masked]>
Date: Wed, 8 Jun 2011 09:47:43 -0400

>People have, we see now, used the "e" part of epp a bit too much.

Can you quantify this?  (Asking out of 
curiosity.)  As in, has there been a failure in 
the protocol?  Has the goal of interoperability 
failed?

Between 2009 and 2010 I poked into EPP 2.0 and 
found that everyone was happy with EPP 1.0 and 
the plethora of extensions, especially with 
Patrick Mevzek (I believe) has implemented the 
bulk of them in one distribution.

>And I dare to say that as I as Area Director when epp was invented could
>have pushed back more, but I did not. So I blame myself.

As one of the co-chairs, I'm willing to pass the blame on. ;)

At 13:55 +0200 6/8/11, Patrik Fältström wrote:

>So yes, we needed extensions, I just think registries could have done better
>if they coordinated a bit more the standardization of the extensions, instead
>of creating one epp extension per registry/domain name combination.
>
>To conclude: I agree that epp is probably not the most efficient protocol to
>do sync:ing, but I rather see one syncing extension than "one per registry".
>That was my (only) point.

I continued in the thread and found this.

I thought the same as you expressed when I did 
some organization for a non-BoF at the Anaheim 
IETF.  Architecturally speaking, I had some 
misgivings about the path EPP had taken.  From 
March 2010 until May 2010 some efforts ramped up 
as a result of the IETF-related meeting, 
including a new mailing list started called 
regops.  In early May, in a CENTR Tech meeting 
some preliminary results were presented - mostly 
the estimate of what it would take to "fix" EPP. 
This presentation was a follow up to a discussion 
during the October 2009 CENTR Tech meeting that 
kind of kicked off this short study.

The discussion at the May 2009 CENTR Tech meeting 
came down to "okay, well, EPP 1.0 is sufficient." 
I took this to mean - in the operations world, 
EPP is just fine.  Operators are satisfied that 
there is code to do all the versions needed.  To 
an operator having two different "redemption 
grace period" extensions, one for COM and one for 
BIZ, is not a problem.  Maybe as an architect it 
sounds ugly - but then who are protocols built 
for, the architects or the operators?

After that meeting I abandoned any upgrade/update 
to EPP and learned to live with the way it is. 
Code rules, specs aren't as important.

---

From: Patrik Fältström <[email-masked]>
Date: Wed, 8 Jun 2011 16:32:05 +0200

On 8 jun 2011, at 15.47, Edward Lewis wrote:

> Can you quantify this?  (Asking out of curiosity.)  As in, has there been a failure in the protocol?  Has the goal of interoperability failed?

Yes. I claim that have failed from some point of view. The protocol is interoperable, but what you transfer inside the protocol is different.

Just look at Net::DRI in Perl and you see that every TLD have their own extension. They have their own rules for for example how a domain name is to be written (do they accept [a-z] only, or [a-zA-Z]?). Etc...

I think personally it is because all development of the protocol, and the extensions, are made by the registries, and not enough input from registrars which are the ones that today have to pay the burden of what I call non-interoperability.

Hey, some registries do not even release their specification unless you pay to become a registrar!

Yes, different business models exists for the registries, but I do not see such diversity is needed as exists today in for example the epp specs for the various TLDs.

> Between 2009 and 2010 I poked into EPP 2.0 and found that everyone was happy with EPP 1.0 and the plethora of extensions, especially with Patrick Mevzek (I believe) has implemented the bulk of them in one distribution.

Who are everyone? How many registrars did you talk with that actually have implemented epp from ground up that have connections with say 5-10 different registries where a minority are gTLDs?

> As one of the co-chairs, I'm willing to pass the blame on. ;)

Ha ha ha!

> I thought the same as you expressed when I did some organization for a non-BoF at the Anaheim IETF.  Architecturally speaking, I had some misgivings about the path EPP had taken.  From March 2010 until May 2010 some efforts ramped up as a result of the IETF-related meeting, including a new mailing list started called regops.  In early May, in a CENTR Tech meeting some preliminary results were presented - mostly the estimate of what it would take to "fix" EPP. This presentation was a follow up to a discussion during the October 2009 CENTR Tech meeting that kind of kicked off this short study.
> 
> The discussion at the May 2009 CENTR Tech meeting came down to "okay, well, EPP 1.0 is sufficient."

CENTR is biased to registries.

> I took this to mean - in the operations world, EPP is just fine.  Operators are satisfied that there is code to do all the versions needed.  To an operator having two different "redemption grace period" extensions, one for COM and one for BIZ, is not a problem.

Yes, it IS a problem.

To have one extension for every TLD, that create a nXm matrix of number of extensions times number of TLDs.

> Maybe as an architect it sounds ugly - but then who are protocols built for, the architects or the operators?
> 
> After that meeting I abandoned any upgrade/update to EPP and learned to live with the way it is. Code rules, specs aren't as important.

And I think you made the correct decision. Or you would go mad! ;-)

   Patrik

---

From: Klaus Malorny <[email-masked]>
Date: Wed, 08 Jun 2011 18:20:58 +0200

> EPP is poorly structured for that sort of bulk operation, on purpose.

Hi Andrew,

I understand the criticism and I am actually ambivalent about such an approach. 
My posting was just to point out that there is a middle course between having no 
list and a full list, just from a technical (and not philosophical) perspective.

But generally, where to draw the line regarding the functionality? Why is the 
retrieval of the contents of the the registrar's repository at the registry not 
part of the provisioning? With the same argument you could argue that the 
domain:info command is superfluous in most of the cases -- most of the data you 
should have in your own database. But likely nobody would consider to remove 
this command from the protocol.

I think that a registrar-registry protocol should cover the whole technical 
interaction between the two parties. Any kind of communication besides this 
protocol, including downloading reports from FTP servers, reduces the value of 
the protocol.

On the other hand, this requirement collides with a general demand, the 
frugality of a protocol -- once I read the nice sentence "don't reason about 
what you can add to a protocol, but what you can remove from it" (or 
correspondingly). The balance needs to be found.

Regards,

Klaus

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 8 Jun 2011 09:49:09 -0700

I really think that there should be an extension to define such functionality, but it should be a policy issue whether it is enabled or not.

It's exactly the same case as the domain:check command. there's currently no limit on how many domain names can be queried (well technically 2^32 - 4 bytes long with the EPP over TCP transport) with a single XML document.

Some registries limit the number of domain:checks a registrar can perform within a certain period of time, and that's the way the deal with this problem.

Others limit the total size of the XML payload, etc. 

We can probably work around the issues that have been surfaced in this list, specially if there is consensus that this is needed. The technical barriers, can be worked out.

On Jun 8, 2011, at 9:20 AM, Klaus Malorny wrote:

> I understand the criticism and I am actually ambivalent about such an approach. My posting was just to point out that there is a middle course between having no list and a full list, just from a technical (and not philosophical) perspective.
> 
> But generally, where to draw the line regarding the functionality? Why is the retrieval of the contents of the the registrar's repository at the registry not part of the provisioning? With the same argument you could argue that the domain:info command is superfluous in most of the cases -- most of the data you should have in your own database. But likely nobody would consider to remove this command from the protocol.
> 
> I think that a registrar-registry protocol should cover the whole technical interaction between the two parties. Any kind of communication besides this protocol, including downloading reports from FTP servers, reduces the value of the protocol.
> 
> On the other hand, this requirement collides with a general demand, the frugality of a protocol -- once I read the nice sentence "don't reason about what you can add to a protocol, but what you can remove from it" (or correspondingly). The balance needs to be found.
> 
> Regards,
> 
> Klaus

Francisco Obispo 
Hosted@ Programme Manager
email: [email-masked]
Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
Key fingerprint = 532F 84EB 06B4 3806 D5FA  09C6 463E 614E B38D B1BE

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 8 Jun 2011 13:02:09 -0400

On Wed, Jun 08, 2011 at 06:20:58PM +0200, Klaus Malorny wrote:

> But generally, where to draw the line regarding the functionality?
> Why is the retrieval of the contents of the the registrar's
> repository at the registry not part of the provisioning? With the
> same argument you could argue that the domain:info command is
> superfluous in most of the cases -- most of the data you should have
> in your own database. But likely nobody would consider to remove

Right.  My real objection is that the protocol is designed to be
"retail" rather than "wholesale".  That is, most of the protocol
(actually, I can't think of an exception) is intended to deal with one
object at a time.  A report of the proposed kind would return not one
object, but instead a set of other objects.  

This is also why I think it would be fine to define a report object to
be provisioned: you make the provisioning request, and get back your
answer (I would imagine it would be a URI with the location the report
will be, and the validity period -- perhaps "available by" and
"expires on", both timestamps).  This maintains the model of working
on one object at a time.  I suppose the return would also include a
report identifier, so that you could use the poll queue to find out
when your report was ready, if you wanted.

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Gould, James" <[email-masked]>
Date: Wed, 8 Jun 2011 17:17:50 +0000

Like Andrew I view EPP as a provisioning protocol for one or a small set of objects.  Reports already serve the purpose of providing a full inventory of Registrar objects and their relationships.  Although EPP could be used to support the delivery of reports I don't believe that it was designed for it and I don't believe that it's appropriate to do so from the provisioning system.  The EPP provisioning systems are setup for provisioning activity where generating a response of thousands to millions of records is out of scope.  The systems for the generation and availability of reports are designed for bulk data like what has been discussed.  Let's not try to fit the round peg into the EPP square hole.  

________________________________________
From: [email-masked] [[email-masked]] on behalf of Andrew Sullivan [[email-masked]]
Sent: Wednesday, June 08, 2011 1:02 PM
To: [email-masked]
Subject: Re: [provreg] Lists and EPP

A

---

From: "Gould, James" <[email-masked]>
Date: Wed, 8 Jun 2011 17:29:22 +0000

I believe that EPP is successful due to it's extensibility.  There is little to no way to define a single way for a registry to function which naturally leads to custom extensions.  Without the ability to extend the protocol other methods would be used that would be even worse that includes customizing the base protocol itself or utilizing another protocol or method to accomplish what is desired.  Ideally custom extensions would be opt-in and backward compatible to reduce the impact on clients that don't need or want to support them.  As proposed at the IETF meeting in Anaheim it would be useful to form a working group or a forum for the registries and the registrars to discuss the existing extensions and planned extensions to identify overlap and to identify those extensions that should go through the standards process.  The standards process is long and is a lot of effort, so this would not be warranted for every extension but certainty for some. 
________________________________________
From: [email-masked] [[email-masked]] on behalf of Patrik Fältström [[email-masked]]
Sent: Wednesday, June 08, 2011 10:32 AM
To: Edward Lewis
Cc: [email-masked]
Subject: Re: [provreg] Too much 'E'? was Re:  Lists and EPP

Ha ha ha!

Yes, it IS a problem.

   Patrik

---

From: Patrick Mevzek <[email-masked]>
Date: Wed, 8 Jun 2011 20:14:30 +0200

Ulrich Wisser <[email-masked]> 2011-06-08 10:17
>   How can I get a list of all my domains?
 
[..]

> I believe a list extension would be really helpful. Due to system
> load a registry might need to make some policies about usage but it
> would make live easier for registries and registrars.

Having been in touch with various registrars dealing with multiple
registries, I can see that as a pattern too, so it seems to me
worthwhile to pursue.

I think we can argue a lot about if this matches EPP goals or not,
however among registries one of the only thing really standardized
today is EPP. Whois is not, DAS is not, access to recently released domain
names is not, registrars reports access is not, etc...

As much as it saddens me, IRIS seems to be dead or close to it,
specifically now with the creation of the IETF WEIRDS non working
group, and having ICANN and ARIN pushing a lot towards REST and
against IRIS (for reasons that I do not believe to be technically
sound).

So investing work to see how this can be done over IRIS would be
probably a waste of time or at least registries/registrars should
start using IRIS for D-CHECK and then extend it.

This leaves us with EPP, as we know today with many extensions doing
the same thing, probably due to the NIH syndrom, and as others have
said because there is little cooperation from registrars.

EPP could be seen as a viable solution. Otherwise everyone is doing
the same thing in a different way.

And I think there are ways to accomodate SLAs and constraints like that.

First, such reports do not need to be realtime. Most often, their
content corresponds to the previous day state or even before. So
registries could implement that using another DB to offload work
and guarantee consistency.

I could see such exchange :

- registrars ask for some report: here we would need to define what
  kind of reports, what kind of filters (if any) & constraints, etc.

- the registry could reply back immediately with an 1001 return code,
so that the report generation would be a background task of low
priority that would be done when possible.
when the generation is complete, the registry would add a notification
message in the registrar queue.
It can of course include just an URI as other have suggested, but I'm
not sure it is the best way.
The report results could be chunked (in slices to be determined by registry
policy) and available as registry messages to be fetched by the
registrar.

If registries/registrars fear the mix of registry notifications
messages related to day to day business and report retrieval,
registries could have a separate EPP server just for reports (or a
separate set of registrars credentials just for that), so that at
least registrars could use the same code, just change the connecting
parameters.

But in both cases we would need to precisely define the 
content/format of the report, and again maybe trying to accomodate as
many cases as possible as seen today and/or defining a way to easily 
extend things without breaking other implementations... as it has been
done in EPP.

-- 
Patrick Mevzek

---

From: Christopher Browne <[email-masked]>
Date: Wed, 8 Jun 2011 13:33:12 -0400

2011/6/8 Jim Reid <[email-masked]>:
>> There are a number of other sync-mechanisms I would not mind seeing, but
>> it is important the registries do that the same way... I do see registrars
>> start to be irritated on the high number of different implementations of The
>> Same Thing between registries.
>
> course be good Patrik. I'm not convinced EPP is the tool to use though. Not
> every registry and registrar speaks EPP. There could be icky implementation
> issues too: eg locking the database while something ploughs through it to
> generate the list. [We can't have the database changing while the list is
> generated, eh?] I fear this could be the start of a slippery slope as more
> SQL reporting type stuff gets shoe-horned into EPP: "gimme a list of all the
> domain names I registered for this contact that then got transferred to that
> registrar over the last N months", "which domains I manage reference this
> nameserver (or contact) object?".

When I think about this, my brain heads down quite a similar trail, as
one part of me starts looking for the semantic minutiae of how it
ought to work, and the successive set of problems that lead to
defining a more and more extensive set of commands to characterize the
necessary actions, and to cope with various concurrency and
transaction management problems.  Franciso, downthread, observes that
limits may be indicated in a multiplicity of ways (e.g. - counts,
maximum sizes of requests).

I quickly leap to "wait, stop here!" since adding in such details
winds up mandating quite a lot about implementation, which is
something that the protocol ought to generally protect us from, rather
than fairly directly implying.

I'd be fearful that a "<domain:listall>" command would turn out to be
so complex in its behavior that it's effectively unusable.  CRISP/IRIS
was intended to provide this sort of functionality, but, in practice,
there has been little adoption, which seems suggestive that something
is problematic about it as solution.

None of the directions that are evident seem palatable:
- The "straightforward" way is undesirable as it leads to unacceptably
expensive requests (e.g. - where the response set will be unacceptably
large)
- Performance-sensitive methods will be complex, and need to specify
semantics (e.g. - assumptions about transaction handling,
serialization, and such) that are not desirable to have in the
protocol.

In practice, registry contracts have tended to define a set of
mandatory reports (including formats) for the regular release of this
sort of information, and while that's not "one stop shopping" (or pick
your favorite business-y idiom), it generally does function.

Andrew Sullivan's suggestion of passing URIs around using the poll
queue facility is plausible, certainly involving a goodly amount of
indirection.  It implies an interaction between EPP registry and an
outside URI server (quite likely a web/ftp server).
-- 
Christopher Browne