Subject: [provreg] Domain check in draft-obispo-epp-idn-00.txt

From: MICHAEL YOUNG <[email-masked]>
Date: Sun, 01 Jan 2012 17:49:41 +0530

BTW on language versus script I agree fully, it should be language not
script,

On another note:

Francisco I am looking again at the old IDN I-D
http://tools.ietf.org/id/draft-sienkiewicz-epp-idn-00.txt and note that
your draft excludes the check command.

An operational nit:

So the reason it was included in the sienkiewicz draft is to reduce
operational overhead.  If the registrar is unsure whether a character is
included in the relevant language table (or a combination of characters)
they can perform a domain check with the questionable punycode. If its not
included in the implementation's language table or the combination is not
permitted, the domaincheck comes back as not available with a specified
reason.

Note that the term script is used interchangeably with the concept of
language.  It was a poor choice at the time to not clear make the
distinction but it was intended in implementation that the idn:script
element value was really a "language tag". It probably should read
something like idn:langtag instead.  These examples are all from the .Info
implementation for german "de"

C:    </create>
   C:    <clTRID>CLI-1097602657409</clTRID>
   C:    <extension>
   C:      <idn:create xmlns:idn='urn:iana:xml:ns:idn'
   C:                  xsi:schemaLocation='urn:iana:xml:ns:idn idn.xsd'>
   C:        <idn:script>de</idn:script>
   C:      </idn:create>
   C:    </extension>
   C:  </command>
   C:</epp>

When a <check> request contains an IDN whose translated punycode
   <domain:name> value contains character(s) that are not in the script
   table specified in the <idn:script> element value, the <domain:cd>
   element MUST indicate that the IDN is NOT available.

   Example <check> response when <domain:name> value has conflict with
   the value of <idn:script> element:

   S:<epp xmlns='urn:ietf:params:xml:ns:epp-1.0'
   S:     xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
   S:     xsi:schemaLocation='urn:ietf:params:xml:ns:epp-1.0
   S:     epp-1.0.xsd'>
   S:  <response>
   S:    <result code='1000'>
   S:      <msg lang='en-US'>Command completed successfully</msg>
   S:    </result>
   S:    <resData>
   S:      <domain:chkData
   S:              xmlns:domain='urn:ietf:params:xml:ns:domain-1.0'
   S:              xsi:schemaLocation='urn:ietf:params:xml:ns:domain-1.0
   S:              domain-1.0.xsd'>
   S:        <domain:cd>
   S:          <domain:name avail='0'>xn--dn-mja.info</domain:name>
   S:          <domain:reason>Character from an invalid script
   S:          </domain:reason>
   S:        </domain:cd>
   S:      </domain:chkData>
   S:    </resData>
   S:    <trID>
   S:      <clTRID>CLI-1065207438144</clTRID>
   S:      <svTRID>SRO-1097599157873</svTRID>
   S:    </trID>
   S:  </response>
   S:</epp>

This saves having to fail a domain create discover the same thing, a more
expensive transaction for the registry. Now of course with the excessive
add grace policy with GTLDs, you also probably don't want to generate
creates experimentally, even if you delete them in the five day add grace
period - you might end up getting charged.

Best Regards,

Michael Young
M:647-289-1220

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 04 Jan 2012 18:46:04 +0000

> permitted, the domaincheck comes back as not available with a specified

No, this simply shoves the responsibility onto the registrar. That means the
registrar has two choices: either put in a speed bump to ask the customer
the language (bingo! potential lost sale) or use some heuristic to guess the
language at checking time using codepoint/language whitelists.[1]

Moreover, it complicates slows down the implementation of checking for the
registrar. Say a customer decides they want to register a bunch of French-,
German-, Italian-, and Spanish-language domains. This isn't unlikely: they
might be a pan-European body who wants to register the local variants of
their name. If the registrar is forced to include the a language code when
checking, that means they have to submit separate check commands for each
language. That slows things down a lot unless the registry supports
pipelining, which is far from likely. This is the more important issue.

So, from a registrar's perspective, requiring a language tag to be specified
when checking simply makes the life of the customer and registrar more
difficult just so as the registry can save a few cycles.

> expensive transaction for the registry.

It doesn't, and as I outlined above, it's feasible for the registrar, when
they're doing the domain availability check, to make sure that the domain
matches at least one of the codepoint tables for a language supported by the
registrar. This way they can avoid <domain:check> transactions for domains
that would fail anyway (and thus avoid a failing <domain:create> transaction
later).

There's a big advantage for the registrar to do this: it allows the registrar
to narrow down the list of languages that the IDN might be for, and thus
impose less of a UI burden on the customer, especially if the IDN matches one
and only one table, in which case the customer doesn't need to be asked what
language the IDN is in at all.

Affordances like these are important to making the process of selling domains
as smooth as possible.

So, while I'm fine with asking a language code in a <domain:create> request,
asking it as part of a <domain:check> request makes no sense from a
registrar's or customer's point of view.

K.

[1] In fact, this is what we do ourselves in our own storefront for .biz and
    .tel IDNs to avoid having to ask the customer for a language code when
    all they're doing is seeing if the domain is available in the first place.

-- 
Keith Gaughan, Senior Developer
PGP/GPG key ID: 3E896381
Blacknight Internet Solutions Ltd. <http://blacknight.com/>
12A Barrowside Business Park, Carlow, Ireland
Registered in Ireland, Company No.: 370845

---

From: Patrik Fältström <[email-masked]>
Date: Wed, 4 Jan 2012 20:18:13 +0100

On 4 jan 2012, at 19:46, Keith Gaughan wrote:

> No, this simply shoves the responsibility onto the registrar.

I agree.

Checking MUST be done at the registry.

Registrar push whatever domain name it believes might be possible to register. Either that succeeds or the registry is giving a result back that can be presented to the user.

So the registrar do broad filtering and helps the registrant, but then registry is doing the work.

   Patrik

---

From: Michael Young <[email-masked]>
Date: Wed, 4 Jan 2012 15:37:43 -0500

What can I say, it would be great if the registrar put that kind of intelligence in their storefront and didn't send domain creates that violate a given language table.

Most registrars aren't making that extensive of an effort. Why? What if the registrar messes up on their filter and denies the registrant a registration the registry actually allows. Ooops starts to sound like a lawsuit, better the registrar makes the registry say no. It's cheaper to say no in domain check than a create. However I think there's room here to satisfy both concerns.

Adjust the domain check ext so the language tag element is optional. No language tag and it assumes a straight ASCII registration.

Then those registrars that want to validate their own work can and registrars that aren't sure can test a punycode string against the extended domain check. This seems like a compromise that works, thoughts?

-M

On 2012-01-04, at 2:18 PM, Patrik Fältström <[email-masked]> wrote:

> _______________________________________________
> provreg mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/provreg

---

From: Francisco Obispo <[email-masked]>
Date: Wed, 4 Jan 2012 12:44:33 -0800

I don't see a problem with using the extension during the check command, it could be used as Michael stated, in a complimentary way, and as stated, its use could be totally optional from the registrar side.

I agree that registrars should be performing those tests/validations ahead of time and shouldn't rely exclusively on the registry for those types of validation, because that would increase registry load significantly.

Regards,
Francisco

>> So the registrar do broad filtering and helps the registrant, but then registry is doing the work.

Francisco Obispo 
email: [email-masked]
Phone: +1 650 423 1374 || INOC-DBA *3557* NOC
Key fingerprint = 532F 84EB 06B4 3806 D5FA  09C6 463E 614E B38D B1BE

---

From: Patrik Fältström <[email-masked]>
Date: Wed, 4 Jan 2012 21:55:23 +0100

> What can I say, it would be great if the registrar put that kind of intelligence in their storefront and didn't send domain creates that violate a given language table.

Regardless of whether they do or not, the registry is what finally says yes or no to a registration, right?

> Adjust the domain check ext so the language tag element is optional. No language tag and it assumes a straight ASCII registration.

Yes, check should of course have the same argument(s) etc as create.

> Then those registrars that want to validate their own work can and registrars that aren't sure can test a punycode string against the extended domain check. This seems like a compromise that works,…

Makes sense of course but why do a check and not create directly, that might fail?

You are absolutely correct that a registrar might do a create if they believe they MIGHT have a success. Specifically if you have a race in the form of backorder you do, and you are limited in the number of commands per time unit you are allowed to do. And as you say, a registrar can not risk rejecting a domain registration for a registrant that might in fact work.

If all registries, and I mean all and not only the gTLD registries that the accredited registrars run against, had a few fewer extensions, and used similar rules for things like transfer and whatever else...

So, can you please explain what the problem is with having the registries get some creates that will fail?

Then I will understand better.

   Patrik

---

From: Michael Young <[email-masked]>
Date: Wed, 4 Jan 2012 16:19:52 -0500

Well a domain check is a read only transaction in registry, in a create, you need to hold that create while you validate it in case someone else is racing for the same unique string. Given load balancers and multiple app servers you either time synch at the app server or the DB. Most would go for the DB to lock then you face a rollback if the validation fails. Way back when domain drops weren't rate limited registries faced "add storms". It's not tragic with today's rate limiting controls but I still note why would I encourage any system to do more work unnecessarily,.....

Michael Young

M:647-289-1220

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Wed, 4 Jan 2012 21:34:08 +0000

On 4 Jan 2012, at 21:19, Michael Young wrote:

> Well a domain check is a read only transaction in registry, in a create, you need to hold that create while you validate it in case someone else is racing for the same unique string. Given load balancers and multiple app servers you either time synch at the app server or the DB. Most would go for the DB to lock then you face a rollback if the validation fails. Way back when domain drops weren't rate limited registries faced "add storms". It's not tragic with today's rate limiting controls but I still note why would I encourage any system to do more work unnecessarily,…..

So you want us to do the extra work? :)

>> Then I will understand better.

Mr Michele Neylon
Blacknight Solutions
Hosting & Colocation, Brand Protection
ICANN Accredited Registrar
http://www.blacknight.com/
http://blog.blacknight.com/
http://blacknight.mobi/
http://mneylon.tel
Intl. +353 (0) 59  9183072
US: 213-233-1612 
UK: 0844 484 9361
Direct Dial: +353 (0)59 9183090
Fax. +353 (0) 1 4811 763
-------------------------------
Blacknight Internet Solutions Ltd, Unit 12A,Barrowside Business Park,Sleaty
Road,Graiguecullen,Carlow,Ireland  Company No.: 370845

---

From: Michael Young <[email-masked]>
Date: Wed, 4 Jan 2012 16:48:13 -0500

Well you guys with your countless inexpensive multicore Linux boxes do have the horsepower advantage,.....

Plus let's face it - you're the smart one ;-)

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Wed, 4 Jan 2012 21:52:39 +0000

> Plus let's face it - you're the smart one ;-)

G'wan - try charming your way out of it :)

Mr Michele Neylon
Blacknight Solutions

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 4 Jan 2012 17:00:58 -0500

On Wed, Jan 04, 2012 at 04:19:52PM -0500, Michael Young wrote:

> Well a domain check is a read only transaction in registry, in a
> create, you need to hold that create while you validate it in case
> someone else is racing for the same unique string. Given load
> balancers and multiple app servers you either time synch at the app
> server or the DB. Most would go for the DB to lock then you face a
> rollback if the validation fails.

There are, of course, other possible implementations, some of which
would not face this exact problem.  (And indeed, the effects of
rollback might be implementation-dependent as well.)  Therefore,

> Way back when domain drops weren't rate limited registries faced
> "add storms".

the add storm problem was at least partly an issue related to
implementation, and not the protocol itself.

While it is clearly a good thing that a client attempt to validate its
input as well as possible before submission to an EPP server, it's
equally obviously true that sometimes a client is going to be in a
race to create an object.  The protocol cannot rely on the client
having checked that a creation meets registry policy prior to
submission.  So, while I certainly have no objection to including
extensions to the check command (and think it's a good idea), I sure
want to make sure nobody is planning to rely on check commands to save
us from expensive validation of submitted name objects at create time.

Best,

A

-- 
Andrew Sullivan
<[email-masked]>

---

From: Michael Young <[email-masked]>
Date: Wed, 4 Jan 2012 19:15:49 -0500

Short Answer: I agree the create command has to be able to handle the use cases. Somewhere in all that I noted you were ok with the check ext being optional :-)

---

From: Andrew Sullivan <[email-masked]>
Date: Wed, 4 Jan 2012 22:34:31 -0500

On Wed, Jan 04, 2012 at 07:15:49PM -0500, Michael Young wrote:
 
> the use cases. Somewhere in all that I noted you were ok with the
> check ext being optional :-)

Actually, you misread, then.  I think servers (implementing this
extension) MUST support the extension to check.  Clients MAY send it
or not (which means that implementation is for practical purposes
optional for them.

Best,

Andrew

---

From: MICHAEL YOUNG <[email-masked]>
Date: Wed, 04 Jan 2012 22:44:51 -0500

Sorry do you mean if you were designing this from scratch this is what you
would do? Or do you mean this is how you read one of the existing
proposals (and by that I mean any of the ones mentioned so far)?

Michael

---

From: Patrik Fältström <[email-masked]>
Date: Thu, 5 Jan 2012 07:45:32 +0100

On 4 jan 2012, at 22:19, Michael Young wrote:

> you need to hold that create while you validate it in case someone else is racing for the same unique string

Why?

You want the registrar to do it during a registration if the registrar is to do a check first, and create afterwards.

I.e. the need for a lock MIGHT exist anyway at the time of a registration, and it is only a question who is doing it.

In a registry when you do a create you can as well first do a validation according to your rules before you do the lock and try to actually do the registration (creation of a connection between a domain object and a contact object).

So I do not agree with you that you have to do a lock before you validate that the syntax of the domain name is acceptable.

   Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Thu, 5 Jan 2012 07:49:39 +0100

> proposals (and by that I mean any of the ones mentioned so far)?

If we designed epp from the beginning:

- We would not have the ability for registries to invent their own extensions without IETF review (i.e. like other protocols)

- We would have optimized a few of the commands (specifically create) so that it is closer to the business models

Example of the 2nd are all the registries that require delegation at time of registration (something that should be forbidden, but now some registries are like that), that required creation of contact objects and host objects before the domain object is created and glued together. If at that point in time the create fails, the creation of the other objects would have automatically been undone. Maybe as a transaction, I do not know. We should have thought about it harder.

  Patrik

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 5 Jan 2012 07:33:58 -0500

> proposals (and by that I mean any of the ones mentioned so far)?

I don't think I mean either.  I think what I mean is that the draft
should include your suggested extension to the check command, and that
such extension (like all the rest of them) be mandatory to implement
(i.e. just like the other MUSTs in the draft).

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: Michael Young <[email-masked]>
Date: Thu, 5 Jan 2012 13:51:09 -0500

Thanks

Michael Young

M:647-289-1220

---

From: Michael Young <[email-masked]>
Date: Thu, 5 Jan 2012 14:00:06 -0500

Interesting thoughts - I'm really a big fan of avoiding multiple extensions that effectively do the same thing. I think we could benefit by having some more structure around how ICANN works/depends with IETF efforts. Right now it seems pretty reactive to industry needs versus maybe trying to anticipate needs. 

Michael Young

M:647-289-1220

---

From: Andrew Sullivan <[email-masked]>
Date: Thu, 5 Jan 2012 14:21:16 -0500

On Thu, Jan 05, 2012 at 02:00:06PM -0500, Michael Young wrote:

> Interesting thoughts - I'm really a big fan of avoiding multiple
> extensions that effectively do the same thing. I think we could
> benefit by having some more structure around how ICANN works/depends
> with IETF efforts. Right now it seems pretty reactive to industry
> needs versus maybe trying to anticipate needs.

ICANN doesn't actually control a very large number of the registries
it deals with, as any ccTLD manager will tell you at length if you
mistakenly phrase things as though perhaps ICANN has any influence of
any sort on them.  So it'd probably just cause heartburn to start by
thinking about how ICANN is involved.

Anyway, it seems to me that the basic problem with EPP extensions is
that everyone thinks everyone else's approach is wrong, because most
of the approaches are actually designed around an implicit underlying
data model.  Nobody ever wants to compromise on that, so you either
get extensions that are bloated pigs of optional elements, or else you
get a lot of extensions.  (The preposterous handling of name servers
as both references to host objects and as attributes of domain objects
in the original EPP documents is an example of this.  I understand why
it happened, and I have no criticism that the protocol ended up this
way; but if one wanted to design a clean data exchange protocol,
something this fundamental would not have two completely different
ways of handling the same sort of data.)

I think the problem is not only the number of extensions.  It seems to
me that if they were handled in a more co-ordinated way, that would be
good; but since they're not likely to be any time soon, then we need
to find another way.  The other way would be to fix client systems so
that pluggable extensions were considerably less clumsy to use.  The
main reasons I heard people complaining about extensions in the past
was because they needed to do significant development every time one
was encountered.  But in a well-designed system, that shouldn't be
true.  For a significant extension of additional data elements, one
has to do development anyway in order to make the data store
correspond to the new data.  For an extension that is really just
another, not-invented-here way of achieving the same registration
goal, a mapping of existing elements ought to be trivial.  In the
former case, then, some development is necessary anyway, and it's only
a poor client design that would make adding the additional extension a
big deal.  In the latter case, adding another way of expressing the
same data should be pretty trivial (although, obviously, an excellent
way to introduce pointless bugs).

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Michael Young" <[email-masked]>
Date: Thu, 5 Jan 2012 17:28:10 -0500

Andrew, 10 years of reading your emails and I am still amazed with how well
you write them,.....

Your thoughts on ccTLDs  are so true in the current state, maybe that
balance changes after a couple of new TLDs application rounds, maybe not.

ccTLDs have no entity like ICANN to try and enforce some common practises,
with good reason, they represent sovereign states so it's appropriate they
are independent. However it would be in everyone's interest to reduce work,
effort and confusion by forming standards.

A strategy of making client implementation easier is a  good idea, but it
needs to extend to the registrant and storefront as well. For example
different terminology in different registries for a data element or
modifier/attribute is confusing for a domain user even if we find a way to
make epp client implementation cheaper (look at our discussions of language
versus script even).  I feel like we are just going to have to work through
every common extension and its variations in order to consolidate to a
single standardized version - no easier shortcuts .  That means that each
extension we would try and to do this with has to be flexible enough for
everyone's use cases.  I feel sometimes we get debating uselessly over the
validity of a use case when there's an easy option to accommodate multiple
use cases.  Its really seldom that accommodating one use case means shutting
out another one altogether - although it does happen.

So then, how do we organize to try and make some progress here?  Does
everyone think we should try and organize in the first place?

-M

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Andrew Sullivan
Sent: January-05-12 2:21 PM
To: [email-masked]
Subject: Re: [provreg] Domain check in draft-obispo-epp-idn-00.txt

> needs versus maybe trying to anticipate needs.

ICANN doesn't actually control a very large number of the registries it
deals with, as any ccTLD manager will tell you at length if you mistakenly
phrase things as though perhaps ICANN has any influence of any sort on them.
So it'd probably just cause heartburn to start by thinking about how ICANN
is involved.

Anyway, it seems to me that the basic problem with EPP extensions is that
everyone thinks everyone else's approach is wrong, because most of the
approaches are actually designed around an implicit underlying data model.
Nobody ever wants to compromise on that, so you either get extensions that
are bloated pigs of optional elements, or else you get a lot of extensions.
(The preposterous handling of name servers as both references to host
objects and as attributes of domain objects in the original EPP documents is
an example of this.  I understand why it happened, and I have no criticism
that the protocol ended up this way; but if one wanted to design a clean
data exchange protocol, something this fundamental would not have two
completely different ways of handling the same sort of data.)

I think the problem is not only the number of extensions.  It seems to me
that if they were handled in a more co-ordinated way, that would be good;
but since they're not likely to be any time soon, then we need to find
another way.  The other way would be to fix client systems so that pluggable
extensions were considerably less clumsy to use.  The main reasons I heard
people complaining about extensions in the past was because they needed to
do significant development every time one was encountered.  But in a
well-designed system, that shouldn't be true.  For a significant extension
of additional data elements, one has to do development anyway in order to
make the data store correspond to the new data.  For an extension that is
really just another, not-invented-here way of achieving the same
registration goal, a mapping of existing elements ought to be trivial.  In
the former case, then, some development is necessary anyway, and it's only a
poor client design that would make adding the additional extension a big
deal.  In the latter case, adding another way of expressing the same data
should be pretty trivial (although, obviously, an excellent way to introduce
pointless bugs).

A

---

From: Patrik Fältström <[email-masked]>
Date: Fri, 6 Jan 2012 08:21:15 +0100

On 5 jan 2012, at 23:28, Michael Young wrote:

> versus script even).

Michael, the "store front" registrars have MUST have the same taxonomy etc for all TLDs, so there is already a translation/adoption mechanism in the registrar system. I.e. differences in terminology and data elements between TLDs is already today not visible to the end users.

If registries did harmonize more, it would make registrar work easier. And more registrars would become customers to more registries. And more harmonization would also because the translation engines in the registrars act more similar which implies the users would get more similar user interface when comparing registrars.

Today, having similar user interfaces for for example .EU, .COM, .ORG and .SE is not easy. The TLDs are so completely different in the epp implementation that they are...hmm...different.

   Patrik

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 6 Jan 2012 07:17:40 -0500

> effort and confusion by forming standards.

[. . .]
 
> everyone think we should try and organize in the first place?

This list is already almost as organized as it gets at the IETF, and
I'd suggest that much more organization would hinder, rather than help
the effort.

"More organized" than this at the IETF usually means a working group.
I am not at all convinced that the structure of a WG will help,
because in many cases a WG would just add overhead.  I'm not even sure
one could get chartered, because a BoF might not attract enough
interest to get the sort of meaningful review that would be required.

Moreover, the simple fact is that there are no protocol cops on the
Internet.  I suppose that ICANN could require (of gTLD registries)
some specific RFCs be the mechanism by which this or that feature be
offered in EPP.  But that doesn't require additional organization
within the IETF.  It requires greater participation and support on the
part of registries and registrars.  Registries have had almost 10
years of extensions in which they could have co-ordinated with one
another, but they didn't.  If they want to start, brilliant; but that
requires no more structure than we have now, because ultimately it's
just a matter of coming to agreement about how some extension will
work.  

I guess one thing that perhaps could be a good idea is to promote this
mailing list to people outside the circle of those who originally
participated in or monitored EPP development.  I suspect that there
are people around ICANN, for instance, who either would have an
opinion about EPP extensions or who employ someone wo does, but who
don't know about this list and don't follow it.  Getting them involved
would be a good idea.

Best,

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Gould, James" <[email-masked]>
Date: Fri, 6 Jan 2012 14:24:20 +0000

I see a combination of both the informal interaction of this list as well
as formal interaction of a WG.  As I stated before, EPP was built for
extensibility and extensibility itself is not evil. EPP's extensibility is
a feature but best practices should be followed (e.g. make them opt-in
extensions) when creating them and it makes sense to move some of the
custom extensions up the standards track to create consistency.  There is
a lot of work involved with going up the standards track for an extension
so it really needs to be warranted.  I believe with the launch of a large
set of new TLD's it makes sense to at a minimum have a BoF to discuss the
custom extensions to see if a set of common / standard extensions can be
identified.  Also with the work being done on the launch phase policies
and approaches being worked out at ICANN it makes sense to consider a
parallel IETF WG to define the protocols.  There is the launchphase
extension drafted by Will Tan and Gavin Brown that is a good starting
point for interaction between the Domain Registrar and the Domain
Registry, but what about the protocol with the Trademark Clearinghouse
(TMCH) that can include the provisioning of trademarks (e.g. Trademark
Registrar), the query interface and the notification interface between the
Domain Registry and the TMCH, and there might be an interface required for
the Domain Registrar.  I also have the idea of creating a Domain Registry
EPP Mapping that at a minimum would include support for an info command
and response for providing the features and policies for a TLD or set of
TLD's. This would provide information to the Registrars that could help
automate and subsequently scale the number of TLD's.

-- 

JG

James Gould
Principal Software Engineer
[email-masked]

703-948-3271
21345 Ridgetop Circle
LS2-2-1
Dulles, VA 20166

VerisignInc.com

---

From: MICHAEL YOUNG <[email-masked]>
Date: Fri, 06 Jan 2012 11:52:07 -0500

:-) 

-M

On 12-01-06 2:21 AM, "Patrik Fältström" <[email-masked]> wrote:

>Michael, the "store front" registrars have MUST have the same taxonomy
>etc for all TLDs, so there is already a translation/adoption mechanism in
>the registrar system. I.e. differences in terminology and data elements
>between TLDs is already today not visible to the end users.
>
>If registries did harmonize more, it would make registrar work easier.
>And more registrars would become customers to more registries. And more
>harmonization would also because the translation engines in the
>registrars act more similar which implies the users would get more
>similar user interface when comparing registrars.
>
>Today, having similar user interfaces for for example .EU, .COM, .ORG and
>.SE is not easy. The TLDs are so completely different in the epp
>implementation that they are...hmm...different.
>
>   Patrik
>

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 17:02:57 +0000

> What can I say, it would be great if the registrar put that kind of intelligence in their storefront and didn't send domain creates that violate a given language table.

I agree completely!

> Most registrars aren't making that extensive of an effort. Why?

I think I can shed some light on that, and there are a number of ways that
registries can help registrars with this.

I spent a good amount of time hacking together a set of libraries and tools
to help deal with it on our end. The key was to find a way of representing
the whitelists that (a) wouldn't consume a tonne of memory, (b) would be quick
to unmarshall from disc, (c) would work effectively with both alphabetic
scripts and non-alphabetic scripts, and (d) would have minimal CPU overhead.
After a number of experiments I hit upon compiling the codepoint whitelist
into the following format[1]:

First, I imagined each Unicode plane as divided up into 256 slices. This was
a useful as it corresponds pretty closely with how codepoints were allocated
in the first plane at least, so each slice is typically going to either empty
or fairly densely packed with whitelisted codepoints. I used 256b bitvectors
to represent non-empty slices, along with a single bitvector with all the bits
off for slices with no whitelisted characters.

Next, for each slice in a plane, I stored the index of a bitvector that
corresponds to the whitelisted codepoints within that slice. In addition, it
maintains a bitfield of the planes represented within the table. This allowed
for a compact representation of the whitelisted CJK codepoints in the higher
planes (especially for Chinese) while having only a minor impact on the size
of tables for languages represented BMP and SMP. It also kept the speed of
unmarshalling low.

Finally, there's an optimisation stop to remove any duplicate non-empty
slices. This was particularly useful for Korean.

Each table is tagged with a language code and compiled into a set of files,
one for each registry.

Now, I chose this way of doing things as I needed to be able to load those
tables repeatedly into memory from disc from various languages, including C,
Python and PHP. If our systems were purely written in C and Python, I'd have
just loaded textual representations of the whitelists into memory, but with
PHP, compiling the tables into form that can be quickly unmarshalled was a
huge win.

Getting this right took a lot of thought, however, and the registrar business
is very low margin, so the kind of near obsessive attention to detail that I
showed with this isn't going to be common amongst registrar because acting
fast and loose is more cost-effective.

If you want registrars to behave like you'd prefer (which is the way they
ought to be acting anyway), you have to make it cost them: a demerit system
would help. If a registrar repeatedly sends IDNs to either the <domain:check>
or <domain:create> commands that don't match any of the registry's whitelists,
that registrar ought to be penalised.

As a carrot, registries have to make it easy for registrar to build and
maintain whitelists. IANA's Repository of IDN Practices solves half the
problem here, but it's still deficient. The tables provided typically aren't
trivially parseable. Some, such as Telnic's, are fine, but some of the HTML
codepoint table representations seem as if they were designed to frustrate
ease of parsing. And don't get me started on the tables submitted as PDFs!
It'd also help if IANA provided an Atom feed listing additions and revisions
to the repository so that registrars could automatically parse and download.

More registry operators should publish their whitelists in IANA's practices
repository, and Afilias for one have yet to register anything other than their
German whitelist with IANA, even after over five years of supporting seven
other languages besides German. These things are hardly business secrets,
after all, so I can't see why nobody's got around to emailing the things into
IANA yet, nor can I understand why anybody should have to log into the
registrar relations area to get their hands on them.

> Adjust the domain check ext so the language tag element is optional. No
> language tag and it assumes a straight ASCII registration.

> Then those registrars that want to validate their own work can and
> registrars that aren't sure can test a punycode string against the extended
> domain check. This seems like a compromise that works, thoughts?

That's not really a compromise: that's the status quo of requiring a language
code when checking availability.

Here's a compromise that works for me: registrars should be allowed to submit
straight ASCII domains and IDN domains to both the <domain:check> and
<domain:create> commands, but registries should ensure that the domain is
valid against their codepoint whitelists. If a registrar submits domains to
either command that don't match at least one whitelist, they should be marked
with a demerit and the request rejected. Once a registrar reaches a certain
threshold, the registry should notify the offending registrar that they risk
sanction for submitting bad IDNs. Once they hit a higher threshold, they
should should be notified that they have been penalised for their behaviour.
The form of penalisation should be at the registries discretion, and might
take the form of a higher cost per transaction, or some other form.

That leaves registrars with a choice: either implement IDN support properly in
their domain management system and stores or don't offer IDNs.

K.

[1] I've yet to deal with the issue of variants all that well. That's
    something currently exercising my mind because until I come up with
    a solution, our support for CJK IDNs is less than ideal.

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Fri, 6 Jan 2012 17:04:04 +0000

+10000 Patrik

Registry operators are their own worst enemies!

>> implementation that they are...hmm...different.

Mr Michele Neylon
Blacknight Solutions ♞
http://blacknight.biz
Locall: 1850 929 929
Facebook: http://fb.me/blacknight
Twitter: http://twitter.com/mneylon

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 17:09:49 +0000

On 04/01/12 20:44, Francisco Obispo wrote:

> I agree that registrars should be performing those tests/validations ahead
> of time and shouldn't rely exclusively on the registry for those types of
> validation, because that would increase registry load significantly.

I very much doubt that it would increase registry load significantly, and I
know because I've implemented code to do exactly that kind of validation, and
the additional extra load was negligible in terms of overall request
processing time. It wasn't even a blip compared to even the overhead of
marshalling and unmarshalling EPP stanzas.

Both registrars and registries should be checking IDNs against codepoint
whitelists.

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 17:18:13 +0000

> So you want us to do the extra work? :)

Now, to be frank, doing the extra work when sending <domain:create> isn't a
big deal, but I'd personally prefer to avoid having to divvy up a list of
domains into separate <domain:check> requests if at all possible.

Being able to batch domain availability checks from multiple customers at
once is a big win for us. Language tags in <domain:check> commands negate
this benefit. So, yeah...

K.

---

From: Francisco Obispo <[email-masked]>
Date: Fri, 6 Jan 2012 09:18:44 -0800

> Both registrars and registries should be checking IDNs against codepoint

I agree, _both_ should be doing it,... my point above was about registrars exclusively performing validations using the <check> command.

At the end, I think this is implementation-specific, we might argue that it will or not, but it all comes down to the platform that is handling it.

Thanks 

Francisco Obispo 
email: [email-masked]

---

From: Francisco Obispo <[email-masked]>
Date: Fri, 6 Jan 2012 09:22:54 -0800

you would have to batch them by language, which could be cumbersome. 

The alternative, would be to provide domain:check with A-labels

Francisco Obispo 
email: [email-masked]

---

From: Francisco Obispo <[email-masked]>
Date: Fri, 6 Jan 2012 09:24:30 -0800

hit "send" to early....

> The alternative, would be to provide domain:check with A-labels

and provide an interface in the idn extension to enumerate the language for each <domain:name> supplied.. but it will most likely by ugly.

Francisco

Francisco Obispo 
email: [email-masked]

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 17:34:04 +0000

On 06/01/12 17:18, Francisco Obispo wrote:
> I agree, _both_ should be doing it,... my point above was about registrars
> exclusively performing validations using the <check> command.

Which is exactly why I think registries should be penalising registrars the
are plainly not validating IDNs they submit in <domain:check> commands rather
than requiring language tags be provided in <domain:check> commands.

Essentially, I'd prefer misbehaving registrars were penalised rather than
having well-behaved registrars such as ourselves[1]. Without the language tag,
we're able to batch multiple checks from multiple customers into a single
request. This makes our store more responsive. Requiring a language tag be
provided means this ability to batch checks is reduced because we'll have to
group domains in batches based upon the language code.

I can't emphasise how important it is for a registrar that domain availability
checks be quick. Anything that compromises response time in this area means
potential loss of sales.

> At the end, I think this is implementation-specific, we might argue that it
> will or not, but it all comes down to the platform that is handling it.

K.

[1] Though anybody who's met Michele may disagree!

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 17:42:50 +0000

> you would have to batch them by language, which could be cumbersome. 

This is what we already do for registry operators, such as Neustar, who
already require language codes be provided when performing a <domain:check>.
And yes, it is cumbersome, which is why I'm so dead-set against the extension
requiring language tags be submitted when making <domain:check> request that
contains one or more IDNs.

> The alternative, would be to provide domain:check with A-labels

We use A-labels internally for everything already. It's less prone to errors
such as accidental encoding issues than using U-labels everywhere. The only
times we deal with U-labels is when accepting requests from our store and
validating IDNs.

Unfortunately, that doesn't negate the need to send language codes to
registrars.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 17:46:21 +0000

> and provide an interface in the idn extension to enumerate the language for each <domain:name> supplied.. but it will most likely by ugly.

That's something I thought of myself, and while preferable because it at
least allows for batching, you're right, it would be ugly, not least because
it means the same data gets repeated twice in the one request.

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 18:01:18 +0000

> or not (which means that implementation is for practical purposes

Very true, and that I would be OK with, though I suspect many registrars
would simply ignore the extension of <domain:check>, making the requirement
that registries support is somewhat pointless.

Still, you've left out the most important reason why servers ought to be
checking IDNs: registrars and their customers can lie. Just because they've
filled out they filled out the language tag doesn't mean that language code
has any bearing on reality. At best, if they are validating, it's just a way
for the registry to cut down on the number of whitelists they need to check.

Now, there are reasonable reasons for a registrar to lie. We, for instance,
only take a guess at the language an IDN might be for and for the likes of
Neustar who require a language code be provided, we provide that to them.
The domain might actually be an Irish-language one like
rásrothairnabhaile.biz[1], but that'll fit the table for French, which means
as far as the registry is concerned, that's a valid IDNs.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 18:17:20 +0000

On 05/01/12 06:49, Patrik Fältström wrote:

> - We would not have the ability for registries to invent their own
> extensions without IETF review (i.e. like other protocols)

An unfortunate consequence of that would be that some registries would just
fork the protocol and use something that's almost-but-not-quite EPP. EURid
went and did this, and when I wrote our EPP client, it was built to account
for nonsense like this, which made the code much more complex than I'd like.

> Example of the 2nd are all the registries that require delegation at time
> of registration (something that should be forbidden, but now some
> registries are like that), that required creation of contact objects and
> host objects before the domain object is created and glued together. If
> at that point in time the create fails, the creation of the other objects
> would have automatically been undone. Maybe as a transaction, I do not
> know. We should have thought about it harder.

Actually, I've no problem with creating contacts, then having the subsequent
registration fail. But what I'd prefer to be different is that I think the
split between hosts as objects and hosts as domain attributes was a mistake.
In addition, I think that if a <domain:create>, &c., request is made and a
set of nameserver hosts are specified on it that have not been explicitly
created with <host:create>, the registry should implicitly create those hosts.

Another thing I'd change is provide a way for registries to signal to
registrars that they've garbage collected contacts (and possibly hosts too).

K.

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 18:36:21 +0000

> If registries did harmonize more, it would make registrar work easier.

After all, we're all in the business of selling domains, and it's not as
if registries are in competition with one another. The faster I as a
software engineer employed by a registrar can get integrated with you
as a registry, the better for us both, and the more profit for all!

> And more harmonization would also because the translation engines in the
> registrars act more similar which implies the users would get more similar
> user interface when comparing registrars.

Not to mention a smoother experience for all. But unfortunately politics
comes into things and blinds everybody to what's to their mutual advantage.

> .SE is not easy.

EURid are, due to gradual registrar pressure, moving towards a less 'special'
variation of EPP. That's a positive development.

As far as .com goes (and all the VeriSign-managed zones too), I think the best
thing they could do for registrars is scrap their NameStore extension. I can't
fathom the train of thought that made their developers think that'd be a good
idea. It's up there with running .tv and .cc (thin) on the same EPP server as
.jobs (thick), which messes up any chance clients have of autodetecting if the
registry is thick or thin.

.se is on my (long) list of ccTLDs to interface with, so I can't remember how
much of a pain that's going to be.

> The TLDs are so completely different in the epp

The worst of it is the pride some registry operators have in how different
they are. I've read stuff by non-EPP ccTLDs registrars who practically boast
about how they have their own interface and how EPP couldn't possibly suit
their use cases.

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Fri, 6 Jan 2012 18:42:19 +0000

> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Keith Gaughan
> Sent: Friday, January 06, 2012 1:17 PM
> To: [email-masked]
> Subject: [provreg] Changes we'd make to EPP, was Re: Domain check in
> draft-obispo-epp-idn-00.txt
> 

It wasn't originally designed that way. The split came about as a result of a need to build consensus within the working group.

Scott

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 18:45:20 +0000

On 06/01/12 14:24, Gould, James wrote:

> EPP's extensibility is
> custom extensions up the standards track to create consistency.

Severely lacking currently are informational RFCs concerning best current
practice for those implementing EPP clients and servers.

> I believe with the launch of a large
> identified.

I concur.

> I also have the idea of creating a Domain Registry

I mentioned that very idea on this list before Christmas. The rough consensus
was that things like a registry policy ought to be provided independently of
EPP. Some also expressed scepticism that it'd even be possible to account for
all the possible variations on registry policy. It's something I've been
working on during my free time, but due to my lack of free time lately, is not
something I've made as much progress on as I'd like.

---

From: Keith Gaughan <[email-masked]>
Date: Fri, 06 Jan 2012 18:47:26 +0000

On 06/01/12 18:42, Hollenbeck, Scott wrote:

> It wasn't originally designed that way. The split came about as a result
> of a need to build consensus within the working group.

Yes, I remember reading about that when I was going over the list archives
a while back. I understand why it happened, but it still appear to have been
unfortunate.

K.

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 6 Jan 2012 13:58:06 -0500

> registrars that they've garbage collected contacts (and possibly hosts too).

This, at least, ought to be possible using the poll queue.  I think
the poll queue as it stands is less than ideal, however.  I made a
suggestion about how to make the poll queue better for these purposes
in a long-expired draft (draft-sullivan-epp-experience-00), but they
weren't very popular.

-- 
Andrew Sullivan
[email-masked]

---

From: Andrew Sullivan <[email-masked]>
Date: Fri, 6 Jan 2012 14:03:35 -0500

> that registries support is somewhat pointless.

Of course some would ignore it.  Many registrars do blind create
today, too, rather than performing a check.  

> checking IDNs

I had no intention of suggesting that the server side (formally, the
repository in EPP-speak) needn't check input.  I personally would
prefer that gTLD registries do more validation, not less, and I think
you're a moron if you don't check your input for minimal compliance
with your own policies.

> as far as the registry is concerned, that's a valid IDNs.

Well, guessing will only get you so far.  If your customer sends you a
string in NFD form, what do you do?  What about in ISO8859-1?

A

-- 
Andrew Sullivan
[email-masked]

---

From: "Gould, James" <[email-masked]>
Date: Fri, 6 Jan 2012 20:28:14 +0000

Keith,

> registry is thick or thin.

The NameStore extension is used for routing requests to the correct
back-end registry using a single EPP connection.  The question is whether
you would want to manage a separate session pool per TLD instead of
managing a single session pool per system and passing the NameStore
extension to indicate the logical registry to direct the request to.  The
train of thought is to provide a single destination for a set of EPP
services where there might be multiple services that support the same XML
namespaces (EPP mappings).  Getting meta-data on a TLD level is the goal
of the Registry EPP Mapping that I would like to create so that a
Registrar can query or be notified of the set of EPP mappings, EPP
extensions, features and policies supported on a per TLD level.  Knowing
only the supported EPP mappings and extensions (auto-detection) I don't
believe will be enough to fulfill the goal of automating the enablement of
a new TLD.    

-- 

JG

James Gould

703-948-3271
21345 Ridgetop Circle
LS2-2-1
Dulles, VA 20166

VerisignInc.com

On 1/6/12 1:36 PM, "Keith Gaughan" <[email-masked]> wrote:

>>similar

---

From: "Gould, James" <[email-masked]>
Date: Fri, 6 Jan 2012 21:51:27 +0000

> something I've made as much progress on as I'd like.

A goal is to provide 90% of the feature and policy information to help
drive the client logic.  The mapping certainly wouldn't be small and might
have to make external references for things like reserved names.  I have
done some work on it, but I wanted to know if there is interest in it.
One of the key attributes would be the phases of the TLD (sunrise, rights,
steadyState) with start and end dates.  I might be dreaming that something
useful could be created, but it might be worth a shot.

-- 

JG

James Gould

703-948-3271
21345 Ridgetop Circle
LS2-2-1
Dulles, VA 20166

VerisignInc.com

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 7 Jan 2012 10:47:57 +0100

On 6 jan 2012, at 19:17, Keith Gaughan wrote:

> registration fail.

The case that could be discussed is how to handle transfer of domain objects. What happens with the host and contact objects? That would be good if there was an agreement on that. And I talk about epp here, and not the business model, although they are hard to separate from each other.

   Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Sat, 7 Jan 2012 10:49:48 +0100

> in a long-expired draft (draft-sullivan-epp-experience-00), but they

I rather see the registrar do "garbage_collect()" and the registry respond with a list of objects that was deleted. That would limit the amount of time the view of the world on the registry and registrar side is different.

   Patrik

---

From: Keith Gaughan <[email-masked]>
Date: Mon, 09 Jan 2012 14:53:29 +0000

On 06/01/12 18:58, Andrew Sullivan wrote:

> This, at least, ought to be possible using the poll queue.

Myself and Gavin Brown were chatting about this before Christmas because
CentralNIC have big problems with their registrars creating large numbers
of contact objects and then not using them.

Solving it cleanly is actually suprisingly awkward. It's really something that
ideally ought to be on the contact mapping (and possibly the host mapping
too). The extra message could be introduced as its own type of object (as some
registries have done for low balance notifications), but that's very much a
nasty and awkward way to bolt it on.

Alternatively, it could be done as an extension, which is less nasty and
awkward, but I've yet to come across a single instance of a message queue
message introduced as an extension, and thus chances are registrars would be
blindsided by poll responses with no response data, which could lead to a lot
of bleating[1] from registrars about having to re-engineer how they process
the message queue, even though it's something that's always been a possibility
according to the RFC[2].

I'm considering writing up an I-D for this, which is one of the reasons I
mentioned it.

> I think

I read that a while back, and I remember not being entirely convinced by the
need myself. After all, there's nothing requiring messages on the message
queue from having a particular temporal ordering, and there's certainly no
requirement for message IDs to be monotonically increasing numbers. After all,
there's no requirement for it to be a FIFO.

Registrars who want to solve the problem could do so by using a priority queue
where queue ordering is based off of the time the message was placed on the
queue and a internal numerical priority assigned to it. The queue itself could
then be implemented as a binary or fibonacci heap[3], or some other
datastructure that might suit the registry's needs better.

K.

[1] Given I work for a registrar, I think it's safe for me to say things like
    this.

[2] I've experience of this from another mailing list I'm on from one
    registrar who couldn't understand one is only meant to send back the
    subset of supported objects and extensions they support that were
    mentioned in the greeting when they log in.

[3] The latter would seem to me to be the better all-round choice, though the
    cost of insertions and deletions to a binary heap could be lowered by only
    performing the heapify operations after a small number of pops (say three)
    and when there are medium- or high-priority messages to be placed on the
    message queue.

---

From: Keith Gaughan <[email-masked]>
Date: Mon, 09 Jan 2012 16:30:48 +0000

> today, too, rather than performing a check.  

There's a special circle in hell for people who do that.

> you're a moron if you don't check your input for minimal compliance

Very much agreed, though I was mentioning as I felt it needed to be mentioned.
What's left unsaid can easily be overlooked, after all.

> Well, guessing will only get you so far.

To be clear, we only cheat like this when checking, and we wouldn't be able to
sell .biz IDNs if we didn't, because if we *did* ask what language they were
searching in, that places a roadblock in the way of customers when performing
availability checks, which makes them less likely to buy from us. Fewer
roadblocks means more sales. That's why guessing the language when checking is
actually a good thing from both the registrar and customer's point of view.

> If your customer sends you a string in NFD form, what do you do?

NFD isn't too bad. It's easy enough to detect, and on our end we ensure
(insofar as is possible) that everything's in NFC form before checking.
Combining characters aren't that difficult to detect.

> What about in ISO8859-1?

That's potentially a bigger problem because somebody *may* somehow manage to
construct a ISO8859-1(5) encoded label that looks like valid UTF-8, but in
many cases, it's easy to detect and reject due to unexpected synchronisation
bits. It's not perfect, but it at least helps.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Mon, 09 Jan 2012 17:44:41 +0000

> useful could be created, but it might be worth a shot.

I'd be happy to see what work you've done on this so far, and I'll see if
there's anything I can contribute from the work I've done.

*Anything* in this regard is a good thing, even if it does provide 90% of
the policy information, that means less guesswork and special-purpose hackery
for registrars to deal with. Pairing this up with an informational RFC on
best current practice would help get us close to 100%.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Mon, 09 Jan 2012 18:02:00 +0000

On 07/01/12 09:47, Patrik Fältström wrote:

> The case that could be discussed is how to handle transfer of domain objects.
> What happens with the host and contact objects? That would be good if there
> was an agreement on that.

That's a massive grey area. Some registries duplicate the contact objects[1],
others appear to have a rule stating that contacts associated with a domain
under the control of a registrar are queryable (though naturally not
modifiable) even if that registrar is not the actual owner of the contact.

I doubt we'll ever be able to achieve consensus on which way of doing this
kind of thing is better as both have perfectly reasonable arguments that can
be made for them. If my own work, I take the assumption that if a domain has
been transferred to us, that we don't own the associated contacts. That's
only because it allows our system to ignore the case where the contacts have
been duplicated[2], though I've no strong feelings on which policy is better.
This is exactly the kind of thing that would fit into the policy document
James and I have been discussing here.

K.

[1] Obviously host objects will get duplicated. It really makes no sense
    *not* to duplicate them.

[2] Which is also an argument for an object deletion notification mechanism.

---

From: Ulrich Wisser <[email-masked]>
Date: Tue, 17 Jan 2012 09:12:30 +0100

Hi,

here at .Se we have done some extension to notify registrars of database 
changes done by the registry. Today I believe that we do not even need 
an extension for that.

To notify the registrar of changes we could use
   obj:creData
   obj:infData
   obj:delete
for the respective change. I realize that obj:delete is actually used as 
command but there is nothing in the xsd file that prevents reusing it as 
response. The absence of clTRID would indicate that the command was 
initiated by the registry.

/Ulrich

---

From: Ulrich Wisser <[email-masked]>
Date: Tue, 17 Jan 2012 09:15:36 +0100

On 07.01.12 10:49, Patrik Fältström wrote:
> I rather see the registrar do "garbage_collect()" and the registry respond
> with a list of objects that was deleted. That would limit the amount
of time
> the view of the world on the registry and registrar side is different.
>

The answer could be really large and would probably take so much time 
that the registry would have to answer "pending". So you get the reply 
through the poll queue anyway. Besides that, it would make the registry 
dependent on the call from the registrars. What is the incentive for the 
registrar to clean the registry database?

/Ulrich

---

From: Theo Kramer <[email-masked]>
Date: Tue, 17 Jan 2012 10:48:33 +0200

On 17 Jan 2012, at 10:15 AM, Ulrich Wisser wrote:

> The answer could be really large and would probably take so much time that the registry would have to answer "pending". So you get the reply through the poll queue anyway. Besides that, it would make the registry dependent on the call from the registrars. What is the incentive for the registrar to clean the registry database?

I can see none. Garbage collection is a function of registry maintenance imo.
-- 
Regards
Theo

---

From: "Gould, James" <[email-masked]>
Date: Tue, 17 Jan 2012 14:01:55 +0000

Ulrich,

Can you provide a reference to the extension created by .SE for registry
changes for review?  Using a command (obj:delete) in response to a command
(poll) would not meet RFC 5730.  Use of obj:creData I don't believe would
match the context.  Use of obj:infData could work since it's the most
generic to cover any changes, but it wouldn't provide any specific
information on a garbage collection delete except for maybe text in the
<msgQ><msg> element (e.g. "deleted - orphaned" or something like that).
This is not clean but it wouldn¹t require any new extensions or mappings.

--
  
JG

James Gould
Principal Software Engineer
[email-masked] <http://[email-masked]>
 
703-948-3271 (Office)
12061 Bluemont Way
Reston, VA 20190
VerisignInc.com

---

From: Eric Brunner-Williams <[email-masked]>
Date: Tue, 17 Jan 2012 09:37:30 -0500

On 1/17/12 3:12 AM, Ulrich Wisser wrote:
> here at .Se we have done some extension to notify registrars of
> database changes done by the registry.

I proposed a push mechanism to notify clients, it was not adopted, so
the event model is one-sided, having only poll.

> ... Today I believe that we do not
> even need an extension for that.
> 
> for the respective change. I realize that obj:delete is actually used
> as command but there is nothing in the xsd file that prevents reusing
> it as response. The absence of clTRID would indicate that the command
> was initiated by the registry.

Overloading. Cute, but if one were to change EPP some restrictions
that were reasonable in a universe of a half-dozen registries and
about an order of magnitude larger registrars may be worth
reconsideration.

Eric

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 17 Jan 2012 17:19:22 +0100

On 17 jan 2012, at 09:12, Ulrich Wisser <[email-masked]> wrote:

> Hi,
> 
> here at .Se we have done some extension to notify registrars of database changes done by the registry.

This is a question of definitions. The changes are made in the registry due to operations requested by another registrar. I can not remind myself of any operation originating in the registry. Ulrich do remind me if I am wrong.

   Patrik

> Today I believe that we do not even need an extension for that.
> 
> for the respective change. I realize that obj:delete is actually used as command but there is nothing in the xsd file that prevents reusing it as response. The absence of clTRID would indicate that the command was initiated by the registry.
> 
> /Ulrich
>

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 17 Jan 2012 17:26:48 +0100

On 17 jan 2012, at 09:15, Ulrich Wisser <[email-masked]> wrote:

> What is the incentive for the registrar to clean the registry database?

Same as all contractual requirements registries have on their registrars.

You can only minimize number of contact records by having a different design of your system so that it is easier for registrars and registry to be in sync.

We do btw have a local discussion in Sweden on how to work this out.

   Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 17 Jan 2012 17:29:51 +0100

> I can see none. Garbage collection is a function of registry maintenance imo.

A registry can not know if an existing contact record is to be connected to a domain name that is to be created, as by definition contact records must exist with the registry that is not connected to any domain object.

   Patrik

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Tue, 17 Jan 2012 16:41:47 +0000

Registry operators have got to get out of their vacuum mentality

Mr Michele Neylon
Blacknight Solutions ♞

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 17 Jan 2012 17:50:31 +0100

Something like that ;-)

   Patrik

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 10:58:42 -0600

I think there needs to be a reasonable period where you can determine a
contact object has been abandoned.  Is there really a point to leaving
contacts around that haven't been associated for years?

That's certainly the case in the older EPP registries - contact bloat.
Where this gets to be annoying is in escrow deposits - costs money for no
added value to anyone. Why bloat up escrow deposits with a bunch of dead
contacts in the first place?

Just sayin,……..

Michael

On 12-01-17 10:41 AM, "Michele Neylon :: Blacknight"
<[email-masked]> wrote:

>> A registry can not know if an existing contact record is to be
>>connected to a domain name that is to be created, as by definition
>>contact records must exist with the registry that is not connected to
>>any domain object.
>> 
>>   Patrik

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Tue, 17 Jan 2012 17:04:21 +0000

Yes well I rarely win prizes for being overly diplomatic :)

Registries need to work WITH registrars

There's an attitude with some of them that registrars are just an irritation
They also forget that they aren't the only registry that a registrar has to interact with

End result? Only the very big registries manage to get registrars onboard quickly

Mr Michele Neylon
Blacknight Solutions ♞

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Tue, 17 Jan 2012 17:28:05 +0000

Michael

I don't disagree

I think I had this conversation with somebody, but can't remember if it was on this list, in direct email or at a CENTR meeting

The key thing is that whatever way a registry handles it that they make it clear to registrars how they're making that decision and when they're implementing it

For example, some registries cause part of the contact bloat by sending the contact with a domain on transfer, but then not letting the new registrar do anything without first re-creating the contact :)

(Admittedly the one that springs to mind isn't using EPP .. )

M

Mr Michele Neylon
Blacknight Solutions ♞

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 17 Jan 2012 18:28:43 +0100

The problem here is that once again registries talk about "bloat" and "increased cost" and because of that want to do operations that imply inconsistencies with registrars databases, while registrars argue that for them it is not a problem at all with many contact records and that it is worse if registries make commands that have implications in their database.

Is this not a clear sign we must be better on working together?

My suggestion was because of that that registrars make the gc command and registries report back what was deleted.

Only then registrars can immediately register again objects that must stay.

   Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 17 Jan 2012 18:32:55 +0100

> For example, some registries cause part of the contact bloat by sending the contact with a domain on transfer, but then not letting the new registrar do anything without first re-creating the conta…

Or, what .SE does, duplicate contact record at time of transfer. A, I think, good thing but implemented in maybe a bit naive way. Positive things with it is though much higher than negative. We just must take care of the negative implications... For all parties.

   Patrik

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 11:44:42 -0600

You are right about synchronization, the registry needs to inform the
registrar what its doing.  Not sure the poll queue is a viable mechanism
for that, but its the closest thing we have right now. We can do better
though.

This is definitely an area that should be worked on, some of the new
registry use cases introduce new/additional contact types increasing the
problem size.

-M

On 12-01-17 11:28 AM, "Patrik Fältström" <[email-masked]> wrote:

>The problem here is that once again registries talk about "bloat" and
>"increased cost" and because of that want to do operations that imply
>inconsistencies with registrars databases, while registrars argue that
>for them it is not a problem at all with many contact records and that it
>is worse if registries make commands that have implications in their
>database.
>

---

From: Klaus Malorny <[email-masked]>
Date: Tue, 17 Jan 2012 19:12:11 +0100

> registry use cases introduce new/additional contact types increasing the

Hi all,

interesting discussion so far, just want to throw in two points

- depending on the data protection regulations of the country in which
   the registry is located, unused personal data needs to be deleted
   after some time. So registries may actually have no other choice
   if they take data protection serious.

- I just had an insane idea to avoid poll messages about the deletion of
   contacts -- you may forget it just after you have finished
   reading it (or even before): One could introduce an expiration date for
   contacts. If a contact is created, it is set to one year in the future.
   As long as the contact is used at least by one object, it is automatically
   renewed, let's say a quarter before it expires. The registrar can either
   track the expiration date (and perform a contact:info/check to check
   its existence in case his recorded expiration date is in the past)
   or simply create new contacts, as many registrars do it today.
   For the registry, if the contact passes its expiration date, it is
   simply deleted by the registry without any further fuss.

   There are also some design options to keep the effort in the registrar's
   database low, e.g. synthesize the expiration date dynamically while the
   contact is being used.

Regards,

Klaus

---

From: Klaus Malorny <[email-masked]>
Date: Tue, 17 Jan 2012 19:15:03 +0100

> database low, e.g. synthesize the expiration date dynamically while the

Sorry, should read "in the registry's database", but I guess you already became 
aware of it.

Klaus

---

From: Eric Brunner-Williams <[email-masked]>
Date: Tue, 17 Jan 2012 13:16:42 -0500

On 1/17/12 1:12 PM, Klaus Malorny wrote:
> I just had an insane idea to avoid poll messages about the deletion of
>   contacts ...

temporal scope on objects is not insane. i don't recall anyone
suggesting it, either in the xrp or the epp workouts. heck, if junk
can be signed, it can be dated.

-e

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 17 Jan 2012 19:22:27 +0100

On 17 jan 2012, at 18:51, Christopher Browne wrote:

> a) Why should a registrar want to issue this command in the first
> place?

Because the registrar might have services connected with that contact but not in that specific registry. I.e. one registry do not have any knowledge about all services connected to one contact record.

   Patrik

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 12:23:16 -0600

I think its a very interesting idea.

Michael

---

From: Theo Kramer <[email-masked]>
Date: Tue, 17 Jan 2012 20:50:22 +0200

>  database low, e.g. synthesize the expiration date dynamically while the

+1
-- 
Regards
Theo

---

From: Theo Kramer <[email-masked]>
Date: Tue, 17 Jan 2012 20:54:03 +0200

> Registry operators have got to get out of their vacuum mentality

Registry operator responsibilities include maintaining the db - can't see how this is 'vacuum mentality'.
-- 
Regards
Theo

---

From: Theo Kramer <[email-masked]>
Date: Tue, 17 Jan 2012 20:55:37 +0200

> Same as all contractual requirements registries have on their registrars.

So a clause could be that any unlinked contact over a certain age will be deleted...

-- 
Regards
Theo

---

From: "Michele Neylon :: Blacknight" <[email-masked]>
Date: Tue, 17 Jan 2012 19:00:14 +0000

> Registry operator responsibilities include maintaining the db - can't see how this is 'vacuum mentality'.

See my other emails on this subject

Mr Michele Neylon
Blacknight Solutions

---

From: Eric Brunner-Williams <[email-masked]>
Date: Tue, 17 Jan 2012 15:11:28 -0500

> Registry operator responsibilities include maintaining the db - can't see how this is 'vacuum mentality'.

Michele has this correct.

A registry contract holder exists in isolation. If the contract holder
operates its own facilities based registry platform, and has no other
tenants, and its registrar "arm" (prospective) only provides access to
its own registry, then this dyad of registry operator and registrar
operator exist in isolation, relative to the rest of the
bells-and-whistles-EEP-verse.

A registry platform operator with two or more tenants (or expectations
along those lines) exists differently, and has a longer, or broader,
view of the bells-and-whistles-EEP-verse and the associated policy space.

The degenerate platform operator case is the tenant-only
bells-and-policy aware, and otherwise indifferent, operator.

A registrar meeting the access requirements of two or more policy
distinct (some non-shared bells-and-whistles) registries has to take a
very different view of features, changes, and predicable events.

Those implementing multi-namespace services, whether server-side or
client-side, necessarily take a different view from implementors and
operators of mono-namespaces.

The "vacuum mentality" is natural, and hard to fault, for each
isolated island of misfit toys.

Eric

---

From: Christopher Browne <[email-masked]>
Date: Tue, 17 Jan 2012 12:51:34 -0500

> A registry can not know if an existing contact record is to be connected to a domain name that is to be created, as by definition contact records must exist with the registry that is not connected…

Indeed.

When I have brought this up, the focus is always on the business
policy side, not the technical policy.  Everyone gets very concerned
about the notion that we might be destroying data that "belongs" to
the registrars.

And yes, indeed, we don't know if a record is about to be used.
(Though if it has not been connected to anything for several years,
one might take a pretty strong hint that it has been discarded...)

My favourite idea, which has seen zero up-take, has been the notion of
creating a policy along the lines of "Registrars are allowed to have a
certain number of unused contacts 'for free.'  If the number exceeds
that threshold, we'll bill them something, that'll encourage them to
clean up their unused contacts."  The bit of economist in me likes
that form of feedback, and I certainly don't have in mind for this to
be a profit centre, just to make sure that registrars find it of value
to clean up after themselves.  Pushing that through the various layers
of politics seems pretty daunting.

Alternatively, it would be in principle plausible to add some logic
surrounding the LINKED status, to put a TTL onto contacts any time
they change to be unlinked.  What should initiate GC, and how to
communicate the work done back to the registrars is a good question.

Your idea of a "GC" command is certainly interesting, though I see
things to have reservations about:

a) Why should a registrar want to issue this command in the first
place?  I suspect we have quite a few registrars that never care to
pull data coming from their poll queues, and would expect some overlap
between those that don't care about that and those that would never
bother issuing a <epp:garbage-collect> command.

b) What should the return result be?

  1.  Perhaps merely a "Yep, we'll be getting on to that" response.

  2.  If there are 500000 contacts to trim out, I don't imagine it's
reasonable to expect for this request to wait until that is all done.

  3.  It might be nice to get a list of trimmed contacts, but if there
are 500000 of them, that's not likely reasonable.  Proposals for EPP
extensions offering very large response sets have generally gotten
push-back.

  4.  Perhaps the list of contacts trimmed get thrown into the poll
queue.  But 500000 of them mayn't be reasonable to process.  That may
just shove the problem around a little.  Fixing the "dead contacts
problem" may lead us into having much worse problems with poll queue.

---

From: Christopher Browne <[email-masked]>
Date: Tue, 17 Jan 2012 14:50:08 -0500

>  simply deleted by the registry without any further fuss.

I kind of like that, though I have a somewhat different implementation
thought, basically tying the expiration date to whether or not the
contact is linked.

- Upon creation of a contact, it is initially not linked to anything.
Registry ties in an expiration date, and I have no disagreement with
the notion of that being 1 year in the future.

- Upon linking a contact to an object (e.g. - to a domain), that
establishes LINKED status.  This *removes* that expiration date.

- Any time a domain becomes unlinked (e.g. - the last link to a domain
or other object disappears), the registry ties in an expiration date
again.  The clock is ticking on its removal.

It seems preferable to me to not have an expiry date when there is no
reason to want one.

The semantics of what that date means and how it should be manipulated
seem rather strange at times when the contact is being 'actively
used.'  And if it's there, and reported, it seems likely to cause
confusion.

---

From: "Gould, James" <[email-masked]>
Date: Tue, 17 Jan 2012 20:59:16 +0000

I believe this is getting a bit complex and convoluted.  Contacts are
first class objects in themselves as defined by RFC 5733, so they could be
created at any point with no relation to a domain.  I don't believe the
Registry should take any action to garbage collect them based on the lack
of references from other objects like domains.  In systems like DotName a
contact can be linked to other types of objects (emailfwd, defreg, etc.),
so there is no hard dependency with being linked with a domain.  The same
holds true for orphaned hosts, since hosts are not attributes of the
domain unless the host attribute model is chosen.  It makes more sense to
report on orphaned objects (contacts and hosts) out-of-band and for the
Registries to reach out to the Registrars that have an unusually high
number of them.  I'm not sure if we need a system solution to this that
requires protocol changes.

--
  
JG

James Gould
[email-masked]
 
703-948-3271 (Office)
12061 Bluemont Way
Reston, VA 20190
VerisignInc.com

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 15:51:00 -0600

Sure, you can notify registrars all you want,that doesn't mean they will
or are required to do anything about it. On the other hand, if they don't
choose to continue to leave contact objects around for years, not linked
to anything (domains or other objects), then it's reasonable for a
registry to have a policy that allows them to declare them abandoned and
clean them up.

On the other hand, the registry really shouldn't be changing registrar
created/owned objects without synchronizing their activities somehow,
someway, as aptly put by others on the list.

This problem statement been repeated again and again for years.  Clearly,
there is a desire for a solution, although James you certainly entitled to
your opinion :-)

I still think the expiration idea on contacts is graceful and would
support that approach.  I think it informs registrars that contacts which
aren't linked don't last forever and if they are, the contacts auto-renew
so there's no work on the registrar side.

This approach means the registry isn't randomly mucking with their
contacts in a less predictable manner and having to update the registrar
somehow that they deleted their contact objects.

Some EPP registries have 2x or more abandoned contacts than active/linked
contacts (without naming them explicitly). Can you imagine that in .com?
A billion some-odd dead contacts sitting around in your escrow deposits.
   
I do think this could use a technical solution.

Michael Young

---

From: Theo Kramer <[email-masked]>
Date: Tue, 17 Jan 2012 23:53:36 +0200

On 17 Jan 2012, at 10:59 PM, Gould, James wrote:

> created at any point with no relation to a domain.

Yep - so a cost associated with these would solve the problem...

-- 
Regards
Theo

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 16:04:05 -0600

Charging against objects assumes declaring a billing period and addressing
deferred revenues.  Yes it would work but its expensive since it hits a
lot of downstream billing related systems.

Best Regards,

Michael Young
M:647-289-1220

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 17 Jan 2012 17:07:10 -0500

On Tue, Jan 17, 2012 at 03:51:00PM -0600, MICHAEL YOUNG wrote:

> registry to have a policy that allows them to declare them abandoned
> and clean them up.
> 
> This problem statement been repeated again and again for years.
> Clearly, there is a desire for a solution, although James you

I don't understand why this issue is generating such heat on the list.
What's the issue?  Repositories could have any policy they want on
this.  You can already do any of what you want with the existing
protocol, because the protocol has an "out" for bulk operations where
you don't want to fill up the poll queue:

   Message queues can consume server resources if clients do not
   retrieve and acknowledge messages on a regular basis.  Servers MAY
   implement other mechanisms to dequeue and deliver messages if queue
   maintenance needs exceed server resource consumption limits.  Server
   operators SHOULD consider time-sensitivity and resource management
   factors when selecting a delivery method for service information
   because some message types can be reasonably delivered using non-
   protocol methods that require fewer server resources.

> support that approach.

You'd need an extension or a new contact mapping, though.  

A

-- 
Andrew Sullivan
[email-masked]

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 16:21:08 -0600

>You'd need an extension or a new contact mapping, though.

Yes we would, however the poll queue approach is a LOT of chatter even if
you clean it out periodically like some registries do. Its clunky. More to
the bigger question of doing work or not doing work, EPP has been rolling
for 10 years, I think we could suffer a few well thought out improvements
at this point.    

-M

---

From: Jaap Akkerhuis <[email-masked]>
Date: Tue, 17 Jan 2012 23:23:18 +0100

It might even be required to do so. If I remember well, the Dutch
Data Protection Authority requires that old (privacy related) data
which is superfluous should be removed after 7 years. Something
like that.

There are likely similar rules under different jurisdictions.

        jaap

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 17 Jan 2012 17:31:25 -0500

On Tue, Jan 17, 2012 at 04:21:08PM -0600, MICHAEL YOUNG wrote:
> you clean it out periodically like some registries do.

Please read again the section of RFC 5730 I quoted.  There is no
requirement that you put in a poll queue entry for each of these
changes.  That's actually the point of that passage.

Best,

A
-- 
Andrew Sullivan
[email-masked]

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 16:35:15 -0600

Interesting, you'd have to ask the accountants about it, they do like
precise allocations :-)

Michael Young

On 12-01-17 4:26 PM, "Christopher Browne" <[email-masked]> wrote:

>On Tue, Jan 17, 2012 at 5:04 PM, MICHAEL YOUNG <[email-masked]> wrote:
>Maybe, if you try to bill it in a really detailed way against the objects.
>
>What I imagine is better is to be very much NOT detailed about it, to
>NOT try to tie fees to individual objects, rather, charging round
>amounts for ranges.
>
>"Dear Registrar:
>On the evaluation date for this quarter, you had 5430 unlinked
>contacts.  See your reports for a list.
>We charge $250 for storing 1000 to 10000 unused contacts, $500 for
>10000 to 100000, and bulk storage of over 100000 contacts will be
>billed $2500 per quarter for each 100000.
>Your bill, for 5430 unused contacts, is $250."
>
>What is most preferable is to never need to issue a bill for this, as
>dead, unvaluable objects get cleaned out before the evaluation date.

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 16:38:48 -0600

That leaves you with out of band, why is that better?

Michael Young

---

From: Jim Reid <[email-masked]>
Date: Tue, 17 Jan 2012 23:06:18 +0000

On 17 Jan 2012, at 22:23, Jaap Akkerhuis wrote:

> If I remember well, the Dutch Data Protection Authority requires  
> that old (privacy related) data which is superfluous should be  
> removed after 7 years. Something like that.
>

Yup. IIRC it's the fifth principle of the EU Directive that Personal  
Data can't be kept or processed for longer than is necessary. This  
will be enacted in the national Data Protection law for each EU member  
state. The directive's principles may well underpin legislation in  
other parts of the world too.

So some sort of timer for contact objects could be useful. I would  
have thought though that contact objects would automagically get  
deleted from the registry whenever their reference count from domain  
objects and the like dropped to zero.

---

From: Andrew Sullivan <[email-masked]>
Date: Tue, 17 Jan 2012 20:25:49 -0500

> That leaves you with out of band, why is that better?

Perhaps because EPP is badly tailored for bulk operations.  I can
think of ways in which it might have been otherwise, but the WG
decided not to go that way.  Since the protocol is poorly adapted to
such operations, why try to force them in there?

Or, put it another way: EPP is by no means the only relationship
between an EPP repository operator and the clients: much of the system
depends on pre-existing relationships anyway.  Therefore, it seems at
least as good to use those other communication channels for bulk
operations as to add features to EPP for this.  I am not saying that
it is better or worse; but that there are other options, and before
one is going to change deployed systems to deal with new desired
behaviour, one needs an argument that alterations to those deployed
systems is the right approach to satisfy the desire.

A

-- 
Andrew Sullivan
[email-masked]

---

From: MICHAEL YOUNG <[email-masked]>
Date: Tue, 17 Jan 2012 19:53:01 -0600

I completely agree with the balance you are talking about. No one wants to
meddle lightly with deployed systems.  However, maintaining
standardization means (I think) keeping a protocol relevant to the use
cases that evolve.  We are talking about addressing bulk communications
around an EPP object in this case - so I think its fair enough to consider
short-comings in the protocol.  EPP probably should have had more focus
right from the start in addressing bulk operations.

However, that thought aside, I still think that one could probably address
the abandoned contact use case with an extension approach BUT I would
really want to see that particular extension be standardized.

There are always going to be 2 camps in these discussions, those who are
more than willing to update existing EPP systems with newer protocol
versions/features because they achieve a net benefit, and those whose
existing implementations do a good enough job and don't want to incur the
cost or anticipated risk.

It's a balance between the two positions, always, but I think its safe to
say we had better be prepared to work on the next iteration of EPP.  There
are a lot of new use cases coming down the pipe, as well as a backload of
ccTLD use cases.  If we don't address these, the protocol will eventually
become irrelevant and after all the good work done here, I think it would
be a shame to let it's usefulness erode.

Ok done preaching for tonight.

-M

---

From: Eric Brunner-Williams <[email-masked]>
Date: Tue, 17 Jan 2012 21:02:41 -0500

On 1/17/12 8:25 PM, Andrew Sullivan wrote:
> decided not to go that way.

Yup. One way was in the XRP drafts.

---

From: Ulrich Wisser <[email-masked]>
Date: Wed, 18 Jan 2012 09:23:29 +0100

On 17.01.12 17:19, Patrik Fältström wrote:
> On 17 jan 2012, at 09:12, Ulrich Wisser<[email-masked]>  wrote:
>
>> Hi,
>>

.SE puts message in the poll queue for any change done to the objects in 
the database. Transfer, expire, deactivate, delete, changes to status 
(e.g. SERVER_HOLD on ADR), ...

/Ulrich

---

From: InterNetX - Marco Schrieck <[email-masked]>
Date: Wed, 18 Jan 2012 09:59:54 +0100

Hi All,

At 18.01.12 09:23, worte Ulrich Wisser:
>
> .SE puts message in the poll queue for any change done to the objects
> in the database. Transfer, expire, deactivate, delete, changes to
> status (e.g. SERVER_HOLD on ADR), ...

We as registrar that have implemented many registry APIs must already
handle this. This is done by other registries in the same kind.

So there is only a map between this poll messages to existing method
like "contact deleted by registry". If we have an out of band
notification we have to manage extra credentials, protocol
implementations, montoring, etc... This is from the operative and
development view bad.

My opinion is using the poll queue is good. The registry should do the
GC periodically, to avoid many poll messages, maybe it can send many
deleted contacts via one poll message with a maximum on contacts per
message (eg 1000). Maybe there should also be a limit per day?

Btw. a kind of bulk operation is still existing with obj:check where it
is allowed to check multiple objects at once.

Marco

---

From: Christopher Browne <[email-masked]>
Date: Tue, 17 Jan 2012 17:26:01 -0500

"Dear Registrar:

---

From: Patrik Fältström <[email-masked]>
Date: Wed, 18 Jan 2012 01:33:23 +0100

> So a clause could be that any unlinked contact over a certain age will be deleted...

No, if you do not do a gc now and then, we might revoke your registrar accreditation. Or whatever. As I said, same as other contractual agreements between registries and registrars.

   Patrik

---

From: Ulrich Wisser <[email-masked]>
Date: Wed, 18 Jan 2012 11:12:48 +0100

James,

I believe I've been a bit to vague on my proposal. I would of course 
embed these answers in the <epp:resData/> part of the poll response. 
5730 specifies that this can be done with <obj:trnData/>, so 
<obj:creData/> and <obj:infData/> should be possible too. This way at 
least for create/update/renew/transfer should be covered by rfc 5730. 
The problem is to determine if the message is in response to a former 
request from the registrar or registry initiated. But that might not 
even be so important. In any case the registry is letting the registrar 
know that an object has been changed.

And yes, the usage of <obj:delete/> is "creative". Unfortunately is is 
no <obj:delData/> defined.  But I certainly can get

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<epp xmlns="urn:ietf:params:xml:ns:epp-1.0">
      <response>
        <result code="1301">
          <msg>Command completed successfully; ack to dequeue</msg>
        </result>
        <msgQ count="5" id="12345">
          <qDate>2000-06-08T22:00:00.0Z</qDate>
          <msg>Transfer requested.</msg>
        </msgQ>
        <resData>
           <contact:delete 
xmlns:contact="urn:ietf:params:xml:ns:contact-1.0">
             <contact:id>12345</contact:id>
           </contact:delete>
        </resData>
        <trID>
          <clTRID>ABC-12345</clTRID>
          <svTRID>54321-XYZ</svTRID>
        </trID>
      </response>
    </epp>

to validate. So maybe we take that as "allowed by rfc 5730".

/Ulrich

> On 1/17/12 3:12 AM, "Ulrich Wisser"<[email-masked]>  wrote:
>
>> Hi,
>>

---

From: Ulrich Wisser <[email-masked]>
Date: Wed, 18 Jan 2012 11:17:51 +0100

Sorry forgot the pointer to our extension:

https://www.iis.se/en/domaner/bli-registrar/epp

Here you find our extension, manual and some other EPP stuff.

The extensions defines some <iis:actionNotify/> tags with action in 
create/update/delete/transfer. Inside these tags we embed <obj:infData/> 
etc.

But I believe  that the extra tags are not needed.

/Ulrich

Am 17.01.12 15:01, schrieb Gould, James:
> [email-masked]<http://[email-masked]>
>
> 703-948-3271 (Office)
> 12061 Bluemont Way
> Reston, VA 20190
> VerisignInc.com
>
>
>
>
>
>
>

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 11:10:05 +0000

On 17/01/12 16:29, Patrik Fältström wrote:

> A registry can not know if an existing contact record is to be connected to a
> domain name that is to be created, as by definition contact records must exist
> with the registry that is not connected to any domain object.

Those registries that already do garbage collection (.fr comes to mind
immediately) have rules on when garbage collection happens. In their case,
it's six months of inactivity on the contact.

Besides, I'm proposing a (contact) garbage collection mechanism for the
benefit registries, not registrars. It has a secondary benefit for registrars
in that it gives a standard mechanism for those registries that wish to do
garbage collection to notify registrars of the pending deletion of object.

The data management needs of a registry can get quite unwieldy. Moreover,
certain countries have data retention laws that affect how long a registry
might be able to legally hold on to data that they don't actually have a
reason for retaining. In both cases, and those are far from the only ones,
it ought to be possible for a registry to perform garbage collection on
effectively dead unlinked objects, just so long as the registrar who
created them receives notification.

An open question is when the the registrar should receive notification. I
think the notification ought to be a notification that the objects are due
to be deleted, maybe the day before the object is due to be purged, as this
would give the registrar an opportunity to clean up on their end to avoid
situations such as the one you raised.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 11:13:58 +0000

> contacts around that haven't been associated for years?

AFNIC (.fr, &c.) do garbage collection already, albeit without notifications.
Their policy is that a contact that has been unlinked for six continuous
months is purged. Is a reasonable policy, although from a registrar's
perspective, a notification on the message queue would be nice.

> contacts in the first place?

I'm in full an total agreement, not to mention the legal implications as
far as data retention laws go.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 11:18:58 +0000

On 17/01/12 17:28, Patrik Fältström wrote:

> for them it is not a problem at all with many contact records and that
> it is worse if registries make commands that have implications in
> their database.

Keep in mind that this is actually a proposal coming from an employee of
a registrar!

> My suggestion was because of that that registrars make the gc command
> and registries report back what was deleted.

I personally would prefer it happen automatically as the registry feels is
necessary, but that the notification is sent out at least a day before the
purge occurs.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 11:31:05 +0000

>   simply deleted by the registry without any further fuss.

That's effectively what AFNIC do with their GC mechanism, but the lack of
notification is a bit of an irritation because it's easy for the registrar
to lose track of when the contact actually expires.

The notification--which can be ignored if the registrar wishes--means that the
registry's exact policy on when GC happens has no effect on the registrar,
which is very much advantageous.

I think it's best to think of *:check commands as putting a stay of execution
on the object, given they're essentially a signal that the registrar wishes
to use the object within a very short amount of time. If a registrar performs
a *:check and the registrar responds that said GC-able object exists, that
ought to guarantee that the the object is still around for enough time for it
to be used, maybe fifteen minutes.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 11:37:04 +0000

> created at any point with no relation to a domain.

Whether a registry supports it is entirely down to their policy, and they
can declare that they support the extension in their greeting document.

Whether an object is first class or not doesn't really effect whether it
may expire.

> I don't believe the
> so there is no hard dependency with being linked with a domain.

Then in that case, the registry need not support the extension, in which case
the objects will hang around in the registry's systems until they're
explicitly deleted.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 11:42:28 +0000

> You'd need an extension or a new contact mapping, though.  

I was the one who proposed the idea originally, and I discussed that. I think
the extension route is best, mainly because it allows the notifications to be
general and doesn't require the update on an existing RFC.

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 12:01:56 +0000

> for 10 years, I think we could suffer a few well thought out improvements

When it happens is up to the registry: they can make it as chatty or not as
they like. These would, after all, be batch notifications: it doesn't make
much sense to send a separate notification for each and every separate
(pending) deletion.

Using the queue need not be chatty so long as registries choose when they
send them in a suitable manner.

K.

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 12:11:24 +0000

On 18/01/12 10:12, Ulrich Wisser wrote:
> James,
> 
> I believe I've been a bit to vague on my proposal. I would of course embed
> these answers in the <epp:resData/> part of the poll response. 5730 specifies
> that this can be done with <obj:trnData/>, so <obj:creData/> and
> <obj:infData/> should be possible too. This way at least for
> create/update/renew/transfer should be covered by rfc 5730. The problem is to
> determine if the message is in response to a former request from the registrar
> or registry initiated. But that might not even be so important. In any case
> the registry is letting the registrar know that an object has been changed.
> 
> And yes, the usage of <obj:delete/> is "creative". Unfortunately is is no
> <obj:delData/> defined.

My one objection to that is that it doesn't really allow for batch
notifications: there's a separate notification sent out for each and every
object being garbage collected, which is chatty. Deletion notifications
are something, just like with availability checks, that are amenable to
batching, and, in fact, benefit from being batched, so batching is a win.

K.

---

From: "Hollenbeck, Scott" <[email-masked]>
Date: Wed, 18 Jan 2012 12:27:30 +0000

> Behalf Of MICHAEL YOUNG
> Sent: Tuesday, January 17, 2012 8:53 PM
> To: Andrew Sullivan; [email-masked]
> Subject: Re: [provreg] Changes we'd make to EPP
> 
> 

The use case for bulk operations *was* carefully considered - and not selected for inclusion in the core protocol. It's now 10 years later and no one has written an extension draft that describes that functionality. Maybe someone who feels the need should think about documenting an approach in an I-D.

I was going to suggest that text from the expired XRP draft(s) might be a good place to start, but I don't see the words "bulk" or "batch" anywhere in draft-brunner-xrp-00. Maybe Eric can explain how the concept was addressed.

On a slightly different topic, all this debate about contact synchronization is making me laugh. Be careful what you wish for, thick registry proponents! :)

Scott

---

From: Keith Gaughan <[email-masked]>
Date: Wed, 18 Jan 2012 12:42:22 +0000

On 18/01/12 12:27, Hollenbeck, Scott wrote:

> On a slightly different topic, all this debate about contact
> synchronization is making me laugh. Be careful what you wish for, thick
> registry proponents!

I dunno... I prefer the hell of thick registries over the hell other
registrar's WHOIS servers any day!

---

From: Eric Brunner-Williams <[email-masked]>
Date: Wed, 18 Jan 2012 08:15:03 -0500

> I was going to suggest that text from the expired XRP draft(s) might be a good place to start, but I don't see the words "bulk" or "batch" anywhere in draft-brunner-xrp-00. Maybe Eric can explain h…

My mistake. The idea didn't make it into the XRP draft, but into the
EPP Containter drafts.

   This memo documents how these generic object management operations
   are extended to hierarchies of objects, called container objects or

Brunner-Williams          Expires December 2002                 [Page 2]
^L
Internet-Draft                EPP Container                     May 2002

   containers.

Eric

---

From: Patrick Mevzek <[email-masked]>
Date: Thu, 19 Jan 2012 09:53:00 +0100

Le mercredi 18 janvier 2012 à 12:27 +0000, Hollenbeck, Scott a écrit :
> > EPP probably should have had more focus
> The use case for bulk operations *was* carefully considered - and not selected for inclusion in the core protocol. 

In my view it is done in part, since you can use multiple objects in
check operations.

Which also creates some nice interoperability issues since each registry
put a limit on the number of objets per check command...

-- 
Patrick Mevzek

---

From: Patrick Mevzek <[email-masked]>
Date: Thu, 19 Jan 2012 09:58:34 +0100

> to lose track of when the contact actually expires.

.FR do send notifications when contacts are garbage collected. There is
one notification per contact.

Also, when the registrant does not fulfill the charter, after some
specific out-of-EPP procedure, domain names may be blocked and later
deleted, both actions which are also notified, with one notification per
domain (the notification is basically some part of domain:infData
content)

-- 
Patrick Mevzek

---

From: Keith Gaughan <[email-masked]>
Date: Thu, 19 Jan 2012 11:58:06 +0000

> one notification per contact.

Sorry, yes, that's correct. I was also inaccurate about the length of time
before deletion, which is actually 90 days, with a 15-day grace period after
that. For reference in the discussion, the notification looks like this, more
or less:

<?xml version="1.0" encoding="UTF-8"?>
<epp xmlns="urn:ietf:params:xml:ns:epp-1.0">
  <response>
    <result code="1301">
      <msg>Command completed successfully; ack to dequeue</msg>
    </result>
    <msgQ count="1" id="1">
      <qDate>2012-01-19T11:50:00.0Z</qDate>
      <msg>Contact deletion completed</msg>
    </msgQ>
    <resData>
      <contact:infData
          xmlns="urn:ietf:params:xml:ns:contact-1.0">
        <!-- Original contact data -->
      </contact:infData>
    </resData>
    <extension>
      <frnic:ext xmlns:frnic="http://www.afnic.fr/xml/epp/frnic-1.0">
        <!-- Extended contact information -->
      </frnic:ext>
    </extension>
    <trID>
      <clTRID>ABC-123</clTRID>
      <svTRID>XYZ-123</clTRID>
    </trID>
  </response>
</epp>

Anyway, the overriding point is that AFNIC are an example of an existing
registry that already do object garbage collection.

My bad though. I'd their procedures mixed up in my head with another registry
that does (irregular, and unnotified) garbage collection.

K.