Subject: Re: [regext] Poll messages with unhandled namespaces (was Re: I-D Action: draft-ietf-regext-change-poll-07.txt)

From: Patrick Mevzek <[email-masked]>
Date: Mon, 16 Jul 2018 06:11:34 +0200

On Thu, Jun 14, 2018, at 11:22, Martin Casanova wrote:
> While implementing, another idea came to mind, which I want to put to
> discussion here:
> 
> <?xml version="1.0" encoding="UTF-8"?>
> <epp xmlns="urn:ietf:params:xml:ns:epp-1.0">
>   <response>
>     <result code="1301">
>       <msg lang="en">Command completed successfully; ack to dequeue</msg>
> 
>       <extValue>
>         <value>
>           <extURI>urn:ietf:params:xml:ns:changePoll-1.0</extURI>
>         </value>
>         <reason lang="en">Msg incomplete due to missing extURI at login
> cmd! To fix include at epp/command/login/svcs/svcExtension/extURI</reason>
>       </extValue>
>       <extValue>
>         <value>
>           <extURI>urn:ietf:params:xml:ns:secDNS-1.1</extURI>
>         </value>
>         <reason lang="en">Msg incomplete due to missing extURI at login
> cmd! To fix include at : epp/command/login/svcs/svcExtension/extURI</reason>
>       </extValue>

First on a technical level, replying to registrar basically "fix your client", is not really something nice, nor what registries should be doing.
Notifications are external to registrars, they can appear without any action triggered by them so basically you are filling their "inbox" with some external messages they may or may not care about, and then you are taxing them of being bad because they can not read all messages, and maybe they would not like to.

Some registrars may wish, for their own reasons, not to support some extensions and then at the same time you can not have one message in their queue that blocks all others after.

But more important, on a technical level, I believe the spirit of RFC5730 is that value/extValue element appears for *error messages*, that is all having code 2000 or more. Hence they would not fit for a 1301 return code.
 
Also note:
A <value> element that identifies a client-provided element
            (including XML tag and value) that caused a server error
            condition.

=> a client-provided element... the message you describe is in the response of a poll command by a client, and this poll command has none of that data you put in the <value> element of the server reply.

So while I see the underlying idea, I think you are bending RFC5730 quite a lot to achieve it.

-- 
  Patrick Mevzek
  [email-masked]

---

From: Patrick Mevzek <[email-masked]>
Date: Mon, 16 Jul 2018 06:20:37 +0200

On Thu, Jun 14, 2018, at 16:04, Gould, James wrote:
> This approach looks good to me.  It has the advantage of providing the 
> unhandled information in an element that is meant for machine processing 
> instead of using the <msgQ><msg> element that’s meant is meant to be 
> human readable.  The other advantage is that the contents of the <value> 
> element is not processed by the XML parser (e.g., 
> processContents=”skip”), meaning it would not cause an XML parser error.
> 
> This approach could include the entire unhandled extension block without 
> causing client-side parsing or unmarshalling issues.

This "could" should be a "must", otherwise a registrar has no way to just download the message for later consumption without having the need to login will all possible extensions. 

Again please take into account this example that exists today:
some registries restrict the extensions can be used on login, because some may be related to specific accredition, like secdns.
So some registrars may not even be able to put some extensions there, but may get notifications with messages using these exceptions, as they do not control what kind of messages they get and some may appear due to actions from other parties, like other registrars or the registry itself.

But like I said all of this still quite bends the RFC5730 spirit I think where value/extValue should be mostly for errors and value should reference a client provided element, which is not the case in these examples.

This latest idea from Martin and you is probably the best one we discussed about as of yet, and if I could get convinced to add myself on the consensus for it, I am still uneasy by how it uses RFC5730 structures.

-- 
  Patrick Mevzek
  [email-masked]

---

From: "Gould, James" <[email-masked]>
Date: Mon, 16 Jul 2018 10:58:47 +0000

Patrick,

In this case the server is communicating an error condition, but the condition does not result in an error code being returned.  A poll message really cannot fail, otherwise you get into the poison message scenario.  Even if we disagreed that this is an error condition, the RFC 5730 description does not use a normative SHOULD or MUST so it can be used in a 1301 response .  The <extValue> element can contain the entire extension XML along with a reason for each unhandled extension for later processing.  By inclusion of the unhandled XML in the <extValue> element, the client can get the entire poll message for acking, and there is no issue with returning extensions that may cause client-side XML parsing errors or marshaling errors.  
  
—
 
JG

James Gould
Distinguished Engineer
[email-masked]

703-948-3271
12061 Bluemont Way
Reston, VA 20190

Verisign.com <http://verisigninc.com/> 

On 7/16/18, 12:11 AM, "regext on behalf of Patrick Mevzek" <[email-masked] on behalf of [email-masked]> wrote:

    

    Also note:

    
    -- 
      Patrick Mevzek
      [email-masked]
    
    _______________________________________________
    regext mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/regext

---

From: "Gould, James" <[email-masked]>
Date: Mon, 16 Jul 2018 11:06:31 +0000

Patrick,

Yes, I believe the idea that Martin came up with to use the <extValue> element with the inclusion of the full unhandled XML block is the best option thus far.  It honors the client login services, it includes all of the XML for later processing, and it does not cause XML parsing failures or marshaling failures.  I implemented each of the discussed approaches using a stub server and a validating client, and this approach works best in my opinion.   
  
—
 
JG

James Gould
Distinguished Engineer
[email-masked]

703-948-3271
12061 Bluemont Way
Reston, VA 20190

On 7/16/18, 12:20 AM, "regext on behalf of Patrick Mevzek" <[email-masked] on behalf of [email-masked]> wrote:

    

    
    -- 
      Patrick Mevzek
      [email-masked]

---

From: Martin Casanova <[email-masked]>
Date: Mon, 16 Jul 2018 14:46:08 +0000

James, Patrick

Partly I understand Patrick's argument that the introduction of new types of poll messages could cause a problem for clients if their business logic is not prepared for it, even if technically the message can be received without any problem. Also our rule of restricting not enabled clients to  login with DNSSec has to fall. This rule will be obsolete with the CDS process anyway (RFC-7344 and RFC-8078)

The precondition of this approach is, that we actually can ask all registrars to prepare their clients to at least tolerate new poll messages and to update their business logic in order to process the newly given information properly if they wish to do so. I think this should be the case and no problem for most registrars already.

However we also considered to implement a server side flag to give registrars an out of band way  to “opt out” of receiving poll messages with certain extensions.  Also because we are still discussing if and how triggering of normally registry initialed messages by clients could be realized for integration testing of their business logic.
 
I think it should be good practice to have a process where new poll messages are allowed as per default, eventually with an optional mechanism to spare certain clients from receiving messages they actually don't care about, in order to drive the progress of using EPP extensions.

I will participate this afternoon remotely. See you soon.

Martin Casanova
________________________________________
Von: regext [[email-masked]]&quot; im Auftrag von &quot;Gould, James [jgould=[email-masked]]
Gesendet: Montag, 16. Juli 2018 13:06
An: Patrick Mevzek; [email-masked]
Betreff: Re: [regext] Poll messages with unhandled namespaces (was Re: I-D Action: draft-ietf-regext-change-poll-07.txt)

Patrick,

—

JG

James Gould
Distinguished Engineer
[email-masked]

703-948-3271
12061 Bluemont Way
Reston, VA 20190

    --
      Patrick Mevzek
      [email-masked]

---

From: Thomas Corte <[email-masked]>
Date: Mon, 16 Jul 2018 17:07:20 +0200

Hello,

On 7/16/18 16:46, Martin Casanova wrote:

> ... The precondition of this approach is, that we actually can ask all
> registrars to prepare their clients to at least tolerate new poll
> messages and to update their business logic in order to process the
> newly given information properly if they wish to do so. I think this
> should be the case and no problem for most registrars already.

There is a problem with this approach if the EPP client (like ours) uses
an XML parser that validates against the configured XML schema
definitions. Such a client will inevitably choke on any unknown
namespaces, since reading the message will fail on the parsing layer
already (without the business logic even getting a chance to look at the
message).
Note by the way that using a schema-validating EPP client is actually
beneficial since it allows the code consuming data from the registry's
responses to simply assume that certain data must be present and/or
follow a certain format, without the need for additional checks; i.e.
it's not just a case of being particularly pedantic.

To "tolerate new poll messages" in such a scenario would require
disabling the schema validation altogether, which would reintroduce the
need for addition checks in code that were previously handled by the
parsing layer.

I'm therefore much in favor of the previously discussed solution
involving the <extValue> element. It's not perfect, but it allows
schema-validating EPP clients to read and acknowledge unknown poll
messages which would otherwise block the queue.

Best regards,

Thomas

-- 
____________________________________________________________________
     |       |
     | knipp |            Knipp  Medien und Kommunikation GmbH
      -------                    Technologiepark
                                 Martin-Schmeißer-Weg 9
                                 44227 Dortmund
                                 Deutschland

     Dipl.-Informatiker          Tel:    +49 231 9703-0
     Thomas Corte                Fax:    +49 231 9703-200
     Stellvertretender Leiter    SIP:    [email-masked]
     Software-Entwicklung        E-Mail: [email-masked]

                                 Registereintrag:
                                 Amtsgericht Dortmund, HRB 13728

                                 Geschäftsführer:
                                 Dietmar Knipp, Elmar Knipp

---

From: Patrick Mevzek <[email-masked]>
Date: Mon, 16 Jul 2018 17:41:30 +0200

On Mon, Jul 16, 2018, at 16:46, Martin Casanova wrote:
> The precondition of this approach is, that we actually can ask all 

I am not so sure it would not be a problem, and if we take into account all
registries existing and hence all registrars, I am quite sure this will
not be possible in all cases.
Whatever we do, having a solution based on saying to the registrar "update your client
first" is not going to fare well.
I am even not sure many registries would really like to deploy something like
that, knowing that they will get push back from registrars.
 
> However we also considered to implement a server side flag to give 
> registrars an out of band way  to “opt out” of receiving poll messages 
> with certain extensions.  Also because we are still discussing if and 
> how triggering of normally registry initialed messages by clients could 
> be realized for integration testing of their business logic.

Indeed some registries allow registrars, through a website, to define which messages they want to receive. Not necessarily at the extension level but things like "outgoing transfer", "unlinked contact deletion", etc.

> I think it should be good practice to have a process where new poll 
> messages are allowed as per default,

The problem with that is that it could theoretically work for registrars if
1) they have access to documentation explaining all of these messages
2) they have access to it far in advance.

This does not always happen like that.
And otherwise all validating client will choke.

Imagine: a registry says, on day X we do a maintenance and after it we will create new notifications with namespace Y.
The idea is probably that just on day X all clients are updated immediately to login with namespace Y in order to receive notifications based on namespace Y.
We all know however that not all clients will upgrade at that time, but they may get such a notification. So the registry has to decide it what it does with the message using Y where the registrar did not log in with Y. Saying: "please relogin with Y before being able to poll your messages" is certainly not something registrars will be happy with.

> eventually with an optional 
> mechanism to spare certain clients from receiving messages they actually 
> don't care about, in order to drive the progress of using EPP 
> extensions.

This is indeed more pragmatic. But all this mechanism to define which messages to accept
will be outside the EPP protocol and this WG.

> I will participate this afternoon remotely. See you soon.

I will try to listen too, or be on Jabber.

-- 
  Patrick Mevzek
  [email-masked]

---

From: Patrick Mevzek <[email-masked]>
Date: Mon, 16 Jul 2018 18:02:46 +0200

> will be outside the EPP protocol and this WG.

But please also remember that if we want to tackle this problem in a generic way (and also taking care of different servers and clients strategies regarding handling of namespaces and inline/offline parsing and use) it is not limited to a single extension (the thread started long ago with changePoll) nor in fact limited to poll messages.

Imagine registrar A wanting to request a transfer from registrar B. In some registries it means that regitrar A can do a domain:info on the domain, with the authInfo to get access to all details, and specifically the contacts.
But a domain can have a secDNS part in the domain:info reply.
What happens if the registrar A did not login with the secDNS extension (maybe this case does not exist in gTLDs where DNSSEC is mandatory but again we have other registries cases to take into account)?

Should the domain:info return an error? Return everything as is? Return everything but the secDNS part?

The last case is the worst to me: some registrars may like not to support DNSSEC at all (and hence will not log in at all, or you have other cases where registries mandate specific tests to be "DNSSEC" accredited so it may not even be possible to log in with secDNS extension even if the registrar would like to) but, and especially for this, being able to detect beforehand if some client is trying to transfer to them a domain using DNSSEC, that they would like to refuse transferring.

Of course the above is only one example with a domain:info and the secDNS extension but I am sure we can find others.

This illustrates I think the distinction I made in earlier messages and the different semantic I attach to extensions listed as login: for me they are those that the client announce it will use. Of course, it has no control over messages or objects he is not the origin or the sponsor, all cases where other namespaces may appear.

-- 
  Patrick Mevzek

---

From: Martin Casanova <[email-masked]>
Date: Mon, 16 Jul 2018 16:44:23 +0000

@Thoma Corte: Sorry if my formulation was not precise enough:

“The precondition of this approach is, that we actually can ask all registrars to prepare their clients to at least tolerate new poll messages and to update their business logic in order to process the newly given information properly if they wish to do so.” 

With this statement I wanted to support the  <extValue> approach. No (validating) client will have a  technical problem receiving such a message regardless of what was configured at login. But as I was speaking to a registrar he told me: “So far you only sent me outgoing transfer poll messages and thats all my client is prepared for. ” So in our case we need to announce to registrars that they should build tolerant clients to be prepared to also receive unexpected poll messages with content in <extValue> eg. also low balance notifications in the future.. And that not every poll message will be automatically an outgoing transfer notification anymore.

@Patrick
In deed, for domain info requests on DNSSec domains we so far left out the DNSSec part in the domain info response if the client was not DNSSec enabled at login. In the future this content could also be transmitted in the <extValue>. To me it falls also under the “unhandled namespace” problem.

Martin Casanova
________________________________________
Von: regext [[email-masked]]&quot; im Auftrag von &quot;Patrick Mevzek [[email-masked]]
Gesendet: Montag, 16. Juli 2018 18:02
An: [email-masked]
Betreff: Re: [regext] Poll messages with unhandled namespaces (was Re: I-D Action: draft-ietf-regext-change-poll-07.txt)

--
  Patrick Mevzek