Subject: [regext] Interest in collaborating on an EPP over HTTP draft?

From: Anthony Eden <[email-masked]>
Date: Tue, 22 May 2018 12:09:31 -0400

Greetings folks!

I've thrown together a repo over at GitHub to work on an EPP over HTTP
draft (https://github.com/aeden/epp-over-http). I'd love to know if there
are others from the community who are interested on collaborating. As a
registrar, we'd love to be able to work with registries using HTTP as the
transport protocol in the future.

Note that the current version of this draft deals solely with EPP over HTTP
1.1, it does not consider HTTP 2 at this time.

Feedback is welcome. Thanks!

Sincerely,
Anthony Eden

-- 
DNSimple.com
http://dnsimple.com/
Twitter: @dnsimple

---

From: Patrick Mevzek <[email-masked]>
Date: Wed, 23 May 2018 03:29:38 +0200

Hi Anthony,

On Tue, May 22, 2018, at 18:09, Anthony Eden wrote:
> are others from the community who are interested on collaborating.

As I am sure you are aware, some registries currently uses HTTPS, like .IT
and .PL at least.
It may be a good idea, if not already, to try sharing discussions with them and see if you can converge on something, if you are planning to do a standard.

You may also be aware that when EPP itself was drafted they were then multiple other proposals for transport, besides TLS. There was at least SMTP if I recall correctly and BXXP (BEEP) which I think does not really exist anymore but it looks like to me that many of its features are also present nowadays in HTTP/2.

Maybe there are some ideas to grab from these past attempts, the documents themselves or the discussions.

> As a

I am curious, why particularly prefering HTTP over TCP (or more precisely HTTPS over TLS)?

Did you tak time already to document the differences with TCP, in the realm of EPP, what are the benefits and the drawbacks?
 
> 1.1, it does not consider HTTP 2 at this time.

(I did not look at your code/document yet).
Why is it so?
Just a lack of time or some specific reason against HTTP/2?

If there anything that works with HTTP/1.1 but not /2 it should be documented.
As nowadays, for new works, I think it is best to concentrate on the newer
versions of other standards when you do a layering (hence HTTP/2) instead 
of forcing retro-compatibilies, except if good reasons for that of course, this is what I am curious about. 

-- 
  Patrick Mevzek

---

From: Mario Loffredo <[email-masked]>
Date: Wed, 23 May 2018 10:08:10 +0200

Hi Anthony and Patrick,

I'm interested in collaborating.
I speak on behalf of .it but I'm pretty sure that .pl implements EPP 
over HTTP in the same way we do.
At .it, we have not implemented the EPP Login based on HTTP basic 
authentication, we have left it as is to keep EPP as much as possible 
independent from the trasport layer.
In my opinion, storing the Login information in the request header is 
wrong, because you mix EPP with HTTP information.  Defintively, you 
cannot parse the Login request in the same way of the other requests.

It seems to me that your draft does not consider how to map an EPP 
session over HTTP.
Session tracking over HTTP basically requires that a session ID is 
maintained across multiple requests to the server.
A successfully Login returns a session ID in the response cookies that 
the client is requested to use in the subsequent requests belonging to 
the same EPP session until the session is closed
by Logout or by server timeout. In our implementation, sessions are 
maintained outside the application server so we can easily add new 
machines in a load-balanced architecture.

The Hello request can be used inside a session or outside. Therefore, 
Hello and Login are the only requests not containing  a session ID (the 
Hello may not, the Login must not). Any other request which does not 
contain a session ID as well as any other request containing an invalid 
or unknown session ID is rejected with an error.

EPP errors are returned as part of the response body.

We use HTTPS but security is enforced by allowing the access to our live 
servers only to some IP addresses each registrar has previously defined 
in an appropriate portal.

With regards to multipart messages support, honestly, I must say that we 
have never dealt with this case. The length of EPP requests is limited 
and also the responses, even if they are generally longer than the 
requests, don't need a multipart message.

Regards,
Mario

-- 
Dr. Mario Loffredo
Servizi Internet e Sviluppo Tecnologico
CNR - Istituto di Informatica e Telematica
via G. Moruzzi 1, I-56124 PISA, Italy
E-Mail: [email-masked]
Phone: +39 050 3153497
Web: http://www.iit.cnr.it/mario.loffredo

---

From: Pieter Vandepitte <[email-masked]>
Date: Wed, 23 May 2018 10:28:57 +0200

Hi Anthony,

Exactly the same question as Patrick. As I'm not in the registrar world, but the registry world, I'm very interested in the use cases (or why) of EPP over HTTP instead of EPP over TCP. But very willing to collaborate if there's a case (not just because it is technically more fun to do)

Kind regards

Pieter

> On 23 May 2018, at 03:29, Patrick Mevzek <[email-masked]> wrote:
> 
> Hi Anthony,
> 
> _______________________________________________
> regext mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/regext

---

From: Anthony Eden <[email-masked]>
Date: Wed, 23 May 2018 07:46:28 -0400

Patrick,

Thanks for your thoughts and questions, I do appreciate it. Response
inline...

On Tue, May 22, 2018 at 9:29 PM, Patrick Mevzek <[email-masked]> wrote:

> It may be a good idea, if not already, to try sharing discussions with
> them and see if you can converge on something, if you are planning to do a
> standard.
>

Thanks for the tip. We're just getting into direct integration with
registries, even though we've had our ICANN credentials for quite some
time, thus I was not aware that .IT and .PL were using HTTP already. I
agree that having their input as they're already operating EPP over HTTP
will be good for any successful standard.

>
> You may also be aware that when EPP itself was drafted they were then
> multiple other proposals for transport, besides TLS. There was at least
> SMTP if I recall correctly and BXXP (BEEP) which I think does not really
> exist anymore but it looks like to me that many of its features are also
> present nowadays in HTTP/2.
>

I do see mentions of SMTP and BEEP in RFC 5730 in the transport discussion,
but none of them ever became standards, right?

> themselves or the discussions.

I'll see if I can dig up the drafts or the discussions around them.

> Did you tak time already to document the differences with TCP, in the
> realm of EPP, what are the benefits and the drawbacks?
>

I mention briefly in the introduction of the draft, however my goals with
this draft would be to make it registry implementation easier for small
registries who currently only provide a very small number of concurrent TCP
connections. Stateless protocols should be easier for a small registry to
work with as requests can be distributed across multiple servers without
the need for session tracking.

Additionally I think HTTP would be easier for new registrars to start with
because of existing client-side tooling and libraries.

As for the drawbacks, to take advantage of HTTP 1.1's stateless nature, the
specific requirement of EPP being stateful will need to be relaxed. Also,
there is additional data (headers) that must be transported with HTTP
requests vs. EPP over TCP which increases the amount of bandwidth used per
request.

> Just a lack of time or some specific reason against HTTP/2?

I have nothing against an HTTP/2 implementation, I just don't see it as
having the same benefits as HTTP/1.1 given that HTTP/2 uses a single TCP
connection and multiplexing, effectively negating the stateless benefits of
HTTP/1.1.

-- 
DNSimple.com
http://dnsimple.com/
Twitter: @dnsimple

---

From: "Gould, James" <[email-masked]>
Date: Wed, 23 May 2018 11:57:46 +0000

Anthony,

I reviewed your draft.  We implemented EPP over HTTP a while back, where HTTP was treated as transport in line with EPP over TCP (RFC 5734).  It was shut down when it became clear that EPP over TCP was the preferred option.  The difference with your draft and our implementation is that the draft defines a stateless transport and our implementation was a stateful transport.  We simply framed the EPP packets using HTTP, with HTTP keep-alive, and with full support for the EPP login and logout commands.   Changing from stateful to stateless would really not allow easily replacing the transport on the client-side or the server-side.  We used the following HTTP properties:

  1.  Content length is explicitly specified
  2.  Chunked transfer encoding of HTTP 1.1
  3.  Content-type set to “text/xml”

The 2-way HTTPS connection would be established, the client packet is framed as an HTTP packet and sent to the server, where the server processed the HTTP packet as it would a framed packet with EPP over TCP.  All of the EPP over HTTP commands and responses would behave consistently with EPP over TCP.

—

JG

James Gould
Distinguished Engineer
[email-masked]

703-948-3271
12061 Bluemont Way
Reston, VA 20190

Verisign.com<http://verisigninc.com/>

From: regext <[email-masked]> on behalf of Anthony Eden <anthony.eden=[email-masked]>
Date: Tuesday, May 22, 2018 at 12:09 PM
To: "[email-masked]" <[email-masked]>
Subject: [EXTERNAL] [regext] Interest in collaborating on an EPP over HTTP draft?

Greetings folks!

I've thrown together a repo over at GitHub to work on an EPP over HTTP draft (https://github.com/aeden/epp-over-http). I'd love to know if there are others from the community who are interested on collaborating. As a registrar, we'd love to be able to work with registries using HTTP as the transport protocol in the future.

Note that the current version of this draft deals solely with EPP over HTTP 1.1, it does not consider HTTP 2 at this time.

Sincerely,
Anthony Eden

--
DNSimple.com
http://dnsimple.com/
Twitter: @dnsimple

---

From: Anthony Eden <[email-masked]>
Date: Wed, 23 May 2018 07:58:49 -0400

Mario,

Thank you for your feedback. I'd love to have your direct input on the
draft if and when you have the time (and if it makes sense).

On Wed, May 23, 2018 at 4:08 AM, Mario Loffredo <[email-masked]>
wrote:

> I speak on behalf of .it but I'm pretty sure that .pl implements EPP over
> HTTP in the same way we do.
> wrong, because you mix EPP with HTTP information.  Defintively, you cannot
> parse the Login request in the same way of the other requests.
>
> It seems to me that your draft does not consider how to map an EPP session
> over HTTP.
>

The draft considers each request to be stateful in its own right. Thus
clients are effectively making each request its own session. It's a loose
interpretation of RFC5730's protocol requirements, but I still think it
fits.  Supporting multipart allows for sequences of related frames to be
processed without making a new HTTP request for each.

> A successfully Login returns a session ID in the response cookies that the
> client is requested to use in the subsequent requests belonging to the same
> EPP session until the session is closed
> maintained outside the application server so we can easily add new machines
> in a load-balanced architecture.
>

It sounds like an HTTP/2 implementation could help you significantly given
its multiplexing over a single connection.

> contain a session ID as well as any other request containing an invalid or
> unknown session ID is rejected with an error.
>

The draft indicates this as well.

> servers only to some IP addresses each registrar has previously defined in
> an appropriate portal.
>

Totally reasonable. Defense in depth and all that.

> have never dealt with this case. The length of EPP requests is limited and
> also the responses, even if they are generally longer than the requests,
> don't need a multipart message.
>

The purpose of multipart is to allow related EPP frames to be sent in a
single HTTP request.

-- 
DNSimple.com
http://dnsimple.com/
Twitter: @dnsimple

---

From: Mario Loffredo <[email-masked]>
Date: Wed, 23 May 2018 14:17:28 +0200

Hi Anthony,

I would be pleased to collaborate directly.
Looking back to my previous response, thare are many other issues coming 
in my mind that I would like to discuss.
At a first sight, .it implementation seems basically the same as 
described by James.
I don't want to be too verbose now but I can also explain why we decided 
to use HTTP instead of TCP.

Regards,
Mario

Il 23/05/2018 13:58, Anthony Eden ha scritto:
> On Wed, May 23, 2018 at 4:08 AM, Mario Loffredo 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Hi Anthony and Patrick,
>
>     I speak on behalf of .it but I'm pretty sure that .pl implements
>     EPP over HTTP in the same way we do.
>     authentication, we have left it as is to keep EPP as much as
>     possible independent from the trasport layer.
>     In my opinion, storing the Login information in the request header
>     is wrong, because you mix EPP with HTTP information.  Defintively,
>     you cannot parse the Login request in the same way of the other
>     requests.
>
> clients are effectively making each request its own session. It's a 
> loose interpretation of RFC5730's protocol requirements, but I still 
> think it fits.  Supporting multipart allows for sequences of related 
> frames to be processed without making a new HTTP request for each.
>
>     A successfully Login returns a session ID in the response cookies
>     that the client is requested to use in the subsequent requests
>     belonging to the same EPP session until the session is closed
>     by Logout or by server timeout. In our implementation, sessions
>     are maintained outside the application server so we can easily add
>     new machines in a load-balanced architecture.
>
>
> It sounds like an HTTP/2 implementation could help you significantly 
> given its multiplexing over a single connection.
>
>
>     The Hello request can be used inside a session or outside.
>     Therefore, Hello and Login are the only requests not containing  a
>     session ID (the Hello may not, the Login must not). Any other
>     request which does not contain a session ID as well as any other
>     request containing an invalid or unknown session ID is rejected
>     with an error.
>
>     We use HTTPS but security is enforced by allowing the access to
>     our live servers only to some IP addresses each registrar has
>     previously defined in an appropriate portal.
>
>
> Totally reasonable.. Defense in depth and all that.
>
>
>     With regards to multipart messages support, honestly, I must say
>     that we have never dealt with this case. The length of EPP
>     requests is limited and also the responses, even if they are
>     generally longer than the requests, don't need a multipart message.
>
>
> The purpose of multipart is to allow related EPP frames to be sent in 
> a single HTTP request.
>
>
>     Regards,
>     Mario
>
>
>
>             draft (https://github.com/aeden/epp-over-http
>             <https://github.com/aeden/epp-over-http>). I'd love to
>             know if there
>         It may be a good idea, if not already, to try sharing
>         discussions with them and see if you can converge on
>         something, if you are planning to do a standard.
>
>         You may also be aware that when EPP itself was drafted they
>         were then multiple other proposals for transport, besides TLS.
>         There was at least SMTP if I recall correctly and BXXP (BEEP)
>         which I think does not really exist anymore but it looks like
>         to me that many of its features are also present nowadays in
>         HTTP/2.
>
>         Maybe there are some ideas to grab from these past attempts,
>         the documents themselves or the discussions.
>
>             As a
>         Did you tak time already to document the differences with TCP,
>         in the realm of EPP, what are the benefits and the drawbacks?
>
>         of forcing retro-compatibilies, except if good reasons for
>         that of course, this is what I am curious about.
>
>
>     -- 
>     Dr. Mario Loffredo
>     E-Mail: [email-masked] <mailto:[email-masked]>
>     Phone: +39 050 3153497
>     <http://www.iit.cnr.it/mario.loffredo>
>
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/regext>
>
>
>
>
> -- 
> DNSimple.com
> http://dnsimple.com/
> Twitter: @dnsimple
>
>

---

From: Anthony Eden <[email-masked]>
Date: Wed, 23 May 2018 10:32:34 -0400

On Wed, May 23, 2018 at 7:57 AM, Gould, James <jgould=40verisign.com@dmarc.
ietf.org> wrote:

> Anthony,
>
>
>
> I reviewed your draft.  We implemented EPP over HTTP a while back, where
> HTTP was treated as transport in line with EPP over TCP (RFC 5734).  It was
> shut down when it became clear that EPP over TCP was the preferred option.
> The difference with your draft and our implementation is that the draft
> defines a stateless transport and our implementation was a stateful
> transport.  We simply framed the EPP packets using HTTP, with HTTP
> keep-alive, and with full support for the EPP login and logout commands.
> Changing from stateful to stateless would really not allow easily replacing
> the transport on the client-side or the server-side.  We used the following
> HTTP properties:
>
>
>
> The 2-way HTTPS connection would be established, the client packet is
> framed as an HTTP packet and sent to the server, where the server processed
> the HTTP packet as it would a framed packet with EPP over TCP.  All of the
> EPP over HTTP commands and responses would behave consistently with EPP
> over TCP.
>

Thanks for the details James. I agree that existing EPP clients and servers
that depend on the statefulness of the transport would not be able to use
stateless HTTP. I envision this as more of an alternative for smaller
registries that currently limit their concurrent connections to extremely
low values (as low as max 2 concurrent connections in some cases, as far as
I understand it). Larger registries would probably not see an advantage in
using HTTP 1.1, although might receive benefits from HTTP/2 (I have not
investigated this fully).

Sincerely,
Anthony Eden

-- 
DNSimple.com
http://dnsimple.com/
Twitter: @dnsimple

---

From: Patrick Mevzek <[email-masked]>
Date: Thu, 24 May 2018 07:31:26 +0200

> but none of them ever became standards, right?

You are correct, after all discussions, only EPP over TLS was defined as transport.

I was able to find the following documents describing various transports for EPP:
* EPP over BXXP: https://tools.ietf.org/html/draft-ietf-provreg-epp-beep-02
* EPP over REST: https://tools.ietf.org/html/draft-wullink-restful-epp-00
(so again over HTTP too)
* EPP over SOAP: https://tools.ietf.org/html/draft-liu-epp-soap-00
(some kind of HTTP also, even if SOAP could be layered on top of various transports)
* EPP over SMTP: https://tools.ietf.org/html/draft-brunner-epp-smtp-00

They all have sections on sessions, stateless vs stateful, etc.
 
As for discussions, this page:
http://www.cafax.se/ietf-provreg/maillist/2001-08/threads.html
as links to more than one email about messages and transports.

> the need for session tracking.

My experience does not match this, so I may lack some examples/points,
and I would like to learn more.

I guess it all depends what you define as big or small registries.

Here are some facts I know about:

* for the one I know using EPP over HTTPS they are not exactly registries I would call small: both .IT and .PL are well over 2 millions of domain names. I am sure they would have been smaller when they started EPP over HTTPS and while I lack hard facts on this, I am sure they already had hundreds of thousands of domains then
* I know at least one ccTLD registry even bigger than .IT or .PL, where everything (all TLS connections) are handled by one server
* I think big/small is here not really related to the number of domain names but more the number of registrars, and the number of connections allowed to each of them (connections or sessions, depending how much state you have).
I believe that a small registry could be something with dozens of registrars maybe in the small hundreds number, that is all, and for the number of connections I guess there would be no need of more than 1 or 2 per registrar.
What I want to say here is that alltogether I think it makes a low number of connections to handle.
* all registries impose timeouts on their connections, typically in the 15 minutes average, and some of them apply also hard timeouts, hence releasing a connection after a given time whatever happens.

What is true however is that nowadays with many services using the "cloud",
no other port than 443 may be exposed to the outside world, or maybe no "lower" ports. In the second case, even if 700 is the standard EPP port, any other one could be used.

Also, Nomulus, the Google open source EPP based registry implementation
had the same kind of problems for a long time, quoting their documentation:
GAE Proxy: App Engine Standard only serves HTTP/S traffic. A proxy to forward traffic on EPP and WHOIS ports to App Engine via HTTPS is provided.
(first releases did not have this proxy)

So maybe the problem is more related on where/how the application servers run?

> because of existing client-side tooling and libraries.

I am not sure to understand this point. You have two parts, the transport and the protocol. The protocol is EPP, based on XML. All languages have full support of XML. You may want to have an EPP library for some abstraction over XML but in any case it would be the same library used for generating/parsing the EPP frames, whatever transport you are using, so HTTPS or TLS that does not change that point I believe.
And all languages also have full support for TLS (HTTPS needs it obviously), real problems are more around certificates management and algorithms configuration, which will be exactly the same at the TLS or HTTPS level.

> requests vs. EPP over TCP which increases the amount of bandwidth used per

I believe anyway that even without adding anything, HTTP as a layer adds data/metadata and hence the messages are bigger.

For example, the length of the message will need as many bytes as
"Content-Length: " + size where in TCP there would be just the size.

However I would not think too much about bandwidth use, as XML is already not a very light serialization mechanism.
 
> connection and multiplexing, effectively negating the stateless benefits of

Ok, I can see that. But I think it would still be strange to define a new standard today saying that you need to use HTTP/1 with it and not the newer HTTP/2.

And at least in the gTLD world, things could probably not implemented before existing as an RFC. And some gTLDs are very small. This is not technical related but may need to be taken into account if you wish for large adoption.

-- 
  Patrick Mevzek

---

From: Justin Mack <[email-masked]>
Date: Tue, 29 May 2018 11:40:52 -0600

To add to the list of registries that use EPP over HTTP, we know of:
* .IT
* .PL
* .ES

.NZ used to have an EPP-like XML protocol that used PGP signed XML 
messages over HTTPS, but they since changed to regular EPP.

.JP uses a custom protocol purely over HTTPS.

Just about everyone else uses EPP 0.4 or EPP 1.0, with notable 
exceptions of RRI (.DE) and TMCH for the Trademark Clearinghouse.

EPP can function statelessly, but we need to consider the instant 
availability of created objects to other "stateless sessions". For 
example, creating a Contact or Host object needs to be immediately 
available upon the next (stateless) request.

Justin Mack
MarkMonitor

---

From: Patrick Mevzek <[email-masked]>
Date: Wed, 30 May 2018 04:18:10 +0200

> exceptions of RRI (.DE) and TMCH for the Trademark Clearinghouse.

If you want to list everything that looks like EPP but is not EPP,
you could list .IE too.

However I think this is unrelated: the protocol and the transport should be orthogonal.

The protocol should specify some properties it needs (RFC 5730 for EPP, section 2.1 that should probably be the starting point of all endeavours to define new transports for EPP), and then it should work with any transport having those properties. Both "parts" should be able to evolve/be swapped independently as long as the contract (the common set of properties agreed upon) remains valid.

As for EPP, each new transport should have a specification like it is done in RFC 5734 for TLS.
 
-- 
  Patrick Mevzek

---

From: Ryan Finnesey <[email-masked]>
Date: Thu, 31 May 2018 03:13:31 +0000

It is interesting  to see the discussion about EPP over HTTP.  Right now I am working on a greenfield development project where we will need to connect to the majority of Registries.  Being that this is a greenfield project we have been  keen to architect the system that will interact between the Registries and the Registrar to run within the public cloud.  Azure and or AWS.  We had a discussion the other day about using Azure Functions but had to quickly rule it out because EPP requirement on TCP Sockets.   Azure Functions only support http as a protocol .

Has there been any conversation about the Registries using  REST?  

Cheers
Ryan

-----Original Message-----
From: regext [mailto:[email-masked]] On Behalf Of Patrick Mevzek
Sent: Tuesday, May 29, 2018 10:18 PM
To: [email-masked]
Subject: Re: [regext] Interest in collaborating on an EPP over HTTP draft?

> exceptions of RRI (.DE) and TMCH for the Trademark Clearinghouse.

If you want to list everything that looks like EPP but is not EPP, you could list .IE too.

 
--
  Patrick Mevzek

---

From: Ulrich Wisser <[email-masked]>
Date: Thu, 31 May 2018 15:17:47 +0200

Hi!

In the discussion on http/1.1 or http/2, I would like to mention, that
registries have a special kind of customers, the snapback registrars. An
unfiltered http/2 would allow them to use a lot of resources on the
registry side. And these people really do try to get it all. That is not
really what we want. And I don't think these properties of http/2 are very
well understood currently.

/Ulrich

Ulrich Wisser
[email-masked]