Subject: [Http-srv] Alternative to SRV?

From: Ray Bellis <[email-masked]>
Date: Tue, 21 Aug 2018 19:39:09 +0100

Thanks Adam, for creating the mailing list.

As mentioned at the side-meeting in Montreal, I strongly believe that 
the way forward should be a new RR that is specific for the use of 
HTTP(s) (c.f. MX for SMTP) and that would be automatically looked up by 
recursive resolvers and returned in answers [*]

This recursive lookup step would give this record the equivalent 
performance to CNAME whilst avoiding the complexities (and failings) of 
SRV or ANAME.

I'd like to write that up in a draft, but to do so I'd like to co-author 
  with HTTP specialists to ensure that any such RR has the fields they 
deem necessary without the extra ones that SRV has that we heard are not 
desirable (specifically port numbers and load-balancing / weighting).

Who'd be up for helping with this?

Ray Bellis
ISC Research Fellow

[*] one caveat - the look-up would have to be optional in the 
specification because making it mandatory would prevent the use of the 
RR Expert Review process which doesn't allow for the assignment of
RRs with mandatory server side processing.

---

From: Tony Finch <[email-masked]>
Date: Tue, 21 Aug 2018 20:40:49 +0100

Ray Bellis <[email-masked]> wrote:
>
> As mentioned at the side-meeting in Montreal, I strongly believe that the way
> forward should be a new RR that is specific for the use of HTTP(s) (c.f. MX
> for SMTP) and that would be automatically looked up by recursive resolvers and
> returned in answers [*]

What will the backwards compatibility story be?

In the early days, what advantages will there be for DNS operators that
will make them want to use this record?

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Bailey: Northwest 5 or 6, backing west 5 to 7. Moderate or rough. Showers.
Good.

---

From: Ray Bellis <[email-masked]>
Date: Tue, 21 Aug 2018 21:52:01 +0100

> What will the backwards compatibility story be?

A / AAAA lookup, in parallel.

In the short / medium term the downside would be having to wait for all 
three lookups to complete before proceeding (absent a happy eyeballs 
scenario)

> will make them want to use this record?

The primary one is the ability to put them at the zone apex (or indeed 
alongside any other record that would currently be precluded by CNAME 
rules).

Ray

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Aug 2018 09:01:47 +1000

> On 22 Aug 2018, at 5:40 am, Tony Finch <[email-masked]> wrote:
> 

You look for all 3 record types simultaneously.  If you get back NODATA for the
HTTP record you use the A and AAAA responses.  If you get back data for the HTTP
record and ignore the A and AAAA responses.

If the HTTP lookup fails (NODATA is not a failure) you MUST NOT use the A and AAAA
responses.

Site operators decide when to stop publishing A and AAAA record for old clients
where they are using HTTP like they no longer publish A and AAAA records for SMTP
but instead rely solely on the MX record at the zone apex.  For MX this took 10
to 15 years as it took a long time to replace MTAs.  Browser vendors have shown
that they can replace the client side in less that 3 years.

> will make them want to use this record?

If the browser vendors can push out updates quickly enough it will just be a working
record.  The advantage will be to site owners in that they no longer have to move
their DNS to their CDN provider, they can just add the HTTP record to point at the
service name their CDN provider gives them.

> Http-srv mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/http-srv

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742              INTERNET: [email-masked]

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Aug 2018 09:39:12 +1000

> On 22 Aug 2018, at 4:39 am, Ray Bellis <[email-masked]> wrote:
> 
> As mentioned at the side-meeting in Montreal, I strongly believe that the way forward should be a new RR that is specific for the use of HTTP(s) (c.f. MX for SMTP) and that would be automatically looked up by recursive resolvers and returned in answers [*]
> 
> This recursive lookup step would give this record the equivalent performance to CNAME whilst avoiding the complexities (and failings) of SRV or ANAME.
> 
> I'd like to write that up in a draft, but to do so I'd like to co-author  with HTTP specialists to ensure that any such RR has the fields they deem necessary without the extra ones that SRV has that we heard are not desirable (specifically port numbers and load-balancing / weighting).

Only the port field is a real potential issue with HTTP.  The others can be set appropriately.
I wish we had said port==0 implies use the default/specified port.   We could still say this for
_http._tcp and _https._tcp.  Part of the reason SRV doesn’t automatically cover a protocol but
requires that SRV use be specified for a protocol is to allow for changes like this.

> [*] one caveat - the look-up would have to be optional in the specification because making it mandatory would prevent the use of the RR Expert Review process which doesn't allow for the assignment of

---

From: Martin Thomson <[email-masked]>
Date: Wed, 22 Aug 2018 10:44:44 +1000

I think that you are free to pretend that HTTPS is the only variant of
the protocol.  There's no point in shipping improvements for the
cleartext variant.

What are you thinking about ALPN?

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Aug 2018 12:56:42 +1000

> On 22 Aug 2018, at 10:44 am, Martin Thomson <[email-masked]> wrote:
> 

We have to make stuff work for http as well as https.  There are still sites
that want to use http at the zone’s name where CNAME doesn’t work.  This is a
CURRENT problem.

> What are you thinking about ALPN?

What are you thinking about?  The default would be the status quo.  I would
like to be able to deploy this without requiring servers (excludes proxies)
to be updated.

Mark

---

From: Martin Thomson <[email-masked]>
Date: Wed, 22 Aug 2018 13:25:42 +1000

> that want to use http at the zone’s name where CNAME doesn’t work.  This is a

My point is that we don't have to fix http.  It's a broken protocol
that we support for backwards-compatibility reasons.

> like to be able to deploy this without requiring servers (excludes proxies)

If you take a narrow view and say that we just need A++ or AAAA++, we
get a design that has no real incentive to deploy.  But if we were to
make it easier to deploy QUIC (for example), that might be a valuable
inducement.

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Aug 2018 14:45:38 +1000

> On 22 Aug 2018, at 1:25 pm, Martin Thomson <[email-masked]> wrote:
> 

We are fixing http we are fixing lack of service to server mapping for HTTP
and HTTPS that will work *everywhere* in the DNS namespace.

> make it easier to deploy QUIC (for example), that might be a valuable

QUIC is a pseudo transport.  _https._quic

I’m not going to object to doing something more but too much feature creep
will make this a long process.  If you want DNS servers to fill in the additional
section make the record well structured.  Conversion from txt to wire format is
time consuming.  Nameservers work in wire format.

I suspect the starting point of a new record would be a PTR/CNAME analog.

		<site> HTTP <server>

If you want to add alternate transports

		<site> HTTP <transport> <server>

then you add “A and AAAA records SHOULD retrieved and added to the additional
section prior to returning the response to the HTTP query”.

Mark

---

From: Daniel Stenberg <[email-masked]>
Date: Wed, 22 Aug 2018 07:55:18 +0200 (CEST)

On Wed, 22 Aug 2018, Mark Andrews wrote:

> For MX this took 10 to 15 years as it took a long time to replace MTAs. 
> Browser vendors have shown that they can replace the client side in less 
> that 3 years.

HTTP(S) is used way more widely than just the major browsers, and 
unfortuantely most of the non-browser usage is not as agile when it comes to 
upgrades...

-- 

  / daniel.haxx.se

---

From: Mark Andrews <[email-masked]>
Date: Wed, 22 Aug 2018 16:28:08 +1000

> On 22 Aug 2018, at 3:55 pm, Daniel Stenberg <[email-masked]> wrote:
> 
>> For MX this took 10 to 15 years as it took a long time to replace MTAs. Browser vendors have shown that they can replace the client side in less that 3 years.
> 
> HTTP(S) is used way more widely than just the major browsers, and unfortuantely most of the non-browser usage is not as agile when it comes to upgrades…

Too many people in the IETF are really too impatient when it comes to change.

TXT->SPF was well underway when it got abandoned because quite frankly people were too impatient and then lied about the terms of the experiment to get it stopped.  The experiment was to compare
the protocol SPF vs DKIM.  Conversion from TXT to SPF DNS records was just something that needed to be done for good DNS hygiene.  It was never part of the terms of the experiment.  It would have been trivial to push up the SPF record presence from day one if we had know that the terms of the experiment were going to be changed at evaluation time.

We really don’t need to worry about how long it takes for the last piece of code to be replaced takes.  People will make their own decisions about when it is safe to stop publishing A and AAAA records for HTTP/HTTPS.  If you really want to measure take up you point the SRV record at servers
with different addresses.  Addresses are cheap.

---

From: Ray Bellis <[email-masked]>
Date: Wed, 22 Aug 2018 19:28:09 +0100

> section prior to returning the response to the HTTP query”.

Yes, that's pretty much my expectation, albeit I suspect we'd need the 
latter with a <transport> field, since I do anticipate that there will 
be a need to have different target addresses for HTTP vs HTTP(s) vs 
HTTP-over-FOO, and where the <tranport> field also provides information 
to the browser of which transport to actually try.

I would probably suggest that <transport> should be a (short) text field 
to allow for agility of transport names rather than using a numerical 
registry that might require DNS server upgrades any time a new transport 
is devised.

Ray

---

From: Tony Finch <[email-masked]>
Date: Thu, 23 Aug 2018 14:15:14 +0100

> A / AAAA lookup, in parallel.

Sorry, I meant on the publisher side.

> alongside any other record that would currently be precluded by CNAME rules).

So the plan is that I'll have to get the target names as well as the
target addresses, instead of just the target name (aname / cname) or just
the target addresses. This is not very enticing.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
North Fitzroy: Northwesterly 4 or 5. Slight or moderate. Showers. Good,
occasionally moderate at first.

---

From: Mark Andrews <[email-masked]>
Date: Thu, 23 Aug 2018 23:24:59 +1000

> On 23 Aug 2018, at 11:15 pm, Tony Finch <[email-masked]> wrote:
> 

The plan is to get to the stage when you are outsourcing the HTTP servers
to just have to publish new records with the names of those servers.  You
only *need* to publish A and AAAA’s for legacy clients.  You are also not
forced to publish the new record as the clients will continue to work with
just A and AAAA records.

This matches MX record behaviour but the recommendation is to do all lookups
in parallel rather than doing it serially.

---

From: Tony Finch <[email-masked]>
Date: Thu, 23 Aug 2018 15:12:39 +0100

> forced to publish the new record as the clients will continue to work with

The reason I like ANAME is that all of this backwards compatibility and
migration faff is automated away.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Viking, North Utsire: Southerly becoming cyclonic then westerly, 4 or 5,
increasing 6 at times. Slight or moderate. Rain then showers, squally at
times. Good, occasionally poor at first.

---

From: Ray Bellis <[email-masked]>
Date: Thu, 23 Aug 2018 19:32:38 +0100

> migration faff is automated away.

Badly, IMHO.

In the transitional phase it puts the onus on authoritative servers to 
find the targets of those ANAME records and publish them.

The corollary of that is that any CDN that's subsequently asked by the 
authoritative for the target A/AAAA addresses of an ANAME target will 
give the address relative to the authoritative server's address and not 
that of the end user's recursive server.

Not only is this poor from the CDN point of view (and after all, it's 
outsourced web hosting provision such as CDNs that are the reason for 
all this in the first place) but it puts the "intelligence" in the wrong 
layer of the DNS.

Ray

---

From: Erik Nygren <[email-masked]>
Date: Thu, 23 Aug 2018 17:57:08 -0400

I will point folks again to draft-schwartz-httpbis-dns-alt-svc (
https://tools.ietf.org/html/draft-schwartz-httpbis-dns-alt-svc-02).
Another older variation on this is draft-nygren-service-bindings-00 (
https://tools.ietf.org/html/draft-nygren-service-bindings-00) from a few
years  back takes a similar approach.

Why I like this over something more minimal:

* Allows indicating the transport to the client, enabling clients that
support HTTP/2 or QUIC to start using it immediately without additional
RTTs to get an Alt-Svc from the server (giving browsers a performance
incentive to support it).
* Provides extensibility so that additional hooks can be added in later
without requiring clients to look up more records.  For example:
         * With "ESNI" (encrypted SNI) we could include the ESNI key or its
name in the ALTSVC DNS record.
         * Being able to indicate that the first label can be replaced by
"_wildcard" in the SNI (eg, to send "_wildcard.example.com" as an SNI) is
another sort of extension discussed.
         * Being able to indicate that some service entries are IPv6-only
(no IPv4 A record is present) might be another useful signal to include as
an optimization some day.
* Provides a better data model and much better operational sanity for
things like ESNI in the context of multi-CDN (but also just for cases such
as moving between hosting providers or CDNs).  By tying attributes like the
ESNI key, supported transport protocols, etc within the same RR as the the
servername, different servernames on different hosting environments can
sanely have different properties.
* Fits into the Alt-Svc model already implemented in some clients.

I'm not tied as much to the particular encoding specifics (the existing
Alt-Svc string format, some CBOR encoding, something that moves a few of
the attributes such as server to a more structured RRTYPE but still has
extensible fields per RR, etc).

What this ends up looking like is something like is an rrset along the
lines of:

  _443._https.example.com. IN TYPE???  {priority1}
{transportprotocol1} {servername1} {port1} {extension_fieldset_1}

  _443._https.example.com. IN TYPE???  {priority2}
{transportprotocol2} {servername2} {port2} {extension_fieldset_2}

  _443._https.example.com. IN TYPE???  {priority3}
{transportprotocol3} {servername3} {port3} {extension_fieldset_3}

For example:

  _443._https.example.com. IN ALTSVC  10 quic2
example.com.examplecdn.net 443 ( esni=key124._esni.examplecdn.net )

  _443._https.example.com. IN ALTSVC  20 h2 example.com.examplecdn.net
443 ( esni=key124._esni.examplecdn.net; ipv6-only=true )

  _443._https.example.com. IN ALTSVC  40 http/1.1
legacy.example.com.examplecdn.net 443 (
esni=key124._esni.examplecdn.net )

I suspect that some CDNs would then have customers CNAME over "_443._
https.example.com."
to some name on the CDN.  For example in the example above it might be that
it actually CNAMEs to _443._https.example.com.examplecdn.net which then
means that the A and AAAA records for "example.com.examplecdn.net" can be
returned as additionals from the same examplecdn.net authority along with
the ALTSVC record.

    Erik

---

From: Shumon Huque <[email-masked]>
Date: Fri, 24 Aug 2018 00:06:57 -0400

> years  back takes a similar approach.

I reviewed the dns-alt-svc draft on the [email-masked] list several months
ago and encouraged (unsuccessfully) other DNS folks to look at it. I also
made some specific suggestions to Ben Schwartz which I believe he is
planning to incorporate.

Before designing yet another record to solve this problem, I think we need
to first determine if that path would have support from HTTP folks. Or, if
there is already momentum behind the dns-alt-svc draft, and if it is more
likely to gain traction, then perhaps working on refining it is the best
option. Otherwise we risk defining an SRV version 2 record which will again
be ignored.

Why I like this over something more minimal:
>          * With "ESNI" (encrypted SNI) we could include the ESNI key or
> its name in the ALTSVC DNS record.
>   _443._https.example.com. IN TYPE???  {priority1} {transportprotocol1} {servername1} {port1} {extension_fieldset_1}
>
>   _443._https..example.com <http://https.example.com>. IN TYPE???  {priority2} {transportprotocol2} {servername2} {port2} {extension_fieldset_2}
>
>   _443._https.example.com. IN TYPE???  {priority3} {transportprotocol3} {servername3} {port3} {extension_fieldset_3}
>
>
> For example:
>
>   _443._https.example.com. IN ALTSVC  10 quic2 example.com.examplecdn.net 443 ( esni=key124._esni.examplecdn.net )
>
>   _443._https.example.com. IN ALTSVC  20 h2 example.com.examplecdn.net 443 ( esni=key124._esni.examplecdn.net; ipv6-only=true )
>
>   _443._https.example.com. IN ALTSVC  40 http/1.1 legacy.example.com.examplecdn.net 443 ( esni=key124._esni.examplecdn.net )
>
>
> to some name on the CDN.  For example in the example above it might be
> that it actually CNAMEs to _443._https.example.com.examplecdn.net which
> then means that the A and AAAA records for "example.com.examplecdn.net"
> can be returned as additionals from the same examplecdn.net authority
> along with the ALTSVC record.
>
>     Erik
>

I think this all sounds reasonable.

One of the critiques I had of the dns-alt-svc draft was that the server
names were embedded inside a potentially complex text string, which would
make it more challenging for DNS servers (not only authoritative, but
recursive servers also) to pre-populate the additional section of the
response with the server address records to help minimize latency.
Splitting the server name (and perhaps other components) into a structured
field in the rdata, as you show, could address that. Addition of a priority
field is also a good idea. Without that, the only way to specify priority
(since records within an RRset are unordered) would be to serially specify
the server names and associated parameters inside the same ALTSVC record
string, and it would also make it impossible to specify equal priority
servers for load balancing.

I've also proposed that client behavior when processing ALTSVC responses
should be to shuffle or randomize the ALTSVC RRs (of equal priority), and
also shuffle address records for each server if there are multiple of them
(if they are returned by the server in additional data). This would ensure
better load balancing behavior regardless of whether the DNS server
shuffles or randomizes the records.

-- 
Shumon Huque

---

From: Ray Bellis <[email-masked]>
Date: Fri, 24 Aug 2018 11:36:14 +0100

On 24/08/2018 05:06, Shumon Huque wrote:

> Before designing yet another record to solve this problem, I think we 
> need to first determine if that path would have support from HTTP folks. 
> Or, if there is already momentum behind the dns-alt-svc draft, and if it 
> is more likely to gain traction, then perhaps working on refining it is 
> the best option. Otherwise we risk defining an SRV version 2 record 
> which will again be ignored.

It's completely unclear to me whether ALTSVC would be deployable as the 
primary means of directing web clients to the primary target for the 
initial HTTP connections.

It is, after all, a DNS encoding of the HTTP *Alternate* Service feature.

> [deleted]
> should be to shuffle or randomize the ALTSVC RRs (of equal priority), 
> and also shuffle address records for each server if there are multiple 
> of them (if they are returned by the server in additional data). This 
> would ensure better load balancing behavior regardless of whether the 
> DNS server shuffles or randomizes the records.

We specifically heard at the side meeting that the browser folks don't 
want load balancing information stored in the DNS.  What we haven't 
heard (that I know of) is what the web *content* folks think of that.

I do know that some of the hardware GLB appliances that use DNS tricks 
to implement load-balancing do a pitifully poor job of DNS compliance.

FWIW, I'd also like to do away with underscore labels for these records, 
since (again from the web / DNS hosting point of view) they interact 
badly with DNS wildcard records.

Ray

---

From: Shumon Huque <[email-masked]>
Date: Fri, 24 Aug 2018 07:06:33 -0400

> It is, after all, a DNS encoding of the HTTP *Alternate* Service feature.

I had the same question. But from talking to some HTTP folks at the last
IETF, I believe that they do see it as potentially being the primary means
of redirection, at least down the road when there is widespread support for
it. The need to maintain a HTTP responder at the origin's address records
is to minimize service latency - the initial traffic can come from the
origin address, until the client obtains and connects to the address
records associated with the ALTSVC hosts. Unfortunately this means
maintaining (or standardizing) alias redirection hacks for quite some time.
I think we can improve the latency picture further by requiring DNS
responders to pre-populate the address records in the ALTSVC response.

> heard (that I know of) is what the web *content* folks think of that.

I'm quite sure web content folks do want load balancing via DNS. Witness
the immediate uproar when a recent version of BIND accidentally undid their
RRset response randomization behavior.

> badly with DNS wildcard records.

Yes, I think this issue is definitely worth addressing. One question is:
does HTTP need port or other parameter specific redirection, which
presumably is why the DNS ALTSVC record has the underscore labels. And if
so, what is the best way to achieve that? Pushing those parameters into the
RDATA is one way, but that will lead into predictable arguments about
record subtyping and inability to surgically extract just the required
response.

-- 
Shumon Huque

---

From: Erik Nygren <[email-masked]>
Date: Wed, 29 Aug 2018 14:10:10 -0400

[deleted]

> record subtyping and inability to surgically extract just the required

They unfortunately may be needed as otherwise we'd be back in the same boat
around CNAMEs at the zone apex.  At least for the CDN use-case, once a
non-trivial amount of information shows up in the rrtype, it likely needs
to be managed by the CDN.  Having "_443._https.example.com" means it can be
a CNAME'd whereas if was a record on "example.com" we'd be back in the same
boat.

The underscore labels also provide the ability to have multiple services on
the same hostname.

I haven't fully thought it through, but perhaps there is a better way to
support underscore labels and wildcards in a way that only authorities need
to worry about?  For example, allowing for "_443._https.*" records in an "
example.com" zone in a way that authorities could know how to handle?

      Erik

---

From: "Patrik Fältström" <[email-masked]>
Date: Wed, 29 Aug 2018 20:42:20 +0200

On 29 Aug 2018, at 20:10, Erik Nygren wrote:

> They unfortunately may be needed as otherwise we'd be back in the same boat around CNAMEs at the zone apex.  At least for the CDN use-case, once a non-trivial amount of information shows up in the rrtype, it likely needs to be managed by the CDN.  Having "_443._[https.example.com](<http://https.example.com>)" means it can be a CNAME'd whereas if was a record on "[example.com](<http://example.com>)" we'd be back in the same boat.

More specifically, it can be delegated.

See also RFC 7553 - the URI Resource Record.

  Patrik

---

From: Ray Bellis <[email-masked]>
Date: Wed, 29 Aug 2018 21:30:47 +0100

On 29/08/2018 19:10, Erik Nygren wrote:

> They unfortunately may be needed as otherwise we'd be back in the same 
> boat around CNAMEs at the zone apex.  At least for the CDN use-case, 
> once a non-trivial amount of information shows up in the rrtype, it 
> likely needs to be managed by the CDN.  Having "_443._https.example.com 
> <http://https.example.com>" means it can be a CNAME'd whereas if was a 
> record on "example.com <http://example.com>" we'd be back in the same boat.
> 
> The underscore labels also provide the ability to have multiple services 
> on the same hostname.

So does having multiple RR types.

> support underscore labels and wildcards in a way that only authorities 
> need to worry about?  For example, allowing for "_443._https.*" records 
> in an "example.com <http://example.com>" zone in a way that authorities 
> could know how to handle?

IMHO that's exceedingly unlikely.   The wildcard label has special 
meaning and recursive servers also need to know about it for DNSSEC.

I'd like to hear from actual web content providers as to whether lack of 
support for wildcards would be an issue.

Having worked in that business (albeit many years ago) I suspect it 
would be.  There are many web services that offer content under 
<username>.<domain> where the username portion is synthesized from a DNS 
wildcard record rather than provisioning separate DNS records for every 
single registered username.

Ray