Subject: on DNS records

From: Eliot Lear <[email-masked]>
Date: Tue, 13 Nov 2012 16:07:18 +0100

As I promised last week, I've substantially developed a draft (and some
code) to look more closely at whether a DNS record can be helpful. 
Before I get it out there, I wanted to check goals.  The situation is
this: there is a 2.0-enabled client and it must determine whether or not
the other end can speak 2.0.  Gabriel and Willy have already shown us
that it can be done INSIDE the protocol at the expense of one roundtrip,
but at the risk of a proxy doing the wrong thing (a classic case being
that it allows an Upgrade: header but then barfs all over the upgrade). 
So let's state our goals:

1.  Keep latency down.

  * First, is this a reasonable goal?
  * Can it reasonably be done better than what Willy & Gabriel have laid
    out?

2.  Transport Protocol Discovery

  * Some have suggested that it would be useful to do HTTP over UDP or
    SCTP.  Is that something that is a reasonable goal?

3.  Handle the case where multiple instances of the same application
protocol reside on the same host, but on different ports.

  * Is this a reasonable goal?

4.  No new URI schema

  * Address the 'side of the bus' problem.
  * Is this a reasonable goal?

Does this about cover it?  I claim we can solve all four, but not easily
with SRV.

Eliot

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 13 Nov 2012 10:29:02 -0500

I think the main argument for using a DNS record (which I am less fussed
about) is the complexity issue.

Yes we can keep finding per-protocol hacks to describe upgrade mechanisms.
But they mean that every HTTP/2.0 will always have to rely on having a
HTTP/1.1 implementation around to bootstrap from. So HTTP/2.0 cannot do
anything to reduce the complexity of the protocol implementation, it has to
be additions, nothing taken away.

Putting the information in the DNS means that we can move towards a
situation where all Internet application protocol servers follow the same
basic pattern on startup or a configuration change:

1) Read the configuration data
2) Communicate the data to a 'service announcement' service
3) Service announcement service configures local network to provide
support, including configuring the DNS, firewalls etc.
4) Periodically tell the service announcement service that the server is
still alive

Put SSL certificates in that configuration mechanism and DANE or something
DANE-like for security policy advertisement becomes practical.

Now the authoritative mechanism by which the service advertises the service
characteristics should be DNS, but that does not mean that the mechanism by
which the client obtains that information has to be the DNS client-resolver
protocol. It has to be a trusted protocol certainly, but it seems rather
silly that in 2012 we still have a situation where my web browser relies on
the DNS server provided by Panera rather than one that I choose and trust.

On Tue, Nov 13, 2012 at 10:07 AM, Eliot Lear <[email-masked]> wrote:

> code) to look more closely at whether a DNS record can be helpful.  Before
> I get it out there, I wanted to check goals.  The situation is this: there
> is a 2.0-enabled client and it must determine whether or not the other end
> can speak 2.0.  Gabriel and Willy have already shown us that it can be done
> INSIDE the protocol at the expense of one roundtrip, but at the risk of a
> proxy doing the wrong thing (a classic case being that it allows an
> Upgrade: header but then barfs all over the upgrade).  So let's state our
> goals:
>
> 1.  Keep latency down.
>
>    - First, is this a reasonable goal?
>    - Can it reasonably be done better than what Willy & Gabriel have laid
>    out?
>
>    - Some have suggested that it would be useful to do HTTP over UDP or
>    - Is this a reasonable goal?
>
> 4.  No new URI schema
>
>    - Address the 'side of the bus' problem.
>    - Is this a reasonable goal?
>
>

-- 
Website: http://hallambaker.com/

---

From: James M Snell <[email-masked]>
Date: Tue, 13 Nov 2012 07:42:55 -0800

> be additions, nothing taken away.
Well... I've said it before but we're very much going to need a pure http/2
option here that can operate without dns or http/1... basically, just open
a port and start sending http/2 and assume it'll work... Beyond that, the
DNS approach seems to be the most reasonable, non-hacky option on the table
so far.

- James

> Now the authoritative mechanism by which the service advertises the
> service characteristics should be DNS, but that does not mean that the
> mechanism by which the client obtains that information has to be the DNS
> client-resolver protocol. It has to be a trusted protocol certainly, but it
> seems rather silly that in 2012 we still have a situation where my web
> browser relies on the DNS server provided by Panera rather than one that I
> choose and trust.
>
>
>
>
>>    - Can it reasonably be done better than what Willy & Gabriel have
>>    laid out?
>>

---

From: Eliot Lear <[email-masked]>
Date: Tue, 13 Nov 2012 17:07:36 +0100

James,

The question I asked isn't whether or not to use DNS, but what the
requirements for a record would be.  From Phil I gather that limited
complexity is a goal (we will of course argue what that means).  What
are your goals for a DNS record?

Eliot

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 13 Nov 2012 11:26:55 -0500

I would say that the requirement for the DNS option for signalling upgrade
be that

1) It be an instance of a general mechanism that can be used to signal more
than just protocol version and more than just information to HTTP.

2) It be possible to secure the protocol version information to prevent
downgrade attacks, and upgrade attacks.

In band upgrade hacks are a one-off solution. Complexity is frequently the
result of too many one-off solutions. We do occasionally have complexity
from systems that are too general (ISAKMP, EAP, GSSAPI). But that comes
from schemes that are designed to allow people to achieve the same basic
function in an infinite variety of ways. I can't think of a case where we
have introduced too much complexity by specifying one 'right' way to
implement a function plus one or two fallback mechanisms to work-around
legacy issues.

There are also advantages to defining a DNS mechanism that are not quite
requirements. One of them being that HTTP/1.1 use of DNS is rather a mess
and this provides an opportunity to clean it up.

In particular, HTTP relies on DNS server hacks to implement load balancing
(the round robin hack). SRV is a better solution for Web, URI is a better
solution for Web Services. But neither SRV nor URI are complete in that
respect as neither addresses geographic load balancing.

I do not want to get into the issue of how to implement those features
right here and now. But I would like us to take an approach to getting out
of the HTTP version swamp that provides us with the necessary hooks to get
out of the load balancing, geography etc. swamps in the future.

HTTP/2.0 is an incompatible upgrade. When you break backwards compatibility
you should always have (1) a very good reason to do so and (2) an
explanation for why you do not expect to do so again.

---

From: Amos Jeffries <[email-masked]>
Date: Wed, 14 Nov 2012 13:08:55 +1300

On 14.11.2012 05:26, Phillip Hallam-Baker wrote:
> (the round robin hack).

I was always of the impression that round-robin was a designed feature 
of DNS, not a hack created by HTTP. Maybe I'm missing something but 
don't all other name-based protocols also use DNS round-robin for load 
balancing?

> SRV is a better solution for Web, URI is a better

I'd like to see an implementation of round-robin operating within the 
text of a SRV record.

> out of the load balancing, geography etc. swamps in the future.

One could as easily mandate that HTTP implementations perform 
round-robin (or not) on the set of DNS IP results as on the set of SRV 
results. So the above example of "hack" problems appears to be 
irrelevant.

> explanation for why you do not expect to do so again.

Is it? that would be violating the charter requirement that is be 
semantically compatible with HTTP/1.x and fully capable of 2.0->1.1 
gateway translation.

Amos

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 13 Nov 2012 20:30:05 -0500

On Tue, Nov 13, 2012 at 7:08 PM, Amos Jeffries <[email-masked]> wrote:

> I was always of the impression that round-robin was a designed feature of
> DNS, not a hack created by HTTP. Maybe I'm missing something but don't all
> other name-based protocols also use DNS round-robin for load balancing?

It was invented by Eric Binna, Lou Montouli, Rob McCool and the folk at
NCSA as a desperate BIND hack to save their ass when downloads of Mosaic
required a server farm to support.

> I'd like to see an implementation of round-robin operating within the text
> of a SRV record.

SRV was designed to render round robin unnecessary. It has the same load
balancing capability as the MX record. In fact it is simply a
generalization of MX.

> One could as easily mandate that HTTP implementations perform round-robin
> (or not) on the set of DNS IP results as on the set of SRV results. So the
> above example of "hack" problems appears to be irrelevant.

Round robin does not work very well and never has done. The problem is that
DNS records are cached and the caching resolver has no idea what the
relative priority of the servers is. Nor does it have any idea about
matching the request on a geographic basis.

> semantically compatible with HTTP/1.x and fully capable of 2.0->1.1 gateway
> translation.

A HTTP/1.1 server will not be able to understand 2.0 traffic unless
upgraded. Hence it is a backwards incompatible change with a
transition/upgrade strategy.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Wed, 14 Nov 2012 08:05:41 +0000

IMO it's a real shame that the designers of the SRV record decided to 
code the transport protocol into the record name, rather than the 
record data.

It means the client must pre-suppose what the transport protocol must 
be before doing the lookup, whereas if the transport were in the 
record, it would become server-driven, and extensible, e.g. add 
tls-over-tcp to indicate that the server is available over tls on tcp.

Then you'd just look up something like _service.name and get back 
everything you needed to make a connection, e.g. the server, transport 
and sub-channel (e.g. port number or whatever describes the sub-channel 
for the transport).

Adrien

------ Original Message ------
From: "Eliot Lear" <[email-masked]>
To: "[email-masked] Group" <[email-masked]>
Sent: 14/11/2012 4:07:18 a.m.
Subject: on DNS records
>As I promised last week, I've substantially developed a draft (and 
>some code) to look more closely at whether a DNS record can be 
>helpful.  Before I get it out there, I wanted to check goals.  The 
>situation is this: there is a 2.0-enabled client and it must determine 
>whether or not the other end can speak 2.0.  Gabriel and Willy have 
>already shown us that it can be done INSIDE the protocol at the 
>expense of one roundtrip, but at the risk of a proxy doing the wrong 
>thing (a classic case being that it allows an Upgrade: header but then 
>barfs all over the upgrade).  So let's state our goals:
>
>1.  Keep latency down.
>First, is this a reasonable goal? Can it reasonably be done better 
>than what Willy & Gabriel have laid out? 2.  Transport Protocol 
>Discovery
>Some have suggested that it would be useful to do HTTP over UDP or 
>SCTP.  Is that something that is a reasonable goal? 3.  Handle the 
>case where multiple instances of the same application protocol reside 
>on the same host, but on different ports.
>Is this a reasonable goal? 4.  No new URI schema
>Address the 'side of the bus' problem. Is this a reasonable goal? 
>Does this about cover it?  I claim we can solve all four, but not 
>easily with SRV.
>
>Eliot

---

From: Eliot Lear <[email-masked]>
Date: Wed, 14 Nov 2012 13:21:51 +0100

Adrian,

My question is how important is it to make use of different transport
protocols?  I'm asking because there are at least three approaches that
we can take to address DNS:

1.  SRV.  SRV is just fine at providing indirection for host and port,
but not for protocol, and not in such a pretty way when a port number is
specified.  The cost is typically one query per host, which can
SOMETIMES be included in additional information, when the zone is the
same, and when the client considers additional information.
2.  NAPTR combined with SRV.  This provides an additional level of
indirection to allow the choice of protocol.  SIP uses this.  The cost
is two fold: an additional query plus some additional complexity in
terms of processing NAPTR.  It also would prove challenging to handle
the case where different HTTP servers are running on the same host.
3.  An as-of-yet undefined record that describes what services are
running over what protocols on a host.  Think of it as the old WKS
record on steroids.  I've actually spec'd out the record and written
some code for bind to test this idea, but it has its own set of costs:
the first – and biggest – is that it scales linearly with the number of
services that are to be advertised for a given host.  Put another way:
you could just see the advice in an applicability statement "don't use
me unless you really have to."

One other note about SRV:

Use of SRV of any form with regard to TLS would require a substantial
change in how we clients validate hostnames.  I tell you from personal
experience that having a new SAN "Other" type is *not* an easy thing to
ask of CAs.

Comments please?

Eliot

On 11/14/12 9:05 AM, Adrien W. de Croy wrote:
> and sub-channel (e.g. port number or whatever describes the
> sub-channel for the transport).
>  
> Adrien
>
>> situation is this: there is a 2.0-enabled client and it must
>> determine whether or not the other end can speak 2.0.  Gabriel and
>> Willy have already shown us that it can be done INSIDE the protocol
>> at the expense of one roundtrip, but at the risk of a proxy doing the
>> wrong thing (a classic case being that it allows an Upgrade: header
>> but then barfs all over the upgrade).  So let's state our goals:
>>
>> 1.  Keep latency down.
>>
>>   * Can it reasonably be done better than what Willy & Gabriel have
>>     laid out?
>>
>>   * Some have suggested that it would be useful to do HTTP over UDP
>>     or SCTP.  Is that something that is a reasonable goal?
>>

---

From: Martin Thomson <[email-masked]>
Date: Wed, 14 Nov 2012 10:59:18 -0800

On 14 November 2012 04:21, Eliot Lear <[email-masked]> wrote:
> 3.  An as-of-yet undefined record that describes what services are running
> over what protocols on a host.  Think of it as the old WKS record on
> steroids.  I've actually spec'd out the record and written some code for
> bind to test this idea, but it has its own set of costs: the first – and
> biggest – is that it scales linearly with the number of services that are to
> be advertised for a given host.  Put another way: you could just see the
> advice in an applicability statement "don't use me unless you really have
> to."

Just on this, it seems reasonable that the only necessary axis here is
version.  Having multiple records of the same type at the same node is
already possible for SRV for load balancing and failover reasons; this
would add one more reason: versioning.  Two axes is already a lot, a
third is bad enough.

> Use of SRV of any form with regard to TLS would require a substantial change
> in how we clients validate hostnames.  I tell you from personal experience
> that having a new SAN "Other" type is not an easy thing to ask of CAs.

I don't see how you would conclude that.  If you are seeking
'example.com', then that is what you should look for in the
certificate.  It doesn't matter what you had to query the DNS for to
get an IP and port to get there.

This happens all the time already with CNAME records - the browser
still uses the *input* name to validate the certificate, not some
intermediate gunk.

--Martin

---

From: Eliot Lear <[email-masked]>
Date: Wed, 14 Nov 2012 20:09:20 +0100

On 11/14/12 7:59 PM, Martin Thomson wrote:
> version.

Thank you for answering the question.
>   Having multiple records of the same type at the same node is

That's a fair point.  Question: how would you handle SRV with
http://www.example.com:49080?

Eliot

> still uses the *input* name to validate the certificate, not some

I'm saying that TLS processing of hosts with SRV records works
*sometimes* using other protocols, but at least in this circumstance in
thinking about it there is no need for SAN Others.

Eliot

---

From: Martin Thomson <[email-masked]>
Date: Wed, 14 Nov 2012 11:13:55 -0800

> http://www.example.com:49080?

That was raised in the meeting.  SRV isn't enough for that reason
alone.  I know too many hosts that operate on different ports.
Obviously, each port could be given a new name, but that eventually
leads to an increase in the number of wildcard certificates out there.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 14 Nov 2012 14:14:56 -0500

It might be useful to distinguish service from protocol.

If we have an abstract service 'mail' then it might involve the use of
three separate existing application protocols SMTP, POP, IMAP plus new
protocols that do the same thing in XML, JSON or whatever you like.

SRV is only designed to support load balancing and failover. It does not
allow for announcing multiple protocols to do the same thing.

For HTTP the answer might seem rather simple, it is all just Web pages,
right?

But HTTP is also the transport for Web Services and those are much better
published as services in their own right. Say Gopple has a Web service
protocol for mapping and Aggle has a different protocol and they decide to
produce a common spec. The result is not going to be the same as either of
the input specs, it is going to be a third spec.

So now Gopple and Aggle will both be supporting at least two different Web
Service protocols and clients might support one, two or three of them. This
can't happen in the application protocol space because the network effects
and the fact that people can only use one client place heavy and early
pressure on convergence. But in the Web Services space the cost of
supporting multiple protocols is much lower and the convergence pressures
are much less. Witness the fact that none of the implementations of what
purports to be a single frame relay based payment protocol actually
inter-operate.

I have proposed sophisitcated proposals in the past but really what we need
for the purposes of HTTP/2.0 is much less.

On Wed, Nov 14, 2012 at 7:21 AM, Eliot Lear <[email-masked]> wrote:

> protocols?  I'm asking because there are at least three approaches that we
> can take to address DNS:
>
> 1.  SRV.  SRV is just fine at providing indirection for host and port, but
> not for protocol, and not in such a pretty way when a port number is
> specified.  The cost is typically one query per host, which can SOMETIMES
> be included in additional information, when the zone is the same, and when
> the client considers additional information.
> indirection to allow the choice of protocol.  SIP uses this.  The cost is
> two fold: an additional query plus some additional complexity in terms of
> processing NAPTR.  It also would prove challenging to handle the case where
> different HTTP servers are running on the same host.
> biggest – is that it scales linearly with the number of services that are
> to be advertised for a given host.  Put another way: you could just see the
> IMO it's a real shame that the designers of the SRV record decided to code
> the transport protocol into the record name, rather than the record data.
>
> It means the client must pre-suppose what the transport protocol must be
> before doing the lookup, whereas if the transport were in the record, it
> would become server-driven, and extensible, e.g. add tls-over-tcp to
> indicate that the server is available over tls on tcp.
>
> everything you needed to make a connection, e.g. the server, transport and
> sub-channel (e.g. port number or whatever describes the sub-channel for the
> transport).
>
> Adrien
>
> From: "Eliot Lear" <[email-masked]> <[email-masked]>
> To: "[email-masked] Group" <[email-masked]>
> <[email-masked]> <[email-masked]>

---

From: Eliot Lear <[email-masked]>
Date: Wed, 14 Nov 2012 20:18:45 +0100

Martin,
> leads to an increase in the number of wildcard certificates out there.

Thanks.  So you would prefer some other record that simply indicates
what version runs on what port and dump transport protocol?  What do
others think?

Eliot

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Wed, 14 Nov 2012 14:58:48 -0500

Well there are two things here that have the name http

1) The http: URL prefix
2) The HTTP protocol

These are two separate things.

Resolving the first requires the domain and the port (and possibly a method
identifier). The second is where SRV might come in.

So a layered resolution protocol might look something like:

To resolve: http://example.com:2033/fooo

First take the XXX record using the port and protocol as modifiers:

_2033._http.example.com XXX "v=2.0 d=srv tls=r"

This says use http/2.0 as the transport protocol use SRV for discovery and
TLS is required.

The ability to specify the discovery mechanism in the XXX record permits a
site to use SRV or URI or some completely new discovery scheme. We decouple
the URL resolution mechanism from everything else. People could even use
NAPTR if they must.

A really simple deployment would just omit the discovery (d) attribute
entirely and it would fallback to A or AAAA discovery depending on how
people want to be.

In a more complex environment there might be a desire to specify protocol
version on a per host basis. In this case the initial XXX record would have
to announce the protocol variants on offer and then these would form
additional prefixes to the corresponding discovery records:

_2033._http.example.com XXX "v=2.0 v=1.1 vd=srv tls=r"
_1_1._http._tcp.example.com SRV 1 1 80 www1.example.com
_2_2._http._tcp.example.com SRV 1 1 80 www2.example.com

Alternatively people might want to treat port numbers as second layer
decorations in the same way as protocol versions. This has advantages for
use of DNS wildcarding but for reasons I don't want to explain a particular
DNS RR type should NEVER have a prefix or ALWAYS have a prefix. Mixing the
two cases causes horrible corner cases to surface. Only a record that NEVER
takes a prefix should be wildcarded unless you want horrible things to
happen.

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Wed, 14 Nov 2012 20:59:11 +0000

------ Original Message ------
>http://www.example.com:49080?

in the case of SRV, then 49080 would override port advertised in SRV 
record.

In the case of new RR type, you'd have an unambiguous way to advertise 
in the record to use default or URI-specified port.

But really, in most cases, by allowing advertisement of port for a 
site, people would possibly use a new site / host name instead of 
specifying a new port (except for backward compatibility issues I guess)

I don't know if such a thing exists already in SRV - what happens if 
you say port 0, does that mean just use the default port for the 
protocol?

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 14 Nov 2012 23:10:15 +0100

> what version runs on what port and dump transport protocol?  What do

That was one of the issue I raised several times a few months ago
explaining why I think DNS alone cannot be a solution.

The problem we have with DNS is that it's only server-centric. One single
authority decides what to advertise as the way to connect to their site.
That can work for new ports. But not for port 80. The reason is that on
port 80, you have many components along the chain doing many things such
as simple protocol inspection, load balancing, proxying, caching, etc...
that neither the server not the client sides are aware of !

Thus it's foolish to assume that a different protocol can be talked on port
80 without prior upgrade.

Using DNS to advertise a different port could be a solution to avoid the
HTTP Upgrade, because it could allow users to detect that HTTP/2.0 is
spoken natively on a specific port on the server. However it would still
not take into account the fact that client-side filtering on firewalls
and proxies generally don't allow unknown ports to pass.

A new problem with advertising random ports in DNS records is that it
would be much harder to gain unfiltered access than by advertising only
a standardized port. And if we have a standard port we don't need to
advertise the port in the DNS record.

Ports still cause two new troubles :
  - how to decide what protocol version to use on port XYZ ? (I've seen
    Phillip's proposal, I still think that another natural solution would
    be a dedicated scheme)

  - how can we know that the new port it controlled by the same authorities
    as ports 80/443 along the whole path ?

I'm still thinking that we should do more research on the scheme :

   - keep "http://" as the default scheme which MUST only speak HTTP/1.x
     because that has always been the case. A temptative Upgrade to 2.0
     is encouraged but both the clients and the server know that this
     upgrade is optional, may fail and the traffic may remain HTTP/1.

   - have "https://" use the TLS NPN extension to announce support for
     the protocol on both sides. I should say on "all" sides, because TLS
     is a transport protocol and as such is hop-by-hop : if intercepted,
     the interceptor's capabilities are used. At least this has proved to
     work well for SPDY and I don't see why we should refrain from using
     this for HTTP/2.

   - have "http2://" as the optimal replacement scheme for the long term,
     on a dedicated well-known port. To the opponents of the new scheme,
     I would say that this has worked for http->https, and I don't see
     any reason for this not to work again, quite the opposite. In fact,
     there would be much incentive to migrate to support this scheme as
     there would possibly be some performance benefits by doing so. And
     just like many web sites still support https and http for the few
     https-impaired users, we would have https/http2 by default, and
     http as the fallback for those who can only use port 80.

Obviously the switch will take time, a few years to get a fairly common
adoption, but this way we're sure to always provide an upgradable path
from HTTP/1, and without breaking what has been assumed for decades (eg:
http://x.y.z:port/ is http and not anything else, then we'd equally
have http2://x.y.z:port/ be http2 and not http).

So in the end, we'd get the following possibilities to connect to a web
site from a user agent :

  - if scheme is "https://", use the best HTTP version advertised as NPN
    by the server ;

  - if scheme is "http2://", use raw HTTP/2 exclusively over the designated
    port and report an error if it fails (as we do right now with https).
    THIS is what will drive quick opening of the new port on the client side.

  - if scheme is "http://", attempt an 1.1->2.0 Upgrade over the designated
    port, with the possibility that it will remain 1.x only.

Regards,
Willy

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Wed, 14 Nov 2012 22:30:21 +0000

If we had change of scheme on the table (which I don't think we do), 
and possible change of port, then need it be http at all?

e.g. are we then talking about new protocol on new port?

I'd suggest "http2://" as a scheme name will be unpopular, maybe 
something more user-friendly for example "web://"

I think also in the unlikely event that such a proposal went any 
further, we could look at a new way to determine how to connect etc.

Web content authors don't want to be concerned about the protocol used 
to deliver their pages.

So any new scheme would need to take that out of the hands of the 
content author, and would therefore need to allow use of http/1.1, 
http/1.1/tls, http2, ftp, ldap, newproto etc etc.

If the spec required that to connect for a web:// URL the client must 
do a DNS WEB (new RR) record lookup, which could then specify 
transport, port and server name then all this could be published by the 
site operator.

This would then be extensible, and allow for other transport or 
transfer protocols to be used for the web in future.

Adrien

------ Original Message ------
From: "Willy Tarreau" <[email-masked]>
To: "Eliot Lear" <[email-masked]>
Cc: "Martin Thomson" <[email-masked]>;"Adrien W. de Croy" 
<[email-masked]>;"[email-masked] Group" <[email-masked]>
Sent: 15/11/2012 11:10:15 a.m.
Subject: Re: on DNS records
>http://x.y.z:port/ <safelink(12103c78-49e7-4542-85ee-aa0bc2755e19):http://x.y.z:port/> is http and not anything else, then we'd equally

---

From: Patrick McManus <[email-masked]>
Date: Wed, 14 Nov 2012 17:44:09 -0500

> explaining why I think DNS alone cannot be a solution.
it cannot do the job alone - but it can provide the best service (i.e.
similar level of service as NPN on tls) for many best-practice use cases of
http://. Other cases can use an additional approach (alternate-protocol,
upgrade, etc..) which will certainly be necessary to fill in the gaps. SRV
is essentially a routing mechanism, if you're doing routing some other way
(i.e. a proxy, or a port in the URL, or something that manipulates your
dns) then don't use it. We'll need to also provide another option.

But it is totally forseeable to see http://www.example.com/ generate

A? www.example.com

return
A = 10.10.10.10
Additional Records: {SRV _http2-npn._tcp.www.example.com port=443 host=
www.example.com ,
                              SRV
_http2-cleartext._tcp.www.example.comport=81 host=
www.example.com}

and that's a pretty darn powerful sequence that should imo be enabled.

(obviously there are other incantations that don't require the additional
record.. the A and SRV lookups in parallel for instance.. but that's all
implementation dependent.).

I dare say we've been over all this ground. To move things forward on my
part, I have a PoC implementation as a todo item but its going to have to
wait several weeks for other prior commitments to clear.

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 14 Nov 2012 23:47:46 +0100

> e.g. are we then talking about new protocol on new port?

yes, raw HTTP2 on dedicated default port.

> something more user-friendly for example "web://"

Possibly, yes.

> Web content authors don't want to be concerned about the protocol used 

But they already do. I really think that's a wrong argument we've already
read here, because many of them are already used to have their application
frameworks build the URLs automatically. They even ask us (the reverse-proxy
guys) to provide some "X-Protocol: https" or "X-SSL: yes" to help them
decide whether to emit "http://" or "https://" in all the pages the build.
So the scheme is not that much of an issue, and certainly not a show-stopper.

> http/1.1/tls, http2, ftp, ldap, newproto etc etc.

I'm not sure I'm following you here :-/

> transport, port and server name then all this could be published by the 

We should really not put a MUST here, because it is already incompatible
with plain IP addresses. How did you configure your WiFi router at home ?
You certainly connected to http://192.168.1.1/ and you did not have a
DNS yet. Quite frankly, I don't see what this DNS mess will bring, because
everything is contained in the URI and the DNS only knows how the site is
connected to the net, not how to reach it.

> transfer protocols to be used for the web in future.

That's precisely what the scheme was made for ! Define what transport
protocol to use to fetch the resource.

Willy

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 14 Nov 2012 23:57:22 +0100

On Wed, Nov 14, 2012 at 02:37:51PM -0800, James M Snell wrote:
> On Wed, Nov 14, 2012 at 2:10 PM, Willy Tarreau <[email-masked]> wrote:
> 
> > [snip]
> >
> What I'm failing to understand is why the latter is really a problem. If we
> say that http/2 requires a new port, no matter what, the stuff in the
> middle is going to have to be reconfigured anyway to support http/2.
> Otherwise, we just fall back to http/1.

In how much time do you fall back ? That's the issue. For example, at
a customer, I was used to read my yahoo mails using http:// instead of
https:// because the https scheme was redirected to a dirty auth page
on the proxy. This is not a failure for the browser, there was some
valid contents in return for the request.

Fortunately I (the user) was offered the choice to click on the URL bar
and type "http://mail.yahoo.fr/" instead of letting the browser decide
for me. And being offered the choice is important.

Similarly, I've been used to click links to github which were https and
see them time out because the port was not opened. When I'm used to see
this from these locations, I know that I have to convert the link to http://
before even attempting to use https. If I let the browser try for me without
letting me choose, and each time I have to wait for a 10-second timeout, it
will be a terribly unpleasant experience.

> If the http/2 specific port is not
> open, then http/2 is not supported and that is that. It's a feature. Not a
> bug.

I'm not saying it's a bug either, I'm saying that I find it important that
the user is offered the choice of the transport to use to connect to the
site, as it has worked well for ages now.

> I guess I just don't buy in to the whole "It's gotta be port 80"
> argument.

I'm not saying this, I'm saying that on port 80 we must speak an HTTP/1
compatible protocol. HTTP/1 can fortunately be upgraded to 2.0 so we
should have no reason no to offer this since it significantly improves
end-user experience. We have possibilities to offer several ways to
connect at the user's choice and without breaking protocols nor triggering
issues with invisible products. Let's just assume that users know better
what site they can connect to than the hundred of engineers here.

Regards,
Willy

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 15 Nov 2012 00:05:10 +0100

Hi Pat,

On Wed, Nov 14, 2012 at 05:44:09PM -0500, Patrick McManus wrote:
> similar level of service as NPN on tls)

Except that NPN is not mangled by invisible intermediaries.

> for many best-practice use cases of

But the problem is that you don't know that you're passing through proxies,
otherwise I would have no problem with this.

> and that's a pretty darn powerful sequence that should imo be enabled.

I don't claim it's not powerful, I'm saying that it does not offer benefits
from a deployment point of view above other solutions, but it comes with a
whole bunch of new issues.

After how long do you decide that port 81 fails to connect ? And how do
you offer the end-user the possibility to connect via port 80 which he
*knows* works ? Does "http://www.example.com:80/" suffice to force HTTP/1
over port 80 or will some browsers just attempt raw http2 over that port ?

Also the other issue here is that the web becomes totally fragmented, with
any user advertising random ports, including the port 6000 range that is
well-known to pass through many firewalls due to some X11 pass-through,
etc... In the end, for proxy and firewall admins, it becomes a nightmare
to minimally secure their installations.

You see, all these are issues that are overlooked from the DNS admin, but
which are realities on the end-user side. This is why I don't believe that
the server-side admin knows better than his visitors how they should connect
to the net.

Regards,
Willy

---

From: "Adrien W. de Croy" <[email-masked]>
Date: Wed, 14 Nov 2012 23:18:59 +0000

------ Original Message ------
>protocol to use to fetch the resource.

this just comes down to who should define the transport / protocol and 
when.

If you say the protocol and transport are defined in the URI, then it's 
fixed, and if you wish to change protocol or transport, you need to 
change your URIs.

If you adopt an approach where transport and protocol can be 
discovered, then you can change protocol or transport without needing 
to change URIs.

My proposal was just a way of making transport and transfer protocol be 
discoverable (e.g. queried by DNS).  This way a server operator can 
upgrade their server without involving the people who have content 
hosted on it.

Adrien

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 15 Nov 2012 00:22:41 +0100

> upgrade their server without involving the people who have content 

I'm well aware of this, but the issue comes down to how do we allow
end users to fix the mess we deploy and which prevents them from
connecting ?

Also, having the HTTP upgrade supported by default allows all site
owners to support HTTP/2 transparently by the way of an upgrade. A
new scheme would simply allow them to provide their users with a
way to connect with raw frames and skip the upgrade. But that way
nothing is mandatory and each side adopts http/2 at its pace.

Willy

---

From: James M Snell <[email-masked]>
Date: Wed, 14 Nov 2012 14:37:51 -0800

What I'm failing to understand is why the latter is really a problem. If we
say that http/2 requires a new port, no matter what, the stuff in the
middle is going to have to be reconfigured anyway to support http/2.
Otherwise, we just fall back to http/1. If the http/2 specific port is not
open, then http/2 is not supported and that is that. It's a feature. Not a
bug. I guess I just don't buy in to the whole "It's gotta be port 80"
argument.

- James

---

From: Greg Wilkins <[email-masked]>
Date: Thu, 15 Nov 2012 10:41:01 +1100

On 15 November 2012 09:30, Adrien W. de Croy <[email-masked]> wrote:

>
> If we had change of scheme on the table (which I don't think we do), and
> possible change of port, then need it be http at all?
>
> I'd suggest "http2://" as a scheme name will be unpopular, maybe something
> more user-friendly for example "web://"

A URI scheme is not a protocol.    In terms of the web, when a browser
address line starts with "http://", that is specifying how the rest of the
URI should be interpreted, which together with the method and optional
content, defines the semantic of the operation the client is initiating.
That is a separate thing to the wire protocol used to transfer that
request, it just so happens that for the web both are called http (unless
you are using spdy as the wire protocol to transfer http semantics).

We want a upgrade wire protocol, not upgraded semantics.

Well actually we want both - as eventually I'd like to see other semantics
(eg websockets) transported over the HTTP/2.0 wire protocol.

cheers

-- 
Greg Wilkins <[email-masked]>
http://www.webtide.com
Developer advice and support from the Jetty & CometD experts.

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 15 Nov 2012 10:43:31 +1100

Let's keep this discussion focused on DNS records, not URI schemes.

Thanks,

--
Mark Nottingham   http://www.mnot.net/

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 15 Nov 2012 10:43:28 +1100

On 14/11/2012, at 2:07 AM, Eliot Lear <[email-masked]> wrote:

> As I promised last week, I've substantially developed a draft (and some code) to look more closely at whether a DNS record can be helpful.  Before I get it out there, I wanted to check goals.  The situation is this: there is a 2.0-enabled client and it must determine whether or not the other end can speak 2.0.  Gabriel and Willy have already shown us that it can be done INSIDE the protocol at the expense of one roundtrip, but at the risk of a proxy doing the wrong thing (a classic case being that it allows an Upgrade: header but then barfs all over the upgrade).  So let's state our goals:
> 
> 1.  Keep latency down.
> 	• First, is this a reasonable goal?

Safe assumption that it is.

> 	• Can it reasonably be done better than what Willy & Gabriel have laid out?

That's what we're here to find out -- keeping in mind that we're looking for an *optimisation* over the upgrade mechanism at this point, not a replacement for it.

> 	• Some have suggested that it would be useful to do HTTP over UDP or SCTP.  Is that something that is a reasonable goal?

Based on the conversations I've heard, yes.

> 3.  Handle the case where multiple instances of the same application protocol reside on the same host, but on different ports.
> 	• Is this a reasonable goal?

Nice-to-have at most, I'd say. I.e., I personally would be OK saying that the DNS-based optimisation doesn't work for HTTP URIs that specify a non-default port. Yes, we may not be able to get the first round-trip to our router configuration page perfectly optimised, but that's a reasonable price to pay for simplicity. What do others think?

> 4.  No new URI schema
> 	• Address the 'side of the bus' problem.
> 	• Is this a reasonable goal?

Treat it as a fundamental constraint, at least for now.

> Does this about cover it?  I claim we can solve all four, but not easily with SRV.

One other requirement is that it's capable of advertising a range of options, not just binary "use HTTP2?". Doesn't have to be an infinite range, small set will do, I think.

A few more things to tease out:

- Are we advertising that port 80 is capable of HTTP/2, or an alternate port for HTTP/2, or (capable of) both?
- Are we allowing the advertisement to resolve to a different host, or restricting to the named host (a la Alternate-Protocol)?

Regards,

---

From: Willy Tarreau <[email-masked]>
Date: Thu, 15 Nov 2012 00:53:34 +0100

> Let's keep this discussion focused on DNS records, not URI schemes.

OK but then what solution do we offer to end-users to fix the mess
the DNS will announce to their browsers ?

I'm quite certain many of us won't find it fun to have to wait for a
(say) 10 seconds timeout for each site we try to visit just because
what is advertised via the DNS is blocked at our location.

Willy

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 15 Nov 2012 10:54:54 +1100

> what is advertised via the DNS is blocked at our location.

We'll get there.

---

From: Patrick McManus <[email-masked]>
Date: Wed, 14 Nov 2012 19:54:24 -0500

On Wed, Nov 14, 2012 at 6:43 PM, Mark Nottingham <[email-masked]> wrote:

> , I personally would be OK saying that the DNS-based optimisation doesn't
> work for HTTP URIs that specify a non-default port. Yes, we may not be able
> to get the first round-trip to our router configuration page perfectly
> optimised, but that's a reasonable price to pay for simplicity. What do
> others think?
>
>
That's my assumption too - I'm fine with that.

> port for HTTP/2, or (capable of) both?

imo it has to be an arbitrary port.. if you send anything other than http/1
on port 80 across the general internet you'll get some breakage.

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 15 Nov 2012 11:56:10 +1100

On 15/11/2012, at 11:54 AM, Patrick McManus <[email-masked]> wrote:
> imo it has to be an arbitrary port.. if you send anything other than http/1 on port 80 across the general internet you'll get some breakage.

Agreed... however, I don't see a reason to make it impossible to say "I support HTTP/2 on port 80 too", so the client can optimistically try to use it first (as long as it can handle breakage).

Cheers,

---

From: Patrick McManus <[email-masked]>
Date: Wed, 14 Nov 2012 20:01:17 -0500

multiple ports are fine by me.. just not limited to 80.

in general, whether this is a-p, upgrade, or srv, this is also the spot to
mux between tls as well as draft versions of the protocol. (i.e. upgrade:
http2-draft-00-tls)

On Wed, Nov 14, 2012 at 7:56 PM, Mark Nottingham <[email-masked]> wrote:

> > imo it has to be an arbitrary port.. if you send anything other than
> http/1 on port 80 across the general internet you'll get some breakage.
>
> Agreed... however, I don't see a reason to make it impossible to say "I
> support HTTP/2 on port 80 too", so the client can optimistically try to use
> it first (as long as it can handle breakage).
>
> Cheers,
>
>
>
> --

---

From: Amos Jeffries <[email-masked]>
Date: Thu, 15 Nov 2012 14:31:58 +1300

>> If we had change of scheme on the table (which I don't think we do),

Maybe. If so it is on another table over that way --->

This thread needs to handle the case that the client for whatever 
reason has chosen to use only an http:// URL and DNS to fetch the 
content *and* opts not to Upgrade:.

>> something more user-friendly for example "web://"

Might as well redefine HTML to use gopher:// to fetch that XML/JSON 
data, and urn:// on multimedia then. No I'm not joking. Both are 
individually more efficient at that type of transfer than HTTP 
messaging. What is the use of retaining HTTP/1.x semantics if a 
completely different protocol, different scheme, different port is going 
to be the outcome?

> So the scheme is not that much of an issue, and certainly not a 

The best one I have seen was a request for "Via: 1.1 example.com:443 
(HTTPS)" - which *is* a reasonable extension of Via when you think about 
it being completely agnostic to the particular port in use. Although 
even better would be: "Via: HTTP/1.1 example.com:443 (TLS)" which gives 
transparency about the scheme and transport layer combinations as well.

That said the "//example.com/" page URL syntax is a wonderful boon to 
my developer side-life. Unfortunately RFC 2616 requires an absolute URL 
in Location: headers, so a lot of frameworks and custom scripts do not 
support sending it back on redirects.

AYJ

---

From: Eliot Lear <[email-masked]>
Date: Thu, 15 Nov 2012 06:38:56 +0100

On 11/15/12 2:01 AM, Patrick McManus wrote:
> in general, whether this is a-p, upgrade, or srv, this is also the
> spot to mux between tls as well as draft versions of the protocol.
> (i.e. upgrade: http2-draft-00-tls)

Be careful!  Different security properties would lead to downgrade
attacks without the proper use of DNSSEC!!

Eliot

---

From: Roland Zink <[email-masked]>
Date: Thu, 15 Nov 2012 10:30:09 +0100

On 14.11.2012 23:37, James M Snell wrote:
>
> On Wed, Nov 14, 2012 at 2:10 PM, Willy Tarreau <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     [snip]
>
> What I'm failing to understand is why the latter is really a problem. 
> If we say that http/2 requires a new port, no matter what, the stuff 
> in the middle is going to have to be reconfigured anyway to support 
> http/2. Otherwise, we just fall back to http/1. If the http/2 specific 
> port is not open, then http/2 is not supported and that is that. It's 
> a feature. Not a bug. I guess I just don't buy in to the whole "It's 
> gotta be port 80" argument.
>
> - James
I think the argument is that with a default port for http/2 it is 
possible to reconfigure existing equipment like firewalls. With 
arbitrary ports the administrator has no possibility to allow direct 
http/2 traffic. For intermediaries which don't see the DNS traffic it 
will be difficult to detect http/2 without a known port for it.

- Roland

---

From: Roland Zink <[email-masked]>
Date: Thu, 15 Nov 2012 12:03:01 +0100

> Agreed... however, I don't see a reason to make it impossible to say "I support HTTP/2 on port 80 too", so the client can optimistically try to use it first (as long as it can handle breakage).

When it can be determined from the first few bytes on the wire if it is 
HTTP/1 or HTTP/2 then servers and network elements could handle both 
protocols on the same port. The first bit may be enough for this (0 for 
ASCII HTTP and 1 for HTTP/2 control message?). However in the beginning 
there could be unnecessary breakages and a HTTP/2 default port may be a 
way around it. For example traffic may routed around HTTP/1 only network 
elements.

Roland Zink   http://home.zinks.de/