Subject: [Doh] DoH client-server interoperability vs. strict HTTP parameter checking

From: Christoph <[email-masked]>
Date: Sat, 01 Jun 2019 12:36:00 +0000

Hello,

we run a public DoH service and while running and comparing
different DoH server implementations in practice we noticed that
some have a higher error rate than others.
The error rate was measured by looking at the fraction of each HTTP
response code.

It turned out that those server implementations with a higher error rate
just do stricter HTTP parameter checking. Not necessarily the content of
the parameter but the list of parameters contained in the request.

The result is that some DoH clients are not compatible with some DoH
servers.

The error condition is caused by DoH clients that have additional HTTP
parameters in the request that is not specified in section 4.1 of
RFC8484 in combination with DoH server implementations that enforce the
absence of any additional parameters.

To increase interoperability we reached out to (some) affected DoH
client and server vendors but we would also like to hear opinions from
this WG.

Should DoH servers reject requests with a '400 Bad Request'
response when the request contains HTTP parameters not defined in RFC8484
or
should DoH servers be less strict and simply ignore additional HTTP
parameters and proceed with processing the relevant HTTP parameters
required to provide an answer?

thanks,
Christoph

---

From: Paul Hoffman <[email-masked]>
Date: Sat, 1 Jun 2019 14:42:41 +0000

> response when the request contains HTTP parameters not defined in RFC8484

No.

> required to provide an answer?

Yes.

RFC 8484 makes it clear that a "DoH server" is just an HTTP server that understands the DoH semantics. Trying to limit it beyond that is just wrong (or maybe just lazy).

--Paul Hoffman

---

From: "Mark Delany" <[email-masked]>
Date: 1 Jun 2019 16:08:30 +0000

> RFC 8484 makes it clear that a "DoH server" is just an HTTP server that understands the DoH semantics. Trying to limit it beyond that is just wrong (or maybe just lazy).

(I'm not sure about lazy since an HTTP server probably has to have
more code included to make such tests.)

As to the original question, the problem for a server that rejects
unrecognized HTTP parameters is, how does it differentiate between a
non-standard parameter and a parameter which was standardized in a
future version of RFC8484 written after the server was deployed?

Surely a server has to expect that it will eventually be talking to a
client from the future which will, by definition, be adding unknowable
parameters.

Clearly a client is taking a small but non-zero risk that its
non-standard parameters may collide with the future as well, but the
namespace is kinda large so the risk is low.

Out of curiousity, what are these extraneous parameters and why is the
client sending them? Are they relevant to the DoH query?

Mark.

---

From: Christoph <[email-masked]>
Date: Sat, 01 Jun 2019 16:39:00 +0000

>>> required to provide an answer?

Thanks for the clear reply I hope this is heard.

> client sending them? Are they relevant to the DoH query?

Before version 05, the draft had additional parameter (ct)
which some (old) clients apparently still send.

---

From: Patrick McManus <[email-masked]>
Date: Sat, 1 Jun 2019 17:22:13 -0400

Hi.

8484 servers specify their set of doh uris via uri templates.. which
certainly could involve params not in 8484. Such a service would require
those params.

As for whether a server should be liberal in receiving a uri that does not
match it's configured template ... That's a specific instance of a question
perhaps more hotly debated than doh itself..

https://tools.ietf.org/html/draft-iab-protocol-maintenance-03

8484 does not, and really cannot, proscribe the error handling for whether
the rest of the http server fixes the issue

On Sat, Jun 1, 2019, 8:36 AM Christoph <[email-masked]> wrote:

> _______________________________________________
> Doh mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/doh
>
>

---

From: bert hubert <[email-masked]>
Date: Sun, 2 Jun 2019 10:49:56 +0200

> client from the future which will, by definition, be adding unknowable

And in the DNS world we've often let clients know we don't get their
newfangled speak (FORMERR).  Silently ignoring unexpected parameters is a
great way to proliferate half-working configurations.  "Did
parentalcontrol=0&malwarefilter=1 get accepted or not?  I have no idea".

Secondly, the accepting nature of most HTTP services has created reams of
tracking possibilities, viz utm_source and fbclid which now get tacked
on at random and track you even if you copy paste the URL to someone.

So in short, I would not go out and recommend that servers simply ignore
unknown parameters. They might accidentally do it but it fosters bad things
like silent errors & even more tracking than cookies, TLS resumption tickets
and HTTPS agent headers already bring.

	Bert

---

From: "Mark Delany" <[email-masked]>
Date: 2 Jun 2019 12:17:54 +0000

> parentalcontrol=0&malwarefilter=1 get accepted or not?  I have no idea".

There are certainly pros and cons to this as well as many counter-examples.

Haven't we just spent the last decade trying to encourage things like caches,
forwarders and firewalls to accept and pass thru DNS RR types they don't
recognize?  After all, how would we ever extend DNS if every component rejected
every RR type and OPT sub-type it didn't understand?

I'm not sure I see the distinction between disallowing unexpected HTTP
parameters - as you recommend - and disallowing unexpected DNS parameters -
which I presume you don't recommend.

> and HTTPS agent headers already bring.

I sympathize with the problem and like many here I too have been on the
receiving end of protocol implementations which were far too lax in what they
sent/accepted. But you're effectively recommending that DoH shouldn't use the
extension mechanisms built into HTTP. At that point you have to ask whether it's
even HTTP any more.

A related problem is that most DoH implementors probably rely on pre-existing
HTTP componentry rather than rolling their own from scratch. Said componetry is
notorious for adding headers without necessarily giving the application much
control over it. After all, it's worked fine for the last 20+ years, why stop
now? So at least on that front I think the horse has well and truly bolted.

I suppose one could go down the path of a DoH HTTP profile or version but most
internet protocols seem to shy away from the strictures of profiling and
versioning - perhaps because upgrades tend to turn into monumental events.

I don't think there's ever been a good solution to this dilemma: accepting
unknown parameters is a protocol risk yet accepting unknown parameters is also
the preferred mechanism for version interop.

Mark.

---

From: Joe Abley <[email-masked]>
Date: Sun, 2 Jun 2019 11:54:20 -0400

Hi Mark,

On 2 Jun 2019, at 08:17, Mark Delany <[email-masked] <mailto:[email-masked]>> wrote:

> recognize?

I'm not sure that's actually a particularly good example. New RRTYPEs are not new parameters or new protocol elements; they're existing ones with different code-points. The distinction is important. To the kinds of intermediate actors you mention, wire-format RRSets ought to be passed through as opaque structures.

> After all, how would we ever extend DNS if every component rejected

EDNS is a better example, since RFC 6891 makes it clear that unknown options MUST be ignored.

In the very general sense (ignoring protocol specifics relating to the DNS and HTTP) it feels like it might be handy to be able to specify a capabilities requirement with a query. Something like

 - I want an answer to the question PIR.ORG/IN/SOA <http://pir.org/IN/SOA> with RD=1
 - I want query minimisation and secure transport for recursive queries
 - I want you to confirm explicitly that you understand the request for query minimisation

A receiving server could ignore the request for secure transport if it didn't understand it, but would refuse to process the query if it didn't understand query minimisation (and hence would otherwise silently discard the request).

Joe

---

From: Jim Reid <[email-masked]>
Date: Sun, 2 Jun 2019 17:21:52 +0100

> On 2 Jun 2019, at 16:54, Joe Abley <[email-masked]> wrote:
> 
>  - I want an answer to the question PIR.ORG/IN/SOA with RD=1

Hmmm. The add list might be a more appropriate place to discuss this. Or maybe dnsop. It’s not clear if these sorts of capabilities could/should be signalled in DoH or vanilla DNS.

---

From: "Mark Delany" <[email-masked]>
Date: 2 Jun 2019 20:57:35 +0000

> >  - I want you to confirm explicitly that you understand the request for query minimisation

Yes, I had thought about mentioning capabilities but I doubted anyone would have
read below the fold of my email :-)

One thing I wondered about is how well capabilities work in a multi-hop and
multi-transport system that has DoH in the path. In the very general sense
should a stub resolver asking for capabilities have said query punch all the way
through to an auth server?

There is already a smattering of adhoc capability signals in DNS such as TC=1
and maximum response size which probably should have received some attention in
8484 so if nothing else maybe formalization will at least raise the question for
future protocol designers: "how do we deal with capabilities?".

> Hmmm. The add list might be a more appropriate place to discuss this. Or maybe dnsop. It???s not clear if these sorts of capabilities could/should be signalled in DoH or vanilla DNS.

It's certainly a big topic and probably risks the second-system syndrome.

Mark.

---

From: Vladimír Čunát <[email-masked]>
Date: Thu, 6 Jun 2019 19:14:21 +0200

> RFC 8484 makes it clear that a "DoH server" is just an HTTP server that understands the DoH semantics. Trying to limit it beyond that is just wrong (or maybe just lazy).

OK, I'm certainly not familiar with all HTTP customs, I'm afraid, so
this expectation wasn't clear to me just from reading RFC 8484.

On 6/1/19 11:22 PM, Patrick McManus wrote:
> 8484 does not, and really cannot, proscribe the error handling for
> whether the rest of the http server fixes the issue

I fail to see reasons why an RFC can't (or shouldn't) specify behavior
around unknown parts in the URL.  In this case I suspect that a clear
decision will be better than leaving it unspecified, *regardless* of
which way it will be decided.  I believe the robustness principle is
about handling messages that do *not* conform to such a specification.

> client from the future which will, by definition, be adding unknowable

I wouldn't really expect extensibility on *this* layer.  For DNS itself
there's the whole DNS message in the request and answer (we have EDNS
there).  The HTTP layer would be covered by HTTP headers, I thought.

In particular both these extension "layers" communicate in *both*
directions, which seems very useful for interoperability (first Bert's
paragraph, essentially).  I'm not aware of the HTTP parameters being
usable in both directions, but I'm certainly not a good judge in the
HTTP parts.  I'd generally hope for DoH trying to avoid transmitting
additional information that's not in the embedded DNS messages.

On 6/2/19 5:54 PM, Joe Abley wrote:
>  - I want an answer to the question PIR.ORG/IN/SOA
> <http://PIR.ORG/IN/SOA> with RD=1

DoH is sending binary DNS messages, i.e. they include the RD flag inside
and you actually have to set it the way you want it.  You can also put
EDNS inside and receive it back - that's the probable target of your
second and third example, as I see nothing specific to DoH in there, but
so far there aren't any standardized extensions usable for these (I
believe).

P.S.:
In Knot Resolver case it actually is a bit easier to be strict in this
respect, but that's not important for the protocol discussion ;-)

--Vladimir

---

From: Joe Abley <[email-masked]>
Date: Thu, 6 Jun 2019 13:26:16 -0400

On 6 Jun 2019, at 13:14, Vladimír Čunát <[email-masked]> wrote:

> DoH is sending binary DNS messages, i.e. they include the RD flag inside and you actually have to set it the way you want it.  You can also put EDNS inside and receive it back - that's the probable target of your second and third example, as I see nothing specific to DoH in there, but so far there aren't any standardized extensions usable for these (I believe).
> 
I haven't thought about this very hard so this is not really an opinion, but I agree: it seems sensible that if there is some kind of request/require capabilities work to be done it shouldn't be transport-specific.

Joe

---

From: Patrick McManus <[email-masked]>
Date: Thu, 6 Jun 2019 15:09:25 -0400

On Thu, Jun 6, 2019 at 1:14 PM Vladimír Čunát <[email-masked]>
wrote:

> which way it will be decided.  I believe the robustness principle is about
> handling messages that do *not* conform to such a specification.
>

The DoH service is scoped by the URI template.. things that match that
template are defined by 8484, but HTTP servers certainly can implement
other services (such as the hypothetical doh fixup gateway) on other URIs
and 8484 cannot define them. This independence is an important part of
HTTP. So this particular boundary is a really hard place to put a hard
error requirement on.