Subject: [dnsoverhttp] Caching model

From: Martin Thomson <[email-masked]>
Date: Mon, 30 Oct 2017 12:36:53 +1100

See https://github.com/paulehoffman/draft-ietf-doh-dns-over-https/issues/14
and several others.

My request is that we agree on what the model is, then we can talk
about the properties we can extract from that.

My understanding is that the DNS client would consult its local stack
and that stack would use HTTP to talk to a DNS API server.  In that
model, there are at least three caches in play: the local DNS resolver
cache, the HTTP cache and the DNS API server cache.

If those caches are ordered as I describe, and the two DNS caches are
driven based on the TTL, I'm struggling to find a role for the HTTP
cache.  There are things that HTTP can do with caching that would be
nice, but I'm not seeing any way to really access HTTP caching
features in that architecture.

It's a little unfortunate, but in that architecture it would probably
be better to disable HTTP caching entirely.  HTTP caching has a bunch
of features that are more flexible, but if that caching is wedged
between two relatively inflexible caches, it won't have any
opportunity to add value.

---

From: Ian Swett <[email-masked]>
Date: Sun, 29 Oct 2017 21:53:56 -0400

I assumed HTTP caching would either be disabled or closely integrated with
the DNS resolver's TTL.  Both seem viable, but disabling HTTP caching seems
easier.

On Sun, Oct 29, 2017 at 9:36 PM, Martin Thomson <[email-masked]>
wrote:

> _______________________________________________
> dnsoverhttp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dnsoverhttp
>

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 30 Oct 2017 02:08:07 +0000

> cache, the HTTP cache and the DNS API server cache.

I'm having a problem with that list. Where are the first and third caches? (I'm pretty sure I understand the second one...)

--Paul Hoffman

---

From: Martin Thomson <[email-masked]>
Date: Mon, 30 Oct 2017 13:59:33 +1100

> I'm having a problem with that list. Where are the first and third caches? (I'm pretty sure I understand the second one...)

The DNS API client and server are one and three respectively.

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 30 Oct 2017 15:59:58 +1100

> On 30 Oct 2017, at 12:36 pm, Martin Thomson <[email-masked]> wrote:
> 
> and that stack would use HTTP to talk to a DNS API server.

Maybe in some deployments one day, but my understanding was that immediately, the application using DOH would be making the HTTP requests itself, not using the OS. Isn't the point here to mix it in with regular HTTP traffic (where "mix" means "on the same connections")?

>  In that

Break out of the browser -- a CDN could easily do interesting things here. Indeed, given the likely deployments here, I'd be surprised if there weren't any HTTP caching going on, in some fashion.

> opportunity to add value.

So, disable the client-side DNS cache and use HTTP caching. The DNS server cache isn't relevant; since it's under the control of the authority (who also controls the HTTP caches), it can be manipulated / cleared / etc. as necessary.

That model leaves you with a much more capable cache -- most modern implementations will give you request collapsing, purge / invalidation, serving stale on error, async revalidation, etc.

--
Mark Nottingham   https://www.mnot.net/

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 30 Oct 2017 14:12:48 +0000

> The DNS API client and server are one and three respectively.

Are you saying that the DNS API client might be keeping its own cache with timeouts?

--Paul

---

From: Martin Thomson <[email-masked]>
Date: Tue, 31 Oct 2017 12:33:06 +1100

> Are you saying that the DNS API client might be keeping its own cache with timeouts?

Well, my DNS library does today.  We might do as Mark suggests, and
take steps to disable that, but that might not be the easiest way to
integrate DOH into an existing stack.  If I wanted to retrofit my
operating system so that gethostbyname() used HTTPS, then that is
(apparently) possible by replacing the protocol-y bits of the code.
But it might be too disruptive to disable caching.  Maybe someone who
has had hands on there can speak to that.

---

From: Ben Schwartz <[email-masked]>
Date: Mon, 30 Oct 2017 21:40:02 -0400

This is not the working group list!

Conveniently that means that I am not chair here, so I can have opinions.

I like the idea of zeroing out the TTL on the wire, and converting the TTL
into an HTTP Expires header.  If the client is using HTTP caching, it can
leave the TTL at zero.  Otherwise, it should reconstitute the DNS TTL from
the Expires header.

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 31 Oct 2017 12:45:56 +1100

> On 31 Oct 2017, at 12:40 pm, Ben Schwartz <[email-masked]> wrote:
> 
> I like the idea of zeroing out the TTL on the wire, and converting the TTL into an HTTP Expires header.  If the client is using HTTP caching, it can leave the TTL at zero.  Otherwise, it should reconstitute the DNS TTL from the Expires header.

+1, although it needs to be the Freshness Lifetime (i.e., accounting for both Cache-Control and Age as well).

> 
> On Mon, Oct 30, 2017 at 9:33 PM, Martin Thomson <[email-masked]> wrote:

---

From: Patrick McManus <[email-masked]>
Date: Tue, 31 Oct 2017 02:28:40 -0400

I think the HTTP cache fundamentally plays a role as a shared cache between
multiple clients. Mark draws that picture with a CDN, which is one
possibility. Another is the javascript use case where the client DNS cache
may not exist or even if it does exist the shared cache plays the role of
heirarchical cache.. to whatever extent different first parties can share
http cache entries they can share doh entries.

in the case where this is a straight line, then it makes sense to disable
one of the two client side caches.. note that its common to have a http
cache that also stores a bunch of post-processed stuff linked to the http
cache entry.. firefox has done/does/will do that with a bunch of things
already (raster formats, javascript parsing, wasm ..) and some kind of
parsed dns record that allowed disabling the standalone dns cache would
seem likely.

On Sunday, October 29, 2017, Martin Thomson <[email-masked]>
wrote:

> [email-masked] <javascript:;>

---

From: Erik Kline <[email-masked]>
Date: Tue, 31 Oct 2017 12:50:33 +0900

(moving dnsoverhttp@ to bcc, adding doh@)

>> has had hands on there can speak to that.

Mobile operators have requirements that the DNS resolver layer on the
device cache answers locally.  IIRC this cache has to be shared among
apps (different apps may ask for the same hostname, like popular ads
and analytics services).