Subject: [DNSOP] opportunistic semi-authoritative caching (Re: DNSOP Call for Adoption - draft-tale-dnsop-serve-stale)

From: Paul Vixie <[email-masked]>
Date: Thu, 07 Sep 2017 18:37:43 -0700

note, there's a proposal contained here.

Jared Mauch wrote:
> On Thu, Sep 07, 2017 at 01:29:47PM -0700, Paul Vixie wrote:
>> if the draft being considered was clear on two points, i'd support adoption.
>>
>> ...
>
> 	Would you see the querying application informing you of intent via
> option code saying "If I'm unable to talk to you once TTL expires, I may serve
> your last known good answer"?

i don't think so. if it was "may i serve your last good answer?" then 
yes. but with it as "i may" and the ? outside the quotes as shown above, 
then no.

> 	What would a server then do if this intent were known?  serve some
> alternate data, or even return REFUSED?  I could see sending a secure notify
> to anyone who requested the QNAME after change, but holding this state may
> end up with complexity similar to what's some have seen with ECS.

this whole approach is incredibly unappealing. as i wrote to jinmei earlier:

> i think the actual problem statement is broader, and that by solving for this
> narrow version, we would lose the complexity/capability tradeoff -- we'd add
> more state and more signaling at a cost higher than what we would get for it.
>
> it's a general reachability problem not specifically ddos problem. reasons for
> not being able to refresh data in a cache include ddos, but also backhoes,
> wire cutters, squirrels, circuit breakers, and bad design/provisioning.
>
> i think the right answer will look like a miniature version of IXFR/AXFR and
> NOTIFY, such that a cache can register its intent to become a partial stealth
> secondary server, and by participating, an authority server can both indicate
> its willingness to have this done, and possibly remember what was transmitted
> so as to facilitate subsequent cache invalidation or zone change notification
> messaging. one could even imagine a dns cookie exchange at the outset, to help
> authenticate later messages. sort of a super-lightweight session protocol.
>
> when the DNS was first popularized, we were using a lot of computers with less
> than four megabytes of memory and fewer than a million instructions per
> second, and our links were thought fast if 56K DDS, or super-fast if 1.544M T1
> or even 2.0M E1. this drove choices of how to encode, how to compress, where
> to synthesize, and whether to authenticate. all of those choices should be up
> for reconsideration now.
>
> if our _vision_ as a technical community is to have little chunks of semi-
> authoritative data cached in stealth-like secondary-like places, and then kept
> up to date, then we should pursue that, because moore's law and the
> privatization and commercialization of the internet have made it now
> practical.

in other words i'd like to solve both partitioning/reachabilities not 
just one. not just "i can't reach a remote server due to a partition" 
but also "i can't learn the role and address of nearby servers due to a 
partition". i guess you could call it "opportunistic semi-authoritative 
caching".

the heuristic for deciding what to cache is probably popularity, but 
also, NS/A/AAAA chains, and also, the full contents of the root zone.

it would only be allowed if dns cookies were exchanged, since i don't 
think we should spam people with notifies after receiving a spoofed 
source stealth-service request.

the desire to stealth-serve would be encoded in an EDNS option bit, 
which if answered in the affirmative, would lead to a REQUEST-NOTIFY 
message that specified the RR-NAME of interest, and an enum saying 
whether it's all RR-SETs there, or just a couple of them, or whether 
it's the whole zone.

this would be in-band content monitoring. a "push" form of DNS. it would 
solve two disconnection problems (distant server unreachable, and also, 
nearby server unidentifiable). it would solve the "root servers might be 
ddos'd or otherwise unreachable" problem without any of the 
configuration hackery specified in RFC 7707. it would not lead to stale 
data. it would not require detailed DNS or server knowledge by an rdns 
operator. in fact it could be put into the middleboxes (wifi, cable, 
dsl) who insist on doing RDNS service now.

it could even be integrated with forwarding.

moore's law, and the internet economy, have been kind. let's use the 
resources we've been given, to come up with a DNS that actually does 
what we want, and doesn't add more static configuration state.

-- 
P Vixie

---

From: Joe Abley <[email-masked]>
Date: Thu, 7 Sep 2017 23:08:43 -0400

Apologies in advance for iPad MIME-crime. See below for crimes committed by workman rather than tools.

> On Sep 7, 2017, at 21:37, Paul Vixie <[email-masked]> wrote:
> 
> i don't think so. if it was "may i serve your last good answer?" then yes. but with it as "i may" and the ? outside the quotes as shown above, then no.

There's a recursive operator with whom Jared and tale may be familiar that some time ago had a feature called "pinning" whereby particular names that were known to be availability-sensitive (their non-availability caused great disturbance in the helpdesk) could be "pinned" -- that is, in the recursive server, they were configured never to expire from the cache. They could be refreshed, but they would not expire.

When I remember learning about this I remember a certain personal conflict. The goal in the implementation I referred to was edge-centric, in the sense that the goal was to reduce the opportunity for end-users to see breakage due to DNS resolution failure. The recursive operator (and, by proxy, the access provider that was fielding the support calls) could probably find out in short order whether this was a safe thing to do for particular domains, and they chose the domains that should be pinned, but I thought and still think that that was a less desirable workflow than the zone manager signalling what the behaviour should be in the event that authoritative servers couldn't be reached. The implementation was pragmatically good, but I thought it could be better.

The conversation I have seen so far is edge-centric; a recursive operator might serve stale data because it believes that a stale answer is better than no answer in all cases. A more granular approach might allow a recursive server to cache a record with multiple TTLs, indicating (for example) that ideally you would refresh this response after N seconds, but if you can't do that then perhaps M seconds would be fine. A large M would reasonably approximate "quite stale". Telling recursive servers what to do in the event of authority servers being unreachable seems better as an authority server operator than having to deal with a variety of behaviours between you and the end-user. This is different from a mechanism whereby a stub or downstream recursive resolver would signal tolerance for stale data in responses.

This could be implemented maybe as an EDNS option used in the request and corresponding response between a recursive server and an authority server. Queries within zones that were not served by their authority servers configured to supply the option or which were not requested by recursive servers prepared to do the necessary would be handled conventionally.

Joe

---

From: Paul Vixie <[email-masked]>
Date: Thu, 07 Sep 2017 22:28:30 -0700

On Thursday, September 07, 2017 11:08:43 PM Joe Abley wrote:
> >> option code saying "If I'm unable to talk to you once TTL expires, I may
> >> serve your last known good answer"?
> > 
> > i don't think so. if it was "may i serve your last good answer?" then yes.
> > but with it as "i may" and the ? outside the quotes as shown above, then
> > no.
> There's a recursive operator with whom Jared and tale may be familiar that
> some time ago had a feature called "pinning" whereby particular names that
> were known to be availability-sensitive (their non-availability caused
> great disturbance in the helpdesk) could be "pinned" -- that is, in the
> recursive server, they were configured never to expire from the cache. They
> could be refreshed, but they would not expire.

the domain name system is the world's first and only distributed, coherent, 
autonomous, reliable database. if someone decides, for the sake of their help 
desk, to use my data for longer than the TTL i signaled, that's an affront to 
both coherence and autonomy, and they should stop.

if they really need this, they should provide a method by which i can specify 
both a TTL and an Expiry, and i will consider publishing both values, and if i 
do, then they can use them the way i intend them. because as i said, autonomy. 
it's my data, and my TTL.

vixie

---

From: Joe Abley <[email-masked]>
Date: Fri, 8 Sep 2017 07:52:19 -0400

> On Sep 8, 2017, at 01:28, Paul Vixie <[email-masked]> wrote:
> 

Yes. That's a more succinct formulation of what I was trying to suggest.

Joe

---

From: Tony Finch <[email-masked]>
Date: Fri, 8 Sep 2017 15:32:32 +0100

Paul Vixie <[email-masked]> wrote:
> do, then they can use them the way i intend them.

RRSIG sort-of does that?

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/  -  I xn--zr8h punycode
Sole, Lundy, Fastnet: West veering northwest 5 to 7. Rough in Sole, otherwise
moderate or rough, occasionally very rough later in Sole and southwest
Fastnet. Thundery showers. Good, occasionally moderate.

---

From: Paul Vixie <[email-masked]>
Date: Fri, 08 Sep 2017 07:59:14 -0700

Tony Finch wrote:
> Paul Vixie<[email-masked]>  wrote:

but it wasn't intended as that, so my use of it cannot be interpreted as 
permission to treat it as that.

-- 
P Vixie

---

From: Evan Hunt <[email-masked]>
Date: Sat, 9 Sep 2017 00:32:48 +0000

On Thu, Sep 07, 2017 at 10:28:30PM -0700, Paul Vixie wrote:
> both a TTL and an Expiry, and i will consider publishing both values, and
> if i do, then they can use them the way i intend them. because as i said,
> autonomy.  it's my data, and my TTL.

I agree, and yet, a DDoS can make your data unavailable for refresh
through no fault of yours, which makes a resolver operator appear to be
broken through no fault of theirs, which makes them want very much to
be able to do this bad thing.

So, TTL stretching goes on the pile with NXDOMAIN redirection, tools that
can be used for censorship, and all the other regrettable things that we
implemented anyway dammit.

(I do like the idea of advertising a separate expiry value though.)

-- 
Evan Hunt -- [email-masked]
Internet Systems Consortium, Inc.

---

From: Paul Vixie <[email-masked]>
Date: Fri, 08 Sep 2017 18:43:52 -0700

> be able to do this bad thing.

understood. this is the same reasoning that led comcast to argue for the 
ability to turn off dnssec for important zones (like jpl.nasa.gov) who 
had fubar'd their keys and/or signatures and/or delegated signers.

> implemented anyway dammit.

not so fast. nxdomain redirection is an attack. censorship is an attack. 
i don't think you mean to group ttl stretching in with those attacks. 
because if you do, then we agree, it is an attack, and ought not be 
done, and certainly ought not be standardized in any form.

> (I do like the idea of advertising a separate expiry value though.)

i think if we're going to put something into the 20-year deployment 
funnel we should treat the fixed costs as high and demand more benefits. 
that's where the proposal up-thread came from.

-- 
P Vixie

---

From: Matthew Kerwin <[email-masked]>
Date: Sat, 9 Sep 2017 12:17:11 +1000

> > do, then they can use them the way i intend them.
Perhaps there's value in looking at HTTP's Cache-Control frankenbeast,
particularly RFC 5861. Being able to specify stale-on-error /
stale-while-revalidate (or explicitly *not*) alongside a hard TTL could be
useful.

Cheers
-- 
  Matthew Kerwin
  http://matthew.kerwin.net.au/

---

From: Evan Hunt <[email-masked]>
Date: Sat, 9 Sep 2017 02:49:18 +0000

> done, and certainly ought not be standardized in any form.

They're both lies, and TTL stretching is a lie, and in principle I
believe the DNS should not lie, but filter-aaaa and dns64 and RPZ all
had good and worthy reasons, and nxdomain redirection had bad reasons
with dollar signs next to them, and here we are.

Just as with RPZ, it seems reasonable to publish guidance on how to
do the kind-of-bad thing in the least bad way.

---

From: Paul Vixie <[email-masked]>
Date: Sat, 09 Sep 2017 08:29:28 -0700

> do the kind-of-bad thing in the least bad way.

rpz is a defense. it assumes that the content owner is trying to hurt 
me. it is therefore one step away from being an attack, and is in any 
case, not an attack.

i think that attack-p is more relevant than lie-p for this discussion.

-- 
P Vixie

---

From: Evan Hunt <[email-masked]>
Date: Sat, 9 Sep 2017 18:06:08 +0000

> me. it is therefore one step away from being an attack, and is in any 

Sure.  And TTL stretching assumes the content owner is a fellow victim,
and someone is trying to hurt both of us by making their site inaccessible
to me.  Both are lies; both have a defensible moral justification.

> i think that attack-p is more relevant than lie-p for this discussion.

The line between attack and not-attack can be surprisingly blurry.  But
given that there are non-malevolent reasons for wanting to serve stale
data, and solutions are being implemented (including one in BIND), I'm
okay with publishing details of the method, same as with RPZ.

---

From: Paul Vixie <[email-masked]>
Date: Sat, 09 Sep 2017 12:22:17 -0700

> to me.  Both are lies; both have a defensible moral justification.

the content owner may have good and specific reasons for the TTL they 
chose, and using that data for longer than that period may be harmful, 
and must be presumed to be harmful unless explicit signaling is added to 
let the content owner specify otherwise.

autonomous, coherent, reliable, distributed: pick four.

-- 
P Vixie

---

From: Vladimír Čunát <[email-masked]>
Date: Mon, 11 Sep 2017 14:57:19 +0200

On 09/09/2017 09:22 PM, Paul Vixie wrote:
> [...]
> and must be presumed to be harmful unless explicit signaling is added
> to let the content owner specify otherwise.

Requiring explicit signalling from authoritative servers, e.g. via EDNS,
might actually work in this case, as that side is also motivated to
implement and deploy support for stale-serving.  We would also avoid all
discussion around whether this needs to be on standards track and marked
as updating 1034+1035.

--Vladimir