Subject: [dns-privacy] some DNS privacy implementation benchmark

From: Carsten Strotmann <[email-masked]>
Date: Sat, 12 Aug 2017 20:49:24 +0000

Hi,

I did a simple (and naive) benchmark of different DNS privacy
implementations available.

The benchmark measures the time needed to query the Alexa top 1000 via
"dig" (sequentially). OS is Linux (amd64, Core2Duo). Network is 100Mbit
FTTH.

DNS-over-TLS was done via the SurfNET DNS-over-TLS resolver.

dnsfwd is a UDP-to-TCP forwarder that keeps the TCP session open
<https://github.com/randomstuff/dnsfwd>

Unbound (1.6.0) opens a new TLS session for each query :(

The DNSCrypt resolver was randomly chosen by the software.

 Protocol/Software                      Time (Sec)  Privacy  DNSSEC 
--------------------------------------------------------------------
 Google DNS (UDP)                               64  --       +      
 DNS-over-TLS (dnsfwd+stunnel)                  67  ++       -      
 local Unbound w/o DNSSEC                      146  -        -      
 local Unbound w. DNSSEC                       163  -        +      
 DNS-over-DNSCrypt (ns0.dnscrypt.is)           243  ++       +      
 DNS-over-Tor                                  254  ++       -      
 DNS-over-TLS (Unbound+dnsfwd+stunnel)         258  ++       +      
 DNS-over-TLS (Unbound+stunnel)                444  ++       +      
 DNS-over-TLS (Unbound buildin TLS)            647  ++       +      

"Stubby" is missing, I having issues getting it to work, I will update
this list once I've got "Stubby" working.

As I have this setup now, is there an working implementation that is
missing and should also be in the list?

DNS-over-QUIC?
DNS-over-HTTP(S)?

Best regards

Carsten Strotmann

---

From: Christian Huitema <[email-masked]>
Date: Sat, 12 Aug 2017 17:36:21 -0700

> implementations available.
...
>  Protocol/Software                      Time (Sec)  Privacy  DNSSEC 

These are interesting measurements, but they mix three variables: the
transport protocol, the RTT between stub and resolver, and the amount of
caching at the resolver.

In my simulations, the key variable is the Round Trip Time between stub
and resolver. The various transports differ in how they handle
connections, retransmission, and head of queue blocking, but these
effects are always expressed in terms of "number of additional RTT times
some probability", the probability generally being at most a few percents.

The other variable in a "stub to recursive" scenario is the amount of
caching done by the resolver. In standard stub to recursive scenarios,
the queries appear to be served from the cache 30% to 40% of the time --
with a better service at big resolvers such as large ISP or Google DNS.
This is because the cache is populated from requests of multiple
clients, plus probably some pre-fetching. Small resolvers typically have
lower cache rate. This matters, because the "recursive to authoritative"
part of the resolution is typically larger than the "stub to resolver" part.

In the unbound scenarios, were you using unbound as a local recursive
server?

> missing and should also be in the list?
You probably need to wait until at least October for realistic
implementations of DNS over QUIC.

-- 
Christian Huitema

---

From: Carsten Strotmann <[email-masked]>
Date: Mon, 14 Aug 2017 08:00:02 +0000

Hello Christian,

> transport protocol, the RTT between stub and resolver, and the amount of

Yes, I know. The goal for this measurements were not to judge the
different protocols, but to give an indication of performance on
implementations available today (this work is part of an magazine
article about implementing DNS-Privacy on Linux/BSD).

> connections, retransmission, and head of queue blocking,

There is a gap between the theoretical abilities of the transports and
the available implementations. I was interested in the user experience
today using the tools that are available in mainstream Linux
distributions (except "dnsfwd", which is only available as
[experimental] source but which I added to not hurt the reputation of
DNS-over-TLS).

> but these

I will re-do the test with a replay of some real-world DNS queries that
will make use of caching. The test I've done now is somewhat artificial,
as it asked 1000 different questions on a cold cache. But that was the
best way to measure the DNS resolution performance trough the different
transports. Getting an answer from a local cache is always 0 ms in a
local LAN.

> In the unbound scenarios, were you using unbound as a local recursive

Yes. The cache was cold at the beginning of each test.

> implementations of DNS over QUIC.

I'm excited about DNS-over-QUIC from the technical point of view, it has
potential. From the point of DNS Privacy, I'm not so thrilled, as I fear
that the one major implementation will be between Chrome Browser and
Google-DNS, which is a Zero-sum game for DNS privacy.

Anyway, I will test drive DNS-over-QUIC once it is available in some
useable form.

Best regards

Carsten Strotmann

---

From: Carsten Strotmann <[email-masked]>
Date: Tue, 15 Aug 2017 15:13:22 +0000

Hello,

here is a followup benchmark test, same setup, but 1000 queries
collected from an office network replayed via "dig" (with duplicates and
all, so the cache is being used).

| Protocol                              | Time (Seconds)  | Privacy | DNSSEC |
|---------------------------------------+-----------------+---------+--------|
| DNS-over-TLS (Unbound+dnsfwd+stunnel) |              10 | ++      | +      |
| local Unbound with DNSSEC             |              11 | -       | +      |
| local Unbound without DNSSEC          |              11 | -       | -      |
| DNS-over-TLS (dnsfwd+stunnel)         |              21 | ++      | -      |
| DNS-over-TLS (Unbound+stunnel)        |              24 | ++      | +      |
| Google DNS (UDP)                      |              30 | --      | +      |
| DNS-over-TLS (Unbound buildin TLS)    |              40 | ++      | +      |
| DNS-over-DNSCrypt (ns0.dnscrypt.is)   |              81 | ++      | +      |
| DNS-over-Tor                          |             103 | ++      | -      |

Looks not too bad.

Carsten Strotmann