Subject: [Add] Chromecast using 8.8.8.8

From: Paul Hoffman <[email-masked]>
Date: Fri, 9 Aug 2019 02:03:42 +0000

Greetings again. Earlier on the list, it was discussed that Google's Chromecast device uses 8.8.8.8 for DNS service regardless of the settings that were provided by DHCP from the wireless access point. I bought a new one and captured the DNS traffic, and that statement is oddly approximately half-true.

For every DNS lookup, the Chromecast sends the the same query to 8.8.8.8 *and* the resolver that it was given by DHCP. A popular way to control a Chromecast is with the GoogleHome application on a phone. On my iPhone on the same wireless access point, GoogleHome always used the resolver from DHCP. Note that GoogleHome gave the Wifi password to the Chromecast during setup, so it could have passed other settings like the resolver address, but appears not to have.

The Chromecast's use of 8.8.8.8 continued after setup and reboot, so it was not just a fluke of the setup process. There is no way in GoogleHome that I could find to set the network settings for the Chromecast other than to change the wireless access point. (The tinkerer in me was tempted to set up a resolver that sometime changes the addresses it sends out and see what the Chromecast does with the differing information, but that's beyond the scope of this problem.)

For comparison, I installed a new Amazon Firestick, and all DNS went to the resolver configured from DHCP. 

I'm not sure how to categorize the Chromecast's odd behavior, but it certainly is sending queries to a DNS resolver that is not expected by the network administrator. I'd be interested to hear from folks on this list if they know of other devices or applications that by default hard-code a different resolver address than the one that comes from the operating system (for applications) or DHCP (for devices).

--Paul Hoffman

---

From: Brian Dickson <[email-masked]>
Date: Thu, 8 Aug 2019 21:12:30 -0700

See Geoff Huston's excellent presentation on similar/related observations
(I saw him present something like this at RIPE this year):
https://www.slideshare.net/apnic/oard-30-what-part-of-no-is-so-hard-for-the-dns-to-understand

Another presentation at the same RIPE or adjacent OARC meeting, explained
the observations: some operators in China are deliberately sending the DNS
queries to both the original resolver, and another resolver, so that
duplicate answers are received, in order to improve performance. First
answer received wins, the second is ignored.

It is a pretty weird thing to do, although arguably reduces the need for
retries...

It only works for UDP/53, BTW.

Brian

On Thu, Aug 8, 2019 at 7:03 PM Paul Hoffman <[email-masked]> wrote:

> Greetings again. Earlier on the list, it was discussed that Google's
> Chromecast device uses 8.8.8.8 for DNS service regardless of the settings
> that were provided by DHCP from the wireless access point. I bought a new
> one and captured the DNS traffic, and that statement is oddly approximately
> half-true.
>
> For every DNS lookup, the Chromecast sends the the same query to 8.8.8.8
> *and* the resolver that it was given by DHCP. A popular way to control a
> Chromecast is with the GoogleHome application on a phone. On my iPhone on
> the same wireless access point, GoogleHome always used the resolver from
> DHCP. Note that GoogleHome gave the Wifi password to the Chromecast during
> setup, so it could have passed other settings like the resolver address,
> but appears not to have.
>
> The Chromecast's use of 8.8.8.8 continued after setup and reboot, so it
> was not just a fluke of the setup process. There is no way in GoogleHome
> that I could find to set the network settings for the Chromecast other than
> to change the wireless access point. (The tinkerer in me was tempted to set
> up a resolver that sometime changes the addresses it sends out and see what
> the Chromecast does with the differing information, but that's beyond the
> scope of this problem.)
>
> I'm not sure how to categorize the Chromecast's odd behavior, but it
> certainly is sending queries to a DNS resolver that is not expected by the
> network administrator. I'd be interested to hear from folks on this list if
> they know of other devices or applications that by default hard-code a
> different resolver address than the one that comes from the operating
> system (for applications) or DHCP (for devices).
>
> --Paul Hoffman
> --
> Add mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/add
>

---

From: Rob Sayre <[email-masked]>
Date: Thu, 8 Aug 2019 23:45:39 -0700

On Thu, Aug 8, 2019 at 7:03 PM Paul Hoffman <[email-masked]> wrote:

> it certainly is sending queries to a DNS resolver that is not expected by
> the network administrator.
>

Thank you for the detailed research.

I have an older Chromecast that set me back about $35, and I have yet to
get it to work on a captive portal requiring an HTML sign in. I carry it in
my luggage--when it works (over WPA etc), it's great.

I'm not sure everyone views "network administrator" as a valid role, at
least in contexts where the administrator does not also control the device.

thanks,
Rob

---

From: Alec Muffett <[email-masked]>
Date: Fri, 9 Aug 2019 08:29:19 +0100

> It is a pretty weird thing to do, although arguably reduces the need for

Um. That's not a performance optimization. That's part of the Great
Firewall's enforcement of censorship, tracking, and detection of
circumvention.

There's a very good video from CCC2015 which explains how this works:

https://youtu.be/zcC5K7QTdvM

- what you are describing is explained in the first third or so, but the
whole thing is worth watching. Doubtless the whole thing has advanced in
the past few years.

Perhaps for compliant citizens there might be some performance win, but
censorship is a high price to pay.

Aside: is it bad that I am horrified to be on an IETF maillist where some
are railing against the value proposition of DoH, and others believe that
the Great Firewall of China is a weird DNS performance hack?

- a

---

From: "Ralf Weber" <[email-masked]>
Date: Fri, 09 Aug 2019 15:18:42 +0200

Moin!

> least in contexts where the administrator does not also control the 
Really. I have been working at ISPs and am used to being told that ISPs 
are bad for the internet in an IETF context, but that really takes it to 
a whole new level. I’m pretty sure you would not be able to write this 
email if all people administering networks where they don’t control 
the end devices would stop working.

And thanks to Paul for investigating this. To me this means that Google 
sees all your video sites if you use a Chromecast. While I don’t own 
one I know a couple of people who do and as a network administrator will 
help them to block that.

So long
-Ralf
—--
Ralf Weber

---

From: Bret Jordan <[email-masked]>
Date: Fri, 9 Aug 2019 08:24:54 -0600

Yeah, there is a lot of very expensive equipment and people that are needed to make this thing work day-2-day. Lots of people here complain about the controls hotels put on _their_ network.  But have you ever tried to use one that has 3000 ms latency and 50% packet loss?  If not, I have one for you in Amsterdam that you can go stay at.  You tend to be very _grateful_ for networks that do work, even if they have some hoops you have to jump through or if you can not access some content. 

Thanks,
Bret
PGP Fingerprint: 63B4 FC53 680A 6B7D 1447  F2C0 74F8 ACAE 7415 0050
"Without cryptography vihv vivc ce xhrnrw, however, the only thing that can not be unscrambled is an egg."

> On Aug 9, 2019, at 7:18 AM, Ralf Weber <[email-masked]> wrote:
> 
> Moin!
> 
> Really. I have been working at ISPs and am used to being told that ISPs are bad for the internet in an IETF context, but that really takes it to a whole new level. I’m pretty sure you would not be able to write this email if all people administering networks where they don’t control the end devices would stop working.
> 
> And thanks to Paul for investigating this. To me this means that Google sees all your video sites if you use a Chromecast. While I don’t own one I know a couple of people who do and as a network administrator will help them to block that.
> 
> So long
> -Ralf
> —--
> Ralf Weber
> 
> -- 
> Add mailing list
> [email-masked]

---

From: Paul Wouters <[email-masked]>
Date: Fri, 9 Aug 2019 10:44:10 -0400 (EDT)

> For every DNS lookup, the Chromecast sends the the same query to 8.8.8.8 *and* the resolver that it was given by DHCP. A popular way to control a Chromecast is with the GoogleHome application on a…

This is not the behaviour I am seeing on my Chromecast Ultra. Local DNS
is completely ignored and never queried, and the only DNS packets I see
go to 8.8.8.8.

Paul

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 9 Aug 2019 14:48:46 +0000

> is completely ignored and never queried, and the only DNS packets I see

Thanks for the report! I'll order one now and make some pcaps.

I'm still quite interested in other reports of devices and applications that by default don't use the configured resolver. Hopefully they are not too expensive... 

--Paul Hoffman

---

From: Michael Richardson <[email-masked]>
Date: Fri, 09 Aug 2019 11:32:11 -0400

Paul Wouters <[email-masked]> wrote:
    >> For every DNS lookup, the Chromecast sends the the same query to
    >>8.8.8.8 *and* the resolver that it was given by DHCP. A popular way to
    >>control a Chromecast is with the GoogleHome application on a phone. On
    >>my iPhone on the same wireless access point, GoogleHome always used the
    >>resolver from DHCP. Note that GoogleHome gave the Wifi password to the
    >>Chromecast during setup, so it could have passed other settings like the
    >>resolver address, but appears not to have.

    > is completely ignored and never queried, and the only DNS packets I see

I wonder if there was something about your local DNS resolver that it didn't
like, and if it "gave up" on it.  What happens if you block it's 8.8.8.8
access?

Oh, to have a MUD file for these devices.

--
Michael Richardson <[email-masked]>, Sandelman Software Works
 -= IPv6 IoT consulting =-

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 9 Aug 2019 16:28:53 +0000

On Aug 9, 2019, at 8:32 AM, Michael Richardson <[email-masked]> wrote:
> like, and if it "gave up" on it.

That's not what the pcaps look like. It immediately started sending the dual queries, and kept up that action throughout, even after reboot. 

>  What happens if you block it's 8.8.8.8
> access?

Dunno. My research questions was "does the Chromecast always send queries to 8.8.8.8", not "how can system admins mess with it". I'll maybe look later into other related questions.

--Paul Hoffman

---

From: Paul Wouters <[email-masked]>
Date: Fri, 9 Aug 2019 12:42:04 -0400 (EDT)

> like, and if it "gave up" on it.

Unless it remembers that over reboots, no. It simply does not send a
single packet to my resolver. It does send pings to 8.8.8.8 and the
network's default gateway, but nothing to my DNS servers.

Paul

---

From: Rob Sayre <[email-masked]>
Date: Fri, 9 Aug 2019 11:26:45 -0700

On Fri, Aug 9, 2019 at 7:49 AM Paul Hoffman <[email-masked]> wrote:

> I'm still quite interested in other reports of devices and applications
> that by default don't use the configured resolver. Hopefully they are not
> too expensive...
>

It looks like Roku set top boxes and some apps on them (e.g. Netflix) also
do this. Searching for "Roku 8.8.8.8" should turn up reports.

thanks,
Rob

---

From: Brian Dickson <[email-masked]>
Date: Fri, 9 Aug 2019 11:27:15 -0700

I don’t have one, but I am wondering about how it uses 8.8.8.8, ie if it somehow authenticates its identity, or if it trusts any answer from 8.8.8.8.

Maybe you could stand up a fake 8.8.8.8 including any necessary routes, and see if it accepts it?

Inquiring minds want to know...

Brian 

Sent from my iPhone

> On Aug 9, 2019, at 7:48 AM, Paul Hoffman <[email-masked]> wrote:
>

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 9 Aug 2019 19:05:52 +0000

On Aug 9, 2019, at 11:26 AM, Rob Sayre <[email-masked]> wrote:
> It looks like Roku set top boxes and some apps on them (e.g. Netflix) also do this. Searching for "Roku 8.8.8.8" should turn up reports.

Thanks! I now have one on order for testing as well. I hadn't thought of testing the individual apps, but I will do that also. (I am failing at keeping the "lab" aspect of my work in the office and out of the living room...)

--Paul Hoffman

---

From: Michael Richardson <[email-masked]>
Date: Fri, 09 Aug 2019 16:30:39 -0400

> That's not what the pcaps look like. It immediately started sending the
    > dual queries, and kept up that action throughout, even after reboot.

yes, I was actually asking Paul Wouters about his Ultra.

---

From: Daniel Stenberg <[email-masked]>
Date: Fri, 9 Aug 2019 23:08:36 +0200 (CEST)

On Fri, 9 Aug 2019, Brian Dickson wrote:

> I don’t have one, but I am wondering about how it uses 8.8.8.8, ie if it 
> somehow authenticates its identity, or if it trusts any answer from 8.8.8.8.
>

We're possibly not quite on topic here anymore, but...

There are plenty of people online [1] at least saying they could just redirect 
the traffic to their own resolver [2].

[1] = https://news.ycombinator.com/item?id=19170671
[2] = https://ba.net/adblock/vpn/roku-chromecast-fix.html

-- 

  / daniel.haxx.se

---

From: Rob Sayre <[email-masked]>
Date: Fri, 9 Aug 2019 14:30:42 -0700

On Fri, Aug 9, 2019 at 2:09 PM Daniel Stenberg <[email-masked]> wrote:

> > Maybe you could stand up a fake 8.8.8.8 including any necessary routes,
> and
> > see if it accepts it?
>

If the devices used DoT/DoH, I don't think this technique would work, hence
the investigation.

The most common reason to make these changes seems to be to circumvent
region-based content restrictions on streaming video.

thanks,
Rob

---

From: Daniel Stenberg <[email-masked]>
Date: Fri, 9 Aug 2019 23:50:54 +0200 (CEST)

> If the devices used DoT/DoH, I don't think this technique would work, hence 

Yes, DoT and DoH authentication will of course fail after such tricks. But 
these boxes (obviously) don't use those protocols (yet)...

-- 

  / daniel.haxx.se

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Sun, 11 Aug 2019 22:10:32 +0000

> like, and if it "gave up" on it.  What happens if you block it's 8.8.8.8 access?

I was curious, too, what would happen if the address was blocked. So I tried. 
When I told the firewall to drop all packets to 8.8.8.8, my Chromecast just kept trying to reach 8.8.8.8. I gave up after several minutes (it hadn't given up).
No attempt was made to use the resolver advertised by the router (and no attempt was made when 8.8.8.8 packets were accepted).
When I told the firewall to reject packets (ICMP unreachable response) destined for 8.8.8.8, it did try the locally advertised resolver (and seemed to get good responses). It never gave up trying 8.8.8.8 every time it wanted to resolve something. But the casting failed. Chromecast complained of a failure (displayed a message on the TV screen), but didn't explain what failed. 

Just thought I'd share in the fun.
Barbara

---

From: Andrew Campling <[email-masked]>
Date: Mon, 12 Aug 2019 21:16:32 +0000

> STARK, BARBARA H <[email-masked]> wrote:
> When I told the firewall to reject packets (ICMP unreachable response) destined for 8.8.8.8, it did try the locally advertised resolver (and seemed to get good responses). It never gave up trying 8.8.8.8 every 
> time it wanted to resolve something. But the casting failed. Chromecast complained of a failure (displayed a message on the TV screen), but didn't explain what failed. 

> Just thought I'd share in the fun.

I guess we'll be seeing more badly behaved devices and apps as DoH is implemented, except of course that it will be harder to detect and stop them. This really underlines why DoH is a poor choice vs DoT or DNS53 in terms of user choice and control.  

Andrew