"in the 10+ years since the feature was added, we now find that half of the DNS root server traffic is very likely due to Chromium’s probes. That equates to about 60 billion queries to the root server system on a typical day."

This is absolutely bonkers.

· · Mastodon Twitter Crossposter · 6 · 92 · 44

The tl;dr on that is that Chrome's omnibox tries to guess whether you're doing a single-word Google search or trying to visit a single-word intranet address, so it does a DNS lookup for the word.

Show thread

But because some ISPs return IPs for nonexistent domains in order to serve advertisements, Chrome makes 3 gibberish requests anytime there's a network change to identify whether the ISP is doing that.

Show thread

SIXTY BILLION DNS requests a day because fewer than 1% of all users ever visit an intranet but Chrome still wants to ask "did you mean http://viagara?" but wants to be sure it actually exists first.

Show thread

That isn't 60 billion requests to ISP-level DNS servers, those 60 billion requests makes it all the way down to root servers, meaning the number of requests to ISP nameservers is much higher.

Show thread

After everyone pees themselves while you tell that earlier Firefox nightmare story around your campfire, you can whip out this Chromium story and they'll never sleep again. You won't even need the flashlight under your chin.

Show thread

@nyquildotorg Meanwhile I find asking if there's a dot in that section of the URL is a perfectly effective hueristic for this task...

@alcinnz yeah, except that a lot of intranet URLs don't have dots, and they inexplicably think it's more important tonhandle that case than to not be assholes to the DNS system 🤷‍♂️

@nyquildotorg What do you expect? They're Google, they don't understand the constraints others operate under!

Maybe I should ask users if this is a subnet domain... Though users can work around my hueristic by prepending HTTPS...

fred at google: hmm we sure open up the gmail dev server just called "gmail1" a lot, but our browser just does a search for live gmail. you think it'd be a problem if we changed browsers everywhere to first go to our internal google server

stan at google: that's perfectly reasonable. i mean we have named servers for like 200 different google product components on this one campus

stacy at google: good news everyone, we just bought our own whole .google tld to solve the server problem

i know that's not how it happened and the real story of how google treats its employees is more tragic but

this is what looking at all the things google does from the outside first makes me imagine

@nyquildotorg @alcinnz to be fair the article doesn't actually express even the author's opinion of whether or not the behavior is appropriate or an undue burden on the root nameservers.

Which is quite possibly intentional / the style of the APNIC blog (hilariously posting a comment opens a ticket in their bug tracker).

I am optimistic that Google/Chromium folks would be open to behavioral adjustments and likely were not aware of the degree of impact here.

@nyquildotorg @alcinnz (obnote: my personal opinions here, not official statements of Google policy, etc, I don't work on Chromium or have any special knowledge about this)

@tw @alcinnz yeah, "being an asshole to the DNS system" is my viewpoint, not the author's. Because they're being an asshole to the DNS system :)

@nyquildotorg @tw @alcinnz The author of the code finally explained in the comments why it was that way:

Peter Kasting
August 22, 2020 at 3:41 am

I’m the original author of this code, though I no longer maintain it.

Just want to give folks a heads-up that we’ve been in discussion with various parties about this for some time now, as we agree the negative effects on the root servers are undesirable. So in terms of “do the Chromium folks know/care”; yes, and yes.

This is a challenging problem to solve, as we’ve repeatedly seen in the past that NXDOMAIN hijacking is pervasive (I can’t speak quantitatively or claim that “exception rather than the norm” is false, but it’s certainly at least very common), and network operators often actively worked around previous attempts to detect it in order to ensure they could hijack without Chromium detecting it.

We do have some proposed designs to fix; at this point much of the issue is engineering bandwidth. Though I’m sure that if ISPs wanted to cooperate with us on a reliable way to detect NXDOMAIN hijacking, we wouldn’t object. Ideally, we wouldn’t have to frustrate them, nor they us.

@trebach @tw @alcinnz or they could just get rid of the "did you mean?" prompt and 100% solve the problem

@tw @alcinnz I also hope someone on the Chromium team learns about this and that that will get it changed. I'm sure it wasn't malicious, but it is ridiculous.

@nyquildotorg @alcinnz I'm on vacation this week but plan to make sure folks I know there hear about it next week if they haven't already

@nyquildotorg @tw @alcinnz From the blog post: "...Are other approaches feasible? For example, Firefox’s captive portal test uses delegated namespace probe queries, directing them away from the root servers towards the browser’s infrastructure."
So Mozilla team have seen the implications of this. Google team didn't. They are just too busy to claim how great they are...


> asking if there's a dot in that section of the URL

I would also think requiring e.g. "http://router" in order to visit a hosts entry named "router" would work fine.

I like to make use of 'word' addresses in my hosts file so i can ssh to my server machine & others by their hostnames, but I wouldn't be upset at having to put http:// in front of them outside of places like the terminal that don't expect you to be doing web searches

@bhtooefr That /is/ ridiculous. I wonder if it really impacts the DNS service at all though. Wouldn't we have just built out bandwidth capacity to handle all that junk?

@tw right? I dont understand how anyone could have signed off on this.

@nyquildotorg I mean, probably no one did, and if they did they almost certainly weren't thinking/expecting it to be that significant. I wonder if the authors have brought this to the attention of folks in Chromium.

@nyquildotorg Google trying to break public DNS!? No way! What a surprise... :flan_sarcazilla:

@nyquildotorg they want to control DNS by breaking everyone else's DNS. an old Microsoft tactic: poison the pool, control the cure.

@uuim I'm pretty sure this was just an action they didn't fully understand the ramificatgions of rather than trying to break DNS.

Sign in to participate in the conversation

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!