Use a .dev domain? Not anymore.
For years, programmers used .dev domains for testing their code. Those days are over.
In the course of my work I sometimes test features on unreleased, upcoming versions of browsers. Earlier this week I opened up the upcoming version of Chrome only to find that the Medium’s development environment — the system we use to build Medium every day — wouldn’t load. Instead, all I saw was this error:
Our development environment worked in the stable version of Chrome, so there had to be something different about this new version. After some sleuthing, I figured out what had changed.
Our development domain name, standard.dev, wouldn’t work in the new version of Chrome unless we accessed it through a secure connection or changed the name. As I read more about the issue I learned that it’s not just Chrome, and we’re not the only ones who face this problem.
Every programmer who tests their website using a domain that ends in .dev will be affected. In every major browser. Within weeks, thousands of development websites will stop loading, reduced to frowny page icons.
This is the story of that change. The story of how the internet grows and bends and brushes up against its past from time to time, sending off sparks. It begins in 1969.
A long time ago, in a network, far far away, a great adventure took place!
Out of the chaos of new ideas for communication, the experiments, the tentative designs, and crucible of testing, there emerged a cornucopia of networks. Beginning with the ARPANET, an endless stream of networks evolved, and ultimately were interlinked to become the Internet.”
RFC 2468, Part 1
Before the internet, before domain names, and before I was born, there was the ARPANET — the little network that could. What started as a network that bridged four research centers in the western United States became the foundations of the internet we know today.
The ARPANET connected computers called hosts together using giant, wired routers called Interface Message Processors (IMPs). These routers served as the nodes of the network, and allowed the research centers to communicate with one another.
To send a message to another host on the network you needed to know its address, the unique number of the host and IMP. An address was like a telephone number: meaningless and impossible to remember. So researchers solved the problem the same way our phones do — they created an address book. This address book was called a hosts file.
By 1973, the ARPANET had grown tenfold, spanning the United States and adding IMPs across the Atlantic in Norway and London. Host files were becoming unweildly. Programmer and composer L. Peter Deutsch wrote,
“It seems about time to put an end to the absurd situation where each site on the network must maintain a different, generally out-of-date, host list for the use of its own operating system or user programs.”
Instead, ARPANET researchers decided to give each host an official name and distribute one master hosts file, HOSTS.TXT, maintained by the researchers at the Network Information Center (NIC). This change was critical because it changed the purpose of the hosts file. Instead of acting like your cell phone’s list of contacts, it acted as a public phone book.
As the network grew, even HOSTS.TXT became unwieldy. In the mid-1980s, researchers created a new, decentralized system for managing and organizing host names at scale. They called this the Domain Name System (DNS), and it introduced two key concepts: hierarchy and resolution.
The first major concept of DNS introduced was a new, hierarchical style of domain names. Instead of navigating to an address like NYT-COOKING, you’d navigate to cooking.nytimes.com. This would allow related websites to be grouped together. In our New York Times example, their main site is www.nytimes.com, their cooking app is cooking.nytimes.com, and their Medium publication is open.nytimes.com. With the DNS hierarchy, it’s clear that all of these websites are related.
But what about the “.com” part? That’s a top-level domain (TLD). DNS was created with only a handful of top-level domains, served to classify sites by purpose. Names like .gov, .edu, and .org are fairly straightforward. And .com, the most ubiquitous TLD of them all? .com stands for “commercial.” The process to create a new TLD was exceptionally strict, and would remain that way for almost thirty years.
Creating TLDs also served another purpose — they facilitated the migration from ARPANET addresses to proper domain names. Every host name on the ARPANET was given a temporary domain name with the .arpa TLD. Domains like NYT-COOKING automatically became nyt-cooking.arpa when accessed using DNS, so the entirety of the ARPANET was still accessible when DNS was first implemented.
The second major concept of DNS introduced was domain name resolution. Instead of relying on a hosts file on your own computer, your computer would ask a domain name server to resolve a name into an address on the network. That way, the address was (almost) always up to date.
But the hosts file persisted. It remained your own personal list of addresses, used to augment the DNS results. Programmers could add an entry to the hosts file if they wanted to override the domain name server, which became especially useful for testing network-related code.
The introduction of domain name servers also introduced a problem. What if you were writing code (or later, building a website) that wasn’t connected to the network yet? How could you test it? Programmers introduced the concept of a loopback address to solve this problem. With a loopback, a host could send traffic to an address and receive that same traffic as if it had come from another machine. This allowed them to test their code without connecting to any name servers.
When IPs were introduced, the loopback address was named localhost and added to every computer’s host file. It remains the de facto address for testing websites to this day.
When Medium was created, we used localhost for the address of our development server. It was simple, ubiquitous, and easy to set up. But as Medium grew from a basic website into a network, our needs changed.
By 2015, our use of localhost was causing issues. When we added a way for users to log in to Medium through Facebook, Facebook wouldn’t allow us to test the feature with localhost as a URL. At the same time, we decided to allow publications to write on Medium using their own domain name instead of medium.com. Splitting our platform across multiple domains was not a small amount of engineering work. This included adding multiple fake domains into our development environment to test our new code and mimic the real world.
We decided to use our hosts file (and later, a fake domain name server called dnsmasq) to create fake custom domains and transform localhost into standard.dev.
We’re not the only ones to rename localhost. Over the years web services became increasingly complex, requiring programmers to simultaneously run multiple interconnected services to test their code.
localhost began to lose its significance. Each service runs on a different port, written as a number at the end of the domain name. Your website might run on localhost:3000, your interactive tests at localhost:4002, your events service on localhost:5000, etc. At some point the ports become meaningless numbers. So, like phone numbers and IP addresses, we give our services names.
These names don’t point to real websites, but that’s why we have a hosts file. We add our new names to the hosts file and override the domain name server for testing. (This isn’t the only way to solve this problem — there are many solutions at varying degrees of scale, but we’ll save them another story.)
A handful of engineers chose standard.dev because .dev was a common suffix, and our engineering configurations already supported it. Mostly, the engineer who proposed the name enjoyed the pun on “standard deviation.”
Mission accomplished. 🇺🇸
Almost three years later, a commit to Chromium (the open source project behind Chrome) would throw sand in the gears of our development environment. If we didn’t fix the issue before it spread, our development website would stop loading in major browsers and potentially disrupt engineering across the organization. Frowny page icon, indeed.
What changed? Chromium forced all domains with the .dev TLD to only respond to secure connections. Let’s break this down. What is a secure connection, and why would a browser force a website to be secure?
Security wasn’t baked into the original blueprints of the internet. In 1994, HTTPS was created as a secure, encrypted alternative to HTTP, the default, unencrypted communication standard. (The ‘S’ stands for secure.) At first, HTTPS was primarily used for financial transactions. As the internet wove through the fabric of society the stakes became higher, and privacy became increasingly important to the public. By early 2017, over half of traffic in both Firefox and Chrome was encrypted.
HTTPS evolved since 1994, continually playing a cat-and-mouse game against security vulnerabilities. In 2009, security researcher Moxie Marlinspike introduced a technique called SSL-stripping, which silently downgrades a HTTPS connection into an insecure HTTP connection. To counter this attack, browsers introduced HSTS (HTTPS Strict Transport Security), a way for a website to declare that communications should always be secure and use HTTPS.
But HSTS came with a chicken-and-egg problem. How will the browser know that all communications should be secure if it hasn’t communicated with the website before? The browser’s first request to the website might be unencrypted, at which point the website will reply that the browser should encrypt everything. By that point, it might be too late.
Browsers added a solution for this too. The Chromium team maintains a list of websites that should always have HSTS enabled, and they bake that list into the browser. That way the browser knows all communication with the website should be secure before the first request occurs. They call this the HSTS preload list.
Every major browser includes an HSTS preload list based on the list maintained by the Chromium team. When a change is committed to the Chromium list, it slowly trickles out to Chrome, Firefox, Opera, Safari, IE 11, and Edge.
How familiar. Chromium’s HSTS preload list is HOSTS.TXT all over again — except this time instead of host names and network addresses, the list is full of domain names and security instructions. The list is also subject to many of the same issues as the original HOSTS.TXT. It takes time to distribute the list to each browser, and time for the browser updates to trickle down to us, the users. We’re also starting to see the effects of scale. Today, this list is a cumbersome 41,000 lines long. (Medium is on the list, by the way.)
This brings us back to the Chromium commit. The commit behind all this ruckus was a simple change: engineers added one new line to the list. That line forces secure communications for the entire .dev TLD. Encryption just became mandatory for every website that ends in .dev.
It makes sense to force secure connections for a website and all its subdomains, but why force connections for a whole TLD? How can anyone even enforce that? And of all the domains, why the .dev TLD?
“Someone had to keep track of all the protocols, the identifiers, networks and addresses and ultimately the names of all the things in the networked universe. And someone had to keep track of all the information that erupted with volcanic force from the intensity of the debates and discussions and endless invention that has continued unabated for 30 years. That someone was Jonathan B. Postel, our Internet Assigned Numbers Authority.”
RFC 2468, Part 2
After the DNS was introduced in 1985, the responsibility of managing hosts and addresses didn’t disappear, it shifted. Each top-level domain was managed by a “registry” who decided how its domains would be allocated and registered. But who managed the TLDs? That fell to IANA — the Internet Assigned Numbers Authority. (If there’s anything to take from internet history, it’s that all early technical organizations use abbreviations that unfurl into dry, literal names. There’s a kind of humor in it. Given the internet’s roots in academia and government, I can’t say I’m surprised.)
IANA was first established in 1988. The name sounds grandiose, but this authority was the province of just two people — Jon Postel and Joyce K. Reynolds. Postel was a key figure in the creation of the internet: he was the editor of the Request for Comments (RFC) series that documented and specified the internet since its inception in 1969 and the co-author of the foundational RFCs that described the DNS. Postel had been informally handling the names and numbers since the early days of the ARPANET, so at first IANA was practically an honorary title.
As the internet grew, the duties of IANA became increasingly critical. After a botched initial attempt to privatize domain name allocation in 1994, the management of domain names and their addresses was handed off to a new, nonprofit organization in December 1998. This organization was aptly named ICANN, the Internet Corporation for Assigned Names and Numbers.
Sadly, Postel did not live to see IANA integrated into ICANN. He died in October 1998 of complications from heart surgery, and Reynolds continued their work as a part of ICANN.
Vint Cerf, a fellow internet pioneer, memorialized Postel in RFC 2468, entitled “I REMEMBER IANA.”
“Jon, our beloved IANA, is gone. He has left a monumental legacy for all Internauts to contemplate. Steadfast service for decades, moving when others seemed paralyzed, always finding the right course in a complex minefield of technical and sometimes political obstacles.”
RFC 2468, Part 3
One of the principal disputes surrounding DNS was the management of TLDs. Adding country-code domains like .fr or .uk was straightforward, but how do you decide what to do with other domains? These domains, known as generic TLDs (gTLDs), were trickier. In 1998, the US Dept of Commerce wrote a white paper advising the new organization on this problem:
“The challenge of deciding policy for the addition of new domains will be formidable.
At least in the short run, a prudent concern for the stability of the system suggests that expansion of gTLDs proceed at a deliberate and controlled pace to allow for evaluation of the impact of the new gTLDs and well-reasoned evolution of the domain space.”
In the short run, this remained the case. But in 2011, that all changed. The ICANN board of directors voted to lift almost all restrictions on TLDs, paving the way for almost any word to be registered as a TLD.
You might be familiar with all the new and fancy domain names going around: things like medium.engineering, pizza.cool, and cool.pizza. These are the byproduct of this change. Each gTLD is operated by a registry, who decides how their domain names are allocated and managed. The decision to lift restrictions on TLDs resulted in an explosion of registries and a TLD gold rush. (Unlike the California gold rush, this one came with a $185,000 application fee — not to mention requiring the technical knowhow and funding necessary to run a registry in the first place.)
The .dev TLD is just one of over one thousand top-level domains added to the internet since 2013.
A week ago, I didn’t even know .dev was a real gTLD. Historically it’s just been the realm of programmers who need a fake domain for testing. The domain never really existed, we just told our computers to pretend it does.
But the .dev gTLD does exist. And guess who owns it?
That’s right.
It’s Google.
Suddenly, it all makes sense. Who can decide to make an entire TLD secure? The registry that manages that TLD, of course. And that’s what Google chose to do with .dev.
Why does Google own the .dev domain? Luckily, we can ask them for ourselves — those $185,000 applications for gTLDs are made public, and Google’s is a work of art.
“18(a). Describe the mission/purpose of your proposed gTLD.
The proposed gTLD will provide Google with direct association to the term “dev,” which is an abbreviation of the word, “development.” The mission of this gTLD, .dev, is to provide a dedicated domain space in which Google can enact second-level domains specific to its projects in development.”
Translation: Google wants to use .dev for themselves.
“18(b). How do you expect that your proposed gTLD will benefit registrants, Internet users, and others?
Given its intended use by Google, the .dev gTLD will best add value to the gTLD space by remaining completely closed for the sole use of Google.”
Translation: The best use of .dev is for it to belong to Google and no one else.
“18(c). What operating rules will you adopt to eliminate or minimize social costs?
Members of the public will not be able to register domain names in this new gTLD.”
Well, that’s one way to minimize social costs.
Snarkiness aside, Google has a reasonable explanation for securing .dev. They have a consistent track record of pushing for security across the internet, especially for their own data. This pushing often takes the form of acquiring infrastructure, securing it, and encouraging others to follow their lead. Securing .dev is simply an extension of that agenda.
In the late 2000’s, Google released their own web browser (Chrome), operating system (Android), and public domain name server. Within years, each would grow into the dominant player in its field. Coupled with infrastructure investments in the billions, Google began to steer the direction of the internet at large.
By the time the first gTLD applications were submitted in 2012, Google had locked down a massive portion of the world’s internet infrastructure in every arena. But they were missing a piece. Despite owning the browser and the DNS, Google didn’t own the registry, because until then it wasn’t possible to own a registry.
Operating a registry would allow Google to secure another segment of critical internet infrastructure. A system is only as secure as its weakest link, so Google aims to control and secure as many links as it can. When ICANN changed the rules, Google was ready.
When applications opened for gTLDs in 2012, Google didn’t just apply for .dev. They applied for 101 gTLDs, including .google, .play, and .app. However, Google wasn’t the only company to apply for many of these gTLDs. For some applications, it took years for applicants to negotiate who would end up with the rights to the name. Google’s application for .dev was pending for over a year. Finally, in December 2014, their application for .dev was granted.
In 2015, Chromium added the entire .google TLD to the HSTS preload list with little fanfare. It was the first and only TLD entry in the list for two years, until .dev was added in September and shortly followed by .foo, .page, .app, and .chrome — all Google-owned gTLDs.
The source code includes classic encouragement for others to follow their lead: “At the moment, this only includes Google-owned gTLDs, but other gTLDs are welcome to preload if they are interested.”
As far as I can tell, there’s no easy way around this change. Chrome’s HSTS admin panel helpfully warns, “you cannot delete preloaded entries,” and I’m inclined to believe them. The HSTS preload list ensures all communication with a website is encrypted. If you could remove a website from the list, then you could send an unencrypted request to that website. This would compromise the purpose of the list and be a major security risk.
If you’re using a .dev domain, there’s some urgency around fixing this issue. Commits take around ten weeks to land in Chrome’s stable branch and the .dev change was committed to Chromium on September 15th… ten weeks ago. The commit in question has been in the beta channel for over a month. This change will land in Chrome soon and other browsers shortly thereafter.
To solve the problem in the short term you have one of two options. You can either enable HTTPS in your development environment or change your development domain. These options may vary in difficulty depending on your circumstances.
Enabling HTTPS
Enabling HTTPS is easier when you’re dealing with a real website with a domain that you own. In that case, you can get a certificate that says “yes, I am the owner of this website and it is secure.” Unfortunately, development websites are fake, so you need to manage your own security. This involves creating what’s called a “self-signed certificate” for your development website.
Like a small child, your operating system believes in “stranger danger” and doesn’t trust self-signed certificates. This is because anyone could make one. So once you create your certificate, you also need to teach every computer that accesses the development environment to trust that certificate. (You trust yourself, right?) Otherwise your browser will yell that your development environment is unsafe and insecure and hates puppies. This can be a painful process if you don’t have a process for installing code on development machines.
In Medium’s case, when I discovered this issue we already had a working self-signed SSL certificate and a good process for managing development machines. I wrote a snippet of code to automatically trust our certificate on macOS and updated our development environment to enable HTTPS by default.
While this is a reasonable short-term solution, developing on a public, active TLD doesn’t seem like the best idea in the long-term. This solution is more of a stopgap — it’s a good idea to change your domain too.
Changing your development domain
The other option is to change your .dev domain and never look back. But what domain could we migrate to? With the gTLD gold rush, is anything safe?
As it happens, there are two contenders: .localhost and .test. In 1999 RFC 2606 reserved these, along with .example and .invalid for just this purpose. After the gTLD boom, RFC 6761 reaffirmed and clarified their usage in 2013.
There are two key differences in usage — software is encouraged to recognize .localhost domains as “special” and alter its behavior accordingly. However, software should treat .test as it would any other domain.
Keep in mind that these are guidelines, so the only way to know whether your new domain will work properly is to test it on your code, along with all the services integrated with your code.
Changing your domain might be as easy as changing a line or two in your configuration file, or as difficult as changing thousands of lines of code across your codebase. You’ll also need to make sure your computer knows how to handle .localhost and .test domains. Our favorite host file is still relevant and up for the job (and the dnsmasq tool is an excellent solution for this as well).
Those of us using .dev may feel wronged, but ultimately we were squatting on a domain that was not our own, and was not protected for our use. Now we’re paying the cost. But that’s not to say we’re at fault either.
It’s easy to place the blame on programmers. Why didn’t we just use the reserved domains in the first place? The answer is that we didn’t know — and it’s okay not to know. For all our efforts to educate ourselves, there will always be more to learn. The internet contains far more than any person can know. That’s what we built it for, after all.
It’s also easy to place the blame on Google. Who are they to lock down a frequently-used fixture in the software community? It may have been a little tone-deaf for Google to acquire .dev for internal use, but then again, if Google hadn’t enabled HSTS for the whole TLD most programmers would be none the wiser. And they weren’t the only applicant for the .dev TLD either. This is just another step down a predictable path.
The last player in this drama is ICANN, the organization that enabled this change. Lifting restrictions on gTLDs is akin to opening Pandora’s box — once you loosen constraints on a software system, it’s difficult to reverse course. ICANN could’ve written stricter rules for new gTLDs, but rules are tricky to get right. We’ve known this was a thorny issue since before ICANN existed. Conflict and a few broken eggs were inevitable.
There’s a line in I REMEMBER IANA that sticks with me. “He leaves a legacy of edited documents that tell our collective Internet story, including not only the technical but also the poetic and whimsical as well.” The days of the ARPANET seem simultaneously quaint and grandiose — a handful of people laying the foundation for a network that would go on to be used by half the world and considered a human right. Their work and the problems that come with it persist to this day.
The evolution of the internet has been a messy, incremental affair, and there’s no reason for that to change now. In the meantime, some of us have domain names to change.
