A Small History of Chrome
Discussion of all of Chrome’s goals is out of scope here, but one of the defining goals of the project was speed. Performance is a broad goal that spans all of computer science, and Chrome uses nearly every trick available, from caching to parallelization to just-in-time compilation. Then there was startup speed–how long it the program took to show up on the screen after clicking the icon–which seems a bit frivolous in comparison.
Why care about startup speed? For a browser, a quick startup conveys a feeling of lightness, that doing something on the web is as trival an action as opening a text file. Further, the impact of latency on your happiness and on losing your train of thought is well-studied in human-computer interaction. Latency is especially a focus of web companies like Google or Amazon, who are in a good position to measure and experiment on the effect of latency–and who have done experiments that show that delays of even milliseconds have measurable effects on how frequently people use the site or make purchases. It’s a small frustration that adds up subconsciously.
Chrome’s approach to starting quickly was a clever trick by one of the first engineers on Chrome. As soon as they got their skeleton application to the point where it showed a window on the screen, they created a benchmark measuring that speed along with a continuous build that tracked it. Then, in Brett Wilson’s words, “a very simple rule: this test can never get any slower.”1 As code was added to Chrome, maintenance of this benchmark demanded extra engineering effort2–in some cases work was delayed until it was truly needed, or data used during startup was precomputed–but the primary “trick” to performance, and the one that made the greatest impression on me, was simply to do less work.
I joined the Chrome team without any intention of working on build tools. My background and platform of choice was Linux, and I wanted to be the Linux guy. To limit scope the project was initially Windows-only; I took it as my role to help finish the Windows implementation so that I could then make it run on Linux.
When starting work on other platforms, the first hurdle was sorting out the build system. By that point Chrome was already large (complete, in fact–Chrome for Windows was released in 2008 before any ports had started), so efforts to switch even the Visual Studio-based Windows build to a different build system wholesale were conflicting with ongoing development. It felt like replacing the foundation of a building while it was in use.
Members of the Chrome team came up with an incremental solution called GYP3 which could be used to generate, one subcomponent at a time, the Visual Studio build files already used by Chrome in addition to the build files that would be used on other platforms.
The input to GYP is simple: the desired name of the output accompanied by plain text lists of source files, the occasional custom rule like “process each IDL file to generate an additional source file”, and some conditional behaviors (e.g., only use certain files on certain platforms). GYP then takes this high-level description and generates platform-native build files.4
On the Mac “native build files” meant Xcode project files. On Linux, however, there was no obvious single choice. The initial attempt used the Scons build system, but I was dismayed to discover that a GYP-generated Scons build could take 30 seconds to start while Scons computed which files had changed. I figured that Chrome was roughly the size of the Linux kernel so the approach taken there ought to work. I rolled up my sleeves and wrote the code to make GYP generate plain Makefiles using tricks from the kernel’s Makefiles.
Thus I unintentionally began my descent into build system madness. There are many factors that make building software take time, from slow linkers to poor parallelization, and I dug into all of them. The Makefile approach was initially quite fast but as we ported more of Chrome to Linux, increasing the number of files used in the build, it grew slower.5
As I worked on the port I found one part of the build process especially frustrating: I would make a change to a single file, run make, realize I’d left out a semicolon, run make again, and each time the wait would be long enough that I would forget what I was working on. I thought back to how hard we fought against latency for end users. “How can this be taking so long,” I’d wonder, “there can’t be that much work to do.” As an experiment I started Ninja, to see how simple I could make it.