compilers and abstraction

We really like compilers. They are, broadly, extremely impressive software which solves really cool problems. However, that’s only part of the reason why we find the problem space interesting.

We were thinking about “artificial intelligence” and the way it has taken the world by storm (to my endless disdain). It’s rather disappointing the direction that we as a collective have taken this degree of automation—towards creative work, while leaving humans to handle menial tasks.¹ This is, however, an entirely unsurprising direction. The digital world, into which we’ve integrated generative “AI”, is incredibly abstracted, and interfacing with it is (relatively) simple and reliable.² Meanwhile, the “real world” is deeply complex and unpredictable. As programmers, we love to condemn global mutable state because it brings complexity, and “reality” is simply a massive pile of that. When it comes to software, we tend to do as much as possible to avoid dealing with the real world, to varying degrees of success. Certain field have less luck than others when it comes to doing so (robotics being a prime example).

Compilers, on the other hand, have a great deal of luck when it comes to this. The entire problem space of a compiler fits on a single machine. They can be reasonably modeled as pure functions which take some text files as input and produce compiled artifacts as output.³ They often even make explicit commitments to do no more, restricting themselves to that singular interface. That design also tends to make them fairly easy to test.

The input and output often make this easier as well. The input to a compiler tends to be fairly well-defined, and designed in a way that facilitates machine interpretation. Despite the input medium being incredibly free-form (unstructured text), assuming that the user is technical affords a great deal of forgiveness when it comes to interpretation.⁴ Similarly, the outputs are purely digital; compilers must produce code that tells the machine––another generally well-defined interface—to do what its told, without having to directly interact with the surrounding world.

Even in the field of compilers and developer tools, many of the hard problems are the ones which lack these properties. Diagnostics are notoriously hard to do right, both from a technical perspective and a communicative one. Formatting is another, given that the task is inherently subjective and the output is made for humans, not machines.

We don’t mean to imply that compilers are easy or simple. They’re not. But it is the case that part of the reason we find compilers as compelling as we do is because that particular vector of complexity isn’t always something we love dealing with, and compilers occupy a niche where we don’t have to.