We Could Fix Everything, We Just Don’t

[programmers frantically pulling cables out of the wall]

AI: “Nuclear power. Double teachers’ salaries. Build more houses. Distribute food more fairly. TRAINS—”

— qntmyrrh (@qntm) November 24, 2023I remember growing up with that same old adage of how you could be the next scientist to invent a cure for cancer, or a solution to climate change, or whatever. What they don’t tell you is that we already have solutions for a lot of problems, we just don’t use them. Sometimes this is because the solution is too expensive, but usually it’s because competing interests create a tragedy of the commons. Most problems in the modern age aren’t complicated engineering problems, they’re the same problem: coordination failure.

It was recently unveiled that basically every single UEFI SecureBoot implementation ever made can be bypassed with a malicious image file. This means that any manufacturer that allows the user to customize the boot image is now vulnerable to a complete bypass of SecureBoot and Intel Boot Guard. Luckily, the fix for this is pretty simple: don’t make the logo customizable. But how did something this absurd happen in the first place?

The results from our fuzzing and subsequent bug triaging unequivocally say that none of these image parsers were ever tested by IBVs or OEMs. We can confidently say this because we found crashes in almost every parser we tested. Moreover, the fuzzer was able to find the first crashes after running just for a few seconds and, even worse, certain parsers were crashing on valid images found on the Internet. — binarly.ioIt’s pretty obvious what happened, actually. The image parsers were written with the assumption they’d only ever need to load an image file provided by the manufacturer. When this assumption was violated, all hell broke loose, because we don’t test software anymore. None of this happened because engineering is hard. None of this happened because of some tricky, subtle bug. It happened because the people writing the image parsers made an incredibly stupid mistake and then didn’t bother testing it, because the software industry doesn’t bother with QA anymore. Thus, there was no swiss cheese. There was just one slice of cheese with a gaping hole in it, because it turns out that some manufacturers decided to let users customize their boot image, thinking it would be harmless, and that by itself was enough to wreak havoc.

Every layer of this problem is a different flavor of coordination failure. No one on the team who implemented this either thought that there might need to be a warning about untrusted images, or whoever did bring it up was ignored because it was supposed to be handled by another team. Except whoever was supposed to put in a warning about this either wasn’t told, or buried it inside a technical document nobody ever reads. The vendors who decided to implement user-customizable boot logos didn’t ask whether this would be a problem, or weren’t told about it.

And nobody, not a single layer in this clown train, implemented a proper QA or pentesting process that could have caught this bug, because we just don’t bother testing anything anymore. Our economic incentives have somehow managed to incentivize building the worst possible piece of shit that still technically works. We know how to avoid this situation. We have decades of experience building in-depth QA processes that we are simply ignoring. We could fix this, we just don’t.

This is not exclusive to software, as this fantastic video about the popcorn button explains. Our economic race to the bottom has been sabotaging almost every aspect of engineering in our society. To save a few cents per microwave, the cheap microwaves don’t include a humidity sensor and then lie about having a popcorn button when it can’t actually work properly, which leads to everyone saying “don’t use the popcorn button” and now nobody uses the popcorn buttons even on microwaves that actually have a humidity sensor and a working popcorn button. The cheapskates control the supply chain now. They have pissed in the proverbial pool, and if this sounds familiar, that’s because it’s a classic example of the Tragedy of the Commons.

Except, that’s not an excuse. What’s truly absurd is that the tragedy of the commons isn’t inevitable. We know this because ancient human tribes managed to navigate responsible utilization of common resources all the time. It has no historical basis whatsoever. The tragedy of the commons only happens when you have a total failure of collective action. It is the original symptom of societal enshittification.

[…] many nomadic pastoralist societies of Africa and the Middle East in fact “balanced local stocking ratios against seasonal rangeland conditions in ways that were ecologically sound”, reflecting a desire for lower risk rather than higher profit…We actually have a cure for blood cancer now, by the way. Like, we’ve done it. It’s likely that a similar form of immunotherapy will generalize to most forms of cancer. Unfortunately, the only approved gene therapy we have is for sickle-cell disease and costs $2 million per patient, so most people in America simply assume they will never be able to afford any of these treatments, even if they were dying of cancer, because insurance will never cover it. This is actually really bad, because if nobody can afford the treatment, then biotech companies won’t bother investing into it, because it’s not profitable! We have built a society that can’t properly incentivize CURING CANCER. This is despite the fact that socialized healthcare is a proven effective strategy (as long as the government doesn’t sabotage it). We could fix this, we just don’t.

Some people try to complain that this happens because democracy is hard, or whatever, and they’re also wrong. We know exactly what’s wrong with our current voting systems and CGP Grey even put out a video on it 13 fucking years ago. It inevitably results in a two-party system, because strategic voting is rational behavior, and you can’t break out of this two-party system because of the spoiler effect, and the solution is Ranked Choice Voting (or the Alternative Vote). If you want to go further and address gerrymandering you can use the Single Transferable Vote. All of these better systems were proposed decades ago. We have implemented exactly none of them for the presidential election (except for Maine and Alaska). In fact, America still uses the electoral vote system, which is strictly worse than the popular vote, we all know it’s worse, and we even have a potential solution but we still can’t get rid of it due to counterproductive societal interests.

We HAVE solutions for these problems. We just don’t use them. We could be running fiber-optic cable to every house in America, and we even know how much it would cost. We just don’t because we gave the money to corporations who then used none of it and instead paid themselves huge bonuses. We know that automation is chipping away at low-skill jobs, which means our workforce needs to be better educated, and that providing free college to everyone would be a good idea, we just don’t. We know how to build interstate high-speed commuter rail, we just don’t (although Biden is trying). We could fix everything, we just don’t.

We have no excuses anymore. None of these are novel or difficult problems, not even the tragedy of the commons. We can do better. We don’t need AI to fix things. We don’t need new technology to solve these problems. We already know how to do better. Our society is bad at cooperation simply because it’s run by people who are incentivized to sabotage cooperation in the name of profits. That’s it.

It’s January 1st of the new year, and with all these people wishing each other a “better year”, I am here to remind you that it will only get worse unless we do something. Society getting worse is not something you are hallucinating. It cannot be fixed by you biking to work, or winning the lottery. We are running on the fumes of our wild technological progress of the past 100 years, and our inability to build social systems that can cooperate will destroy civilization as we know it, unless we do something about it.

We live in what is perhaps the most critical turning point in all of human history, and we’re on a ship that has drifted far off course. The rapid current of technology means that we are swept along faster and faster, making it exponentially harder to steer away from the icebergs ahead of us. We must address our coordination failures. We must build systems that foster better cooperation, or this century won’t be a turning point for humanity, it will be the end of humanity.

“All that would remain of us would be a thin layer in some future rock face. This is the future we must avoid at all costs.” — John D. Boswell (melodysheep)