Why it is Important that Software Projects Fail

Abstract

This paper boldly challenges the long established misconception that the catastrophic failure of expensive software projects is detrimental to society. Historical analysis of bureaucracies such as the Australian Tax Office shows that massive software automation has not increased their real efficiency since the 1950s. Any increase in the efficiency of individual workers has simply been consumed by increased bureaucratic complexity, as predicted by Parkinson’s law. As the primary net effect of software is to facilitate bureaucratic complexity it is therefor essential that software projects fail if society is to function effectively. In this way the heavy burden of guilt can be lifted from the shoulders of the numerous project managers that have subconsciously devoted their careers to ensuring that projects rarely, if ever, succeed.

Humans don’t scale. Technology scales. Humans often think that because tech scales, that it can scale humans. They are wrong.

We know this because in 2007 the tax office’s internal budget was AU$11.4 billion, or 1.23% of GDP. In 1955 it performed essentially the same task without automation for A£66.7 million which was 1.33% of the 1955 GDP. The difference is not statistically significant. (Normalizing by GDP (essentially the sum of everyone’s earnings) accounts for the growing population and inflation.)

To many this is a surprising result. How could the staggering amount of automation instigated over the previous fifty years not produce any meaningful effect on productivity?

One big reason is that we have more data to sift through. As Berglas notes, it took a lot of effort to send out memos in 1955. It is trivial to send an email now. No longer are the senders deciding what is important, the receivers are – each and every one of them. Berglas goes on to cite the incredibly complex business rules that are veiled in marketing. Customers are left to figure out the complexities with the support departments why they aren’t getting their special discounts.

The boundless creativity of politicians and bureaucrats to develop new and more complex regulation is bounded only by the bureaucracy’s inability to implement them. The absolute size of the bureaucracy is constrained by external factors, so the only effect of automation can be to increase bureaucratic complexity.

It would simply not have been feasible to implement the Superannuation Co-Contribution scheme in 1955, but automation had made it possible in 2001. Fortunately for society most tax office software projects have also failed, so the act and regulations have been limited to 15,698 pages. But imagine if all the projects had succeeded? We might need to deal with well over 150,000 pages of regulations, and society would be in danger of collapsing under their weight.

This economic collapse is a good example of such a phenomenon. Relying on the tech models to say “Yes this will work”, combined with the sleight of hand of good salesmanship created an economic society of such complexity that no one was able to see it collapse until it was too late.

Humans do not scale. Technology is not for deciding what is important or correct.