This is the first in an irregular series of posts on the subject of software development, its history, its current state, and its potential futures. Software development is a subject I've been passionate about for a long time, and one which has demanded a lot of my professional life. I'd like to present some of my experience and the conclusions I've drawn from it for others to use.
I've been a software developer for almost three decades now, and I probably know all the jokes about how bad software development is. There's some reason for the jokes. Software is too hard to develop, it takes too long, costs too much, and has too many bugs. Big systems aren't reliable, are too hard to understand. No one understands how to manage software development, how to plan it, or how to design it so that it performs the functions that are required.
Probably the best known joke about software is the old line, "If buildings and bridges were built the way we build software, the first woodpecker to come along would destroy civilization." True or not, the fact that so many software professionals repeat the joke indicates that someone believes there's truth in it. It shows a bad case of what I call "concrete envy" in the field, a feeling of inferiority to the older, notionally more mature, engineering disciplines.
Now that I've said the things that everyone knows, let's talk about the real elephant in the living room. The reason nobody knows how to manage software is that nobody really knows what software is. Computing is a bastard field, out of electronics by mathematics. Computer science as a research or educational discipline at first was organized in either Electrical Engineering or Mathematics departments, and the two had very different ideas of what "programming" was. Was it science or engineering? Mathematical formulae or electrical diagrams? Abstract symbology or concrete instructions? And then it got more complicated as new generations of computer languages came into use by programmers who had never constructed a patch block or written a program in machine code. They took the abstractions of the languages for granted as real things, not combinations of registers and arithmetic units. Several generations of languages and tools later almost no one in the field has any connection with the original concepts, which are buried under layers of other abstractions.
To most practitioners now, software is neither science nor engineering but a technical craft with its own rules and requirements, its own language and symbology. Its purpose isn't learning about nature or about what can possibly be constructed, it's about building specific devices for specific purposes. It's not done by universities or by engineering labs but by corporations that need software to operate their businesses, and need to manage the cost and schedule of projects to construct it. And the people responsible for the creation of software are often managers who were never programmers themselves and don't have any understanding of what what needs to be done, or what the result of the work will be. Sometimes (especially in open source software communities) the people responsible are also the ones who do the development, but they are less concerned by how the software is used than by how it is designed and built.
Now this is starting to sound like a common software rant, I know. But consider the implications of there being many different kinds of people with different kinds of expertise involved in the software industry. There's no agreement on what they're working on, let alone how to work on it. And then we get into the deep quagmire of requirements versus specifications: the question of how to determine what the software should do, and how to translate that into a description of how the software should work. Many's the project that's bogged down in that pit only to be dug up later by paleontologists in chunks of peat.
So here's the task I've taken on for these posts. I'm going to describe the state that software development is in today, with some attempts to understand how that state came to be. Then I'm going to prescribe what I think is a good, but certainly not the only good, treatment regime to improve matters. Along the way I'll talk about some of the prescriptions that have been handed out before, and why they did or didn't work. I promise that this will not be a completely objective analysis: I have strong opinions, and a particular viewpoint to work from, and that will color what I have to say. I hope that means that it will be colorful as well.
Some topics I will definitely touch on: the checkered history of programming languages, tools, and paradigms, the business of development methodologies, and the effect of the high-tech economy on progranmming. I welcome your comments on what I have to say, and will respond as I can.
Next up: what is software, and what does it mean to "build" it?
Posts
