Posts
This is the first in an irregular series of posts on the subject of software development, its history, its current state, and its potential futures. Software development is a subject I've been passionate about for a long time, and one which has demanded a lot of my professional life. I'd like to present some of my experience and the conclusions I've drawn from it for others to use.
I've been a software developer for almost three decades now, and I probably know all the jokes about how bad software development is. There's some reason for the jokes. Software is too hard to develop, it takes too long, costs too much, and has too many bugs. Big systems aren't reliable, are too hard to understand. No one understands how to manage software development, how to plan it, or how to design it so that it performs the functions that are required.
Probably the best known joke about software is the old line, "If buildings and bridges were built the way we build software, the first woodpecker to come along would destroy civilization." True or not, the fact that so many software professionals repeat the joke indicates that someone believes there's truth in it. It shows a bad case of what I call "concrete envy" in the field, a feeling of inferiority to the older, notionally more mature, engineering disciplines.
Now that I've said the things that everyone knows, let's talk about the real elephant in the living room. The reason nobody knows how to manage software is that nobody really knows what software is. Computing is a bastard field, out of electronics by mathematics. Computer science as a research or educational discipline at first was organized in either Electrical Engineering or Mathematics departments, and the two had very different ideas of what "programming" was. Was it science or engineering? Mathematical formulae or electrical diagrams? Abstract symbology or concrete instructions? And then it got more complicated as new generations of computer languages came into use by programmers who had never constructed a patch block or written a program in machine code. They took the abstractions of the languages for granted as real things, not combinations of registers and arithmetic units. Several generations of languages and tools later almost no one in the field has any connection with the original concepts, which are buried under layers of other abstractions.
To most practitioners now, software is neither science nor engineering but a technical craft with its own rules and requirements, its own language and symbology. Its purpose isn't learning about nature or about what can possibly be constructed, it's about building specific devices for specific purposes. It's not done by universities or by engineering labs but by corporations that need software to operate their businesses, and need to manage the cost and schedule of projects to construct it. And the people responsible for the creation of software are often managers who were never programmers themselves and don't have any understanding of what what needs to be done, or what the result of the work will be. Sometimes (especially in open source software communities) the people responsible are also the ones who do the development, but they are less concerned by how the software is used than by how it is designed and built.
Now this is starting to sound like a common software rant, I know. But consider the implications of there being many different kinds of people with different kinds of expertise involved in the software industry. There's no agreement on what they're working on, let alone how to work on it. And then we get into the deep quagmire of requirements versus specifications: the question of how to determine what the software should do, and how to translate that into a description of how the software should work. Many's the project that's bogged down in that pit only to be dug up later by paleontologists in chunks of peat.
So here's the task I've taken on for these posts. I'm going to describe the state that software development is in today, with some attempts to understand how that state came to be. Then I'm going to prescribe what I think is a good, but certainly not the only good, treatment regime to improve matters. Along the way I'll talk about some of the prescriptions that have been handed out before, and why they did or didn't work. I promise that this will not be a completely objective analysis: I have strong opinions, and a particular viewpoint to work from, and that will color what I have to say. I hope that means that it will be colorful as well.
Some topics I will definitely touch on: the checkered history of programming languages, tools, and paradigms, the business of development methodologies, and the effect of the high-tech economy on progranmming. I welcome your comments on what I have to say, and will respond as I can.
Next up: what is software, and what does it mean to "build" it?
Saturday, August 25, 2007
Subscribe to:
Post Comments (Atom)
5 comments:
Developers with such attitudes of "I didn't feel like putting in comments" are examples of parts of the problems. Lack of competent documentation is common. Passive voice is endemic in much of the documentation that does happen to get written. Lack of documentation that's not code-oriented is also an issue. Coders tend to be down in the minutiae, and don't provide much in the way of overview of "just what is this supposed to do, and -why-? "
Then ther's change control... ah, the joys [NOT!] of trying to trace back through version after version of controlling document trying to figure out what change happened when to what and why, and what the the gestalt is/does...
This discussion should be fun.
A friend of mine is collecting stories about how software professionals learned their trade. She says I'm one of the more unusual examples of the self-taught practitioner alive today. I learned how to code in the age when big-iron dinosaurs still roamed the datacenters of the world, but my introduction to the practice came at the age of about nine. I learned first by debugging the code for the games I wanted to play by finding the transcription errors introduced in the process of me typing them in by hand from copies of Creative Computing. (There were also editorial errors at the magazine. Those were fun.)
As a result, I probably have highly idiosyncratic views about why Software Development Is The Butt Of All Those Jokes. I'll try to contain my weirdness for the duration.
Software is essentially magic. Not just in the terms of that old fannish joke, but in the senses that:
1) Not only can't non-programmers verify the work of a programmer (which is what makes it a profession), but it's difficult even for other programmers -- you need more-or-less as much talent to verify a program as to write it, and usually more time.
2) It has that classic sense of immense power, barely controlled by esoteric knowledge. A single character changed or misplaced in your s/p/e/l/l/program can turn your utility or shellscript into a disaster, waiting its moment to strike.
3) A program is very nearly the wispiest, least material thing that we can even talk about "constructing":
In one sense, electrons flickering through circuits, signals compressed to such speed that "one foot per nanosecond" becomes actually relevant to processor design.
In another, we're knitting lines of almost pure intention into useful structures, some so complex as to resemble sentience.
In a third view, we're taking our generalized computing engine, and slashing away at the insubstantial mass of the things it could do, you might say "cutting away anything that doesn't work like a spreadsheet".
4) Even the big standards bodies have recognized that "software engineering" is still a proto-field -- we simply don't know how to get consistency and reliability anything like that of "real engineers". There's a lot that can be learned, but the real difference between a half-assed programmer and a brilliant one, isn't just about knowledge -- character traits like creativity and self-discipline are critical.
Fascinating! As a non-engineer with an engineering mindset, I tend to read and follow stories about coding, and I often find them comprehensible even when I know nothing about the code involved. I'm already thinking about how your ideas about craft also cross-apply...
AJ
My apologies for taking so long to moderate your comment. As you'll see from the later postings, I've been rather busy with family business lately, and just haven't had time to look at the blog. But I'm back now, and will be watching for comments, as well as posting some of the further remarks on software I promised.
Post a Comment