At my first job as a programmer, I was assigned a mentor, a senior programmer named Matt. Matt was to help me become a good employee. According to the company handbook, Matt was supposed to teach me the company history, how to work with others, how to treat customers, and the technical procedures we used to develop software.
Matt took his role very seriously. Twice a day, he would stop working, light his pipe, and start to smoke. This was the signal that he had something he wanted to tell me. Most of his ideas were vague thoughts about how to be an adult. He had no way to determine whether I applied his ideas, so I had no incentive to learn them.
However, Matt had very strong opinions about how to develop software. He believed that you needed to carefully plan your program before starting to code and that you needed to ensure your code was correct before you submitted the program to the compiler. He would sit for hours at his desk drawing flowcharts—the formal diagrams that described a program’s operation. Once he was convinced that he understood how the program would work and that his ideas were correct, he would begin to code.
He wrote his programs on graph paper, five squares to the inch. The paper allowed him to check his code’s syntax to ensure everything was spelled correctly before he started up his computer and began to type.
Matt was appalled at the way that I programmed and was certain that my professional career was doomed unless I developed good habits. I would sketch a design for a piece of code in a notebook and then start to program. I made no effort to think about the code’s validity before I typed it. I let the compiler check the syntax and waited for the computer to tell me how the code worked.
According to Matt, my approach to programming was a wasteful use of an expensive piece of equipment. It was “the product of an impatient mind.” Indeed, I used six to eight times the amount of computing cycles that Matt used. I also made more mistakes than he did. No program would compile before I had submitted it three or four or five times. Every new routine would require several hours at the machine to remove all the errors. However, I learned more new technologies than Matt. I discovered clever ways to manipulate data, techniques that exploited the parallel nature of the computer, and powerful ways to refactor the program. Matt struggled to adopt many of these ideas. Some he never mastered.
Matt claimed that I learned quickly because I was younger than he was and more open to new ideas. I claimed it was because I had a better technique than he did. I spent my days in conversation with the computer. He spent his days in conversation with himself.
Only after I left Matt’s office did I realize that our programming strategies shared a common goal—to advance our projects as quickly as possible. Matt had begun his career when computers were expensive and a programmer might have access to a machine for 10 or 15 minutes a day. If he discovered that his program didn’t compile or run properly, he couldn’t fix it at that instant. He would have to go back to his text, make the correction, and resubmit the code the following day. The only way for him to make steady progress was to add 10 or 15 lines of carefully drafted code each day.
By contrast, I learned to program in an environment in which machine time was plentiful but machines were far more complicated. I was designing systems that had multiple parts running in parallel. I could not reason my way to a perfect program without having the results from a computer. Hence, the only way for me to make progress was to start the day with a rough idea of what I was trying to achieve and interact with the machine until I had a working product.
When I first encountered Agile techniques, I assumed they were also reactions to methods like the ones that Matt used. After all, Agile appeared shortly after the conventional software engineering was established: Between 1977 and 1995, a group of engineers at the IEEE Computer Society worked to create a series of standards that codified software engineering. These standards drew heavily from quality assurance engineering and offered a formal approach to creating new software systems. These standards are summarized in the “Software Engineering Body of Knowledge,” which can be found at swebok.org.
The IEEE software engineering standards summarize the development process in a series of steps known as the software lifecycle (see Figure 1). This cycle begins, as Matt began his programming, by describing how the software will work, after interviewing the people who will actually use the software and getting detailed specifications for the new system.
For years, the specification process was done in the same way as Matt prepared his software—programmers used paper and pencil to record the requirements. We now have specification systems and formal specification languages to capture all the ideas to be incorporated in a system.
Figure 1. Software lifecycle
The software lifecycle moves, step by step, through the software development tasks. Once we have the specifications, we design the software. From there, we code, test, and deploy. Finally, we get to the assessment process. At that point, we can compare the software to our original specifications, refine our ideas, add new features, and start the cycle again.
Agile spins the software lifecycle much more quickly than traditional software engineering. After getting an initial set of specifications, it rushes through the remaining steps as quickly as possible. “Agile approaches recommend short iterations in the two- to six-week range,” explained two founders of the field, Jim Highsmith and Alisair Cockburn (“Agile Software Development: The Business of Innovation,” Computer, vol. 34, no. 9, 2001, pp. 120-127). In these short periods, “the team makes constant trade-off decisions and adjusts to new information.” Once they are done, they jump completely into the next step in the lifecycle.
I can hear Matt’s complaints about Agile. He would have claimed that it wasted resources, it was sloppy, and it could never produce good software because it had no goal. As he said of my programming approach, he would have argued that Agile was the product of an impatient mind, of someone who wanted to produce a system quickly, even if that system wasn’t correct.
In fact, Agile and other forms of new software development are products of the desire that shaped my programming approach. Agile developers use their techniques to understand the environment in which they are creating their systems. However, they are not looking to get a better picture of the technology that supports the software. After all, we have invested a lot of time and effort to create systems that hide the complexities of computer operation and simplify the process of producing software.
Rather, Agile software developers use their techniques to understand a part of development that is not well-captured by programming systems—the customer’s goals. It assumes that people cannot easily articulate their ideas for a computer system, that they cannot describe what a system can do and how it should do it. Instead of trying to produce detailed specifications, this technique offers customers working software, and the customers in turn tell the developers what they like and don’t like, what works properly, and what is incomplete.
I suppose that I should have adopted Matt’s viewpoint and complained that Agile and the other methods are inefficient and a poor use of resources, but I couldn’t. I once used computing power to teach me the logic of sophisticated machines. We’re now using that power to help us understand the desires and goals of people.
About David Alan Grier
David Alan Grier is a writer and scholar on computing technologies and was President of the IEEE Computer Society in 2013. He writes for Computer magazine. You can find videos of his writings at video.dagrier.net. He has served as editor in chief of IEEE Annals of the History of Computing, as chair of the Magazine Operations Committee and as an editorial board member of Computer. Grier formerly wrote the monthly column “The Known World.” He is an associate professor of science and technology policy at George Washington University in Washington, DC, with a particular interest in policy regarding digital technology and professional societies. He can be reached at grier@computer.org.