Just as the fields of software and hardware development have evolved, the field of software history has likewise matured. At first, the history of software was exclusively focused on technology. Later, there were historical explorations of the software industry and professions. Today the emphasis is on applications and the societal changes resulting from software.
Allow me to begin with a small piece of personal history, which I hope illustrates a more general point about software history.
Starting in 1976, I undertook a doctoral study on the early development of computer programming in Britain. This was to prove a highly technical study that traced in considerable detail the programming systems invented in Britain in the period 1945–1955. I was then employed as an instructor at Sunderland Polytechnic, a second-tier college in the north east of England. I had the good fortune to be supervised by Brian Randell at nearby Newcastle University. Randell was a technical computer scientist and software engineer with a strong interest and commitment to the history of computing—he had recently edited a seminal collection of technical papers Origins of Digital Computers (1972).
Note that I have used the word "technical" several times. Up to the late 1970s, software history was almost exclusively technical. My dissertation was about code and texts. In my research, I managed to locate most of the system programs developed for the first three operational British computers—the Cambridge University EDSAC, the Manchester University Mark I, and the National Physical Laboratory's Pilot ACE. Studying these programs and their texts was utterly absorbing. I also interviewed several of the pioneers of these early systems (they were then mostly in their 50s or 60s), and I studied all of the derivative machines up to about 1955. In the end, my dissertation encapsulated in a single diagram the complex processes of invention and technology transfer. At the time I thought I had done a tolerably good job. I published four lengthy papers in the Annals of the History of Computing
and only a change of job to Warwick University and an itch to move on to fresh fields prevented my producing a monograph. When I look back at my dissertation today, however, I cannot do so without a mild flush of embarrassment. Let me explain why.
In addition to Brian Randell as my supervisor, I felt the need for a second advisor who was a professional historian of science (the history of technology was little recognized as a discipline in Britain at the time). Margaret Gowing, the recently appointed professor of the history of science at Oxford University, agreed to act in this role. Gowing was an economic historian by training, and she had been a driving force behind the civil history of World War II, and subsequently wrote a number of classic works on the development of atomic energy in Britain.
I sent Gowing the draft of a paper I had written about programming for the EDSAC (and which was subsequently the first of my papers published in the Annals). She wrote back to the following effect: What you have written is clearly very good. I know practically nothing about computers, but I can tell that what you have written is good history, so far as it goes. However, let me urge you to look beyond programming technology to consider the kinds of people who were using computers and the problems that they were solving.
I was completely floored by this suggestion. I could not see in 1976 how to integrate the story of users into the history of software. I was not being obtuse or myopic; I just could not see how it was possible. I remember thinking it would be like writing about a bunch of miscellaneous application programs. What would be the point? What would be the coherence? Beyond adding a few sentences about users to my paper, I buried my head in the sand, and pressed ahead with my instincts.
Looking back, some years later, I began to see the point Gowing was making. Today it seems frankly bizarre that I chose to write about the internals of the programming systems and not about the applications. Just consider some of the tasks that the EDSAC was used for. 2
In 1950, J.C.P. Miller and David Wheeler used the machine to generate the world's highest known prime number. In the mid-1950s, John Kendrew used the EDSAC in the elucidation of the myoglobin module, for which he shared the Nobel Prize in 1962. Martin Ryle used the EDSAC and its successor EDSAC 2 for the reduction of radio telescope data, a crucial process in radio astronomy, which he acknowledged in his 1974 Nobel lecture. The users of the other British machines also created path-breaking applications.
For example, at Manchester, Alan Turing did his first numerical experiments on morphogenesis (the growth of living forms) and produced printouts that looked remarkably like cow-spots, and which we can now see as lifting a corner of the curtain on dynamical systems. 3
Christopher Strachey wrote a truly sophisticated program for playing checkers, some years ahead of Arthur Samuels in the US. At the National Physical Laboratory, researchers were conducting some of the first experiments on digital applications, ranging from esoteric machine translation to a commission from the government to explore the potential for the automatic computation of income tax. 4
Also at the Laboratory, Jim Wilkinson (an ACM Turing Award winner in 1971) investigated errors and stability in digital numerical methods and developed advanced matrix programs. These were to prove vital in understanding and preventing "flutter" for the British aircraft industry, which was still reeling from the de Havilland Comet air disaster of 1954.
Yet in 1976, I just could not see that these applications related to software as such. Certainly there seemed to be nothing as concrete as how subroutine linkage was achieved on the EDSAC or the B-line index register was used on the Manchester Mark I.
Well, this biographical mea culpa is not intended as an exercise in humility. Rather it is intended to show that the world of software history was different in the 1970s. The people who wrote software history at that time, and earlier, wrote the best history they could with the knowledge, understandings, and the background they had at the time. With 20/20 hindsight, what they (we) wrote looks constrained, excessively technical, and lacking in breadth of vision. But that's not how it seemed at the time.
I believe it was difficult to write about application software in the 1970s for two reasons. First, software was not nearly such a ubiquitous concept in the 1970s as it is today. For example, the software industry was very small—less than one-twentieth of its current size. Second, we did not think that software characterized applications in quite the way we do today. We thought of an application as comprising systems design, mathematics, fulfillment, and so on, whereas software was just code. Indeed, I don't think that books such as James McKenney's Waves of Change (describing the SABRE airlines reservation system) or Steven Levy's Insanely Great (describing the Macintosh development) would have been classified as software books at all; but today we would certainly characterize them as such, because we now see software as comprising so much more than code. These two books, and others like them, describe the process by which software came to orchestrate standard hardware in unique ways.
Evolution of software history
To portray the evolution of writings about software, in Table 1
and the "Bibliography: Software-Based Information Processing"
sidebar I have listed what in my experience were the most useful or important publications on software history every year since 1967, when distinct publications on software history began to appear. This is not a scientific survey—I have simply gone through the bibliographies of my own writings on software and culled the items I personally found most useful, up to three items for each calendar year. Some years there were no publications at all; other years—rather like London buses—several came along at the same time. By the 1990s, it was sometimes difficult to choose just three, the field was so much enriched. Some might say that not all the works I have selected are about software as such—perhaps, but I certainly used them in my writings.
Table 1. Selected works on software and software-based information processing, with shortened titles. (For full titles, see the "Bibliography: Software-Based Information Processing" sidebar.)
The bibliography is not an exhaustive reading list. It is more in the nature of a subjective sampling of the literature, with the constraint of selecting a maximum of three items per year. A few of the works are outstanding, such as Jean Sammet's Programming Languages
(1969) and Claude Baum's The System Builders
(1981). Others are not particularly good at all; they are simply the most useful items published in that year. Incidentally, the great majority of works are by American or British authors. There are not, as yet, many works on software history published in languages other than English and still fewer have been translated. 5
However, a number of edited volumes have international contributors: these include the proceedings of the History of Programming Languages
conferences edited by Richard Wexelblat (1981) and Tim Bergin et al. (1996), and David Mowery's International Software Industry
To indicate how software writing has changed over time, in Table 1
I have classified each item as "technology," "supply-side industry," "applications," or "institutional, social, political." These are subjective classifications, of course, and not all works were easily pigeon-holed. The table shows how the subject matter has broadened. In the 1960s and 1970s, people wrote about technology—code and software engineering practices. Starting in the 1980s, people began to write about software as an economic activity—primarily the supply-side industry. Some of these studies also began to look tentatively at issues of labor supply and organization. In the 1990s, especially, we began to see books that set software in a much broader institutional, social, or political setting—for example, Emerson Pugh's fine books about IBM, and Arthur Norberg and Judy O'Neill's excellent history of computing at DARPA.
It is only in the past 10 years that scholars have begun to look at applications—a path that was pioneered by JoAnne Yates's 1995 study of insurance-industry software. Thus, over time, software history has evolved from narrow technical studies, through supply-side and economic studies, to broad studies of applications. Software history is a cumulative exercise. To write about broad software applications, it was necessary to know something about the history of the software industry and the institutions with which it interacted. And to write about the software industry, it was necessary to understand the technologies and practices of software design and engineering. What we write today depends heavily on those who went before.
One could equally well have classified the works in Table 1
by their authors' qualifications. This would show that most of the early works were written by software practitioners. In the middle period, journalists, especially business writers, made many contributions. After that, historians and the academy in general began to add many high-quality studies to the field. All these authors have been essential to the field's development, and it is always appropriate for historians to acknowledge the contributions of individuals for whom history is a passion, not a profession.
A lack of proportionality
As scholarly works on the history of software left the publishing mill, it would have been an interesting experiment to put the accumulation of material on programming languages and systems on the left-hand pan of a set of scales and everything else on the right-hand pan. I think it would not have been until about 1990 that the right-hand pan began to weigh heavier.
There are several reasons why programming languages and systems, especially the former, have received such disproportionate attention. First, in the 1970s the history of computing field was largely populated by practitioners, and their vision was constrained by what they knew—primarily the technical attributes of software. Second, as Jean Sammet has noted, several hundred programming languages existed, of which several dozen were in widespread use. 6
The study of programming languages was in and of itself an activity of considerable breadth and scope. Third, the study lent itself to a form of historical textual analysis somewhat like literary criticism, which required little in the way of archival access or oral history programs.
My comments on the study of programming languages are not intended to belittle the endeavors of the pioneers in software history. Indeed, some works were written by outstanding technical experts who also took it on themselves to kick-start the software history field—Jean Sammet, Donald Knuth, and Peter Wegner among them. Nonetheless, these early studies of programming languages were of the low-hanging-fruit variety. Even in the realm of systems software, other aspects were comparatively neglected. Today there is still a dearth of good historical accounts of operating systems and databases.
In the early 1980s, people started to write about the software industry. To date, the emphasis has been on large proprietary software firms. I would single out Claude Baum's excellent The System Builders (1981), the history of the Systems Development Corporation (SDC). This book, together with the superb collection of SDC records in the Charles Babbage Institute (CBI) that complement it, is one of the real gems of software history. Another fine work is Richard Forman's history of Informatics, Fulfilling the Computer's Promise (1985). This little-known, privately published work was commissioned by Walter Bauer, the founder of Informatics and a trustee of the CBI.
As a genre, histories of the software industry are very much like business histories of other industries. There is good and bad, but most books are surprisingly useful. The majority were not written by professional historians but by journalists, company founders and executives, and business school academics. For example, Baum, the author of the SDC history, was "not a writer by profession"—he was an SDC employee of more than 20 years' standing; he worked with a team of four other senior employees who had access to the company archives and between them conducted more than 100 interviews. 7
Forman, the author of the Informatics history, was a graduate student who made extensive use of the company archives, conducted many interviews, and was assisted by Frank W. Wagner, a cofounder of the company and one of the founding editors of the Annals of the History of Computing. Many of these histories are very good indeed, and our field is highly dependent on them—there will never be enough trained historians to do the kind of job one might like.
One of the great strengths of the CBI under Arthur Norberg has been the way it has encouraged the efforts of diverse computer history enthusiasts. For example, the CBI has worked closely with the Software History Center, founded by former software industry executives Burt Grad and Luanne Johnson.
Perhaps the one aspect of the business history of software that is most troubling today is the disproportionate number of studies of the 100 biggest firms, compared with the negligible number of studies of the 100,000 small firms. And among the largest firms there has been a disproportionate interest in Microsoft. If one were to put the books about Microsoft on the left-hand pan of our scales, and all the other books on the right-hand pan, we can guess what the result would be. These cavils apart, however, I would say that the business history of software is in fair shape. But this leaves many unfilled spaces in the software history tapestry.
Voids in software history
In an attempt to stimulate the filling of some of the holes in software history, the CBI and the Heinz Nixdorf MuseumsForum organized a conference, History of Computing: Software Issues, at Paderborn, Germany, in April 2000. The preface to the published proceedings gives an excellent vignette of software history at that time. 8
As the organizers viewed it, software history had focused on software technologies and early set-piece projects such as the SAGE air defense system and the Bank of America's ERMA system. The organizers wanted to broaden the field but without setting an agenda that was so ambitious it was unworkable. After much deliberation, they agreed on a set of five topics; for each, a leading software historian was invited to prepare a position paper and two or three expert commentators were invited to make a response. The topics chosen were as follows:
• Software as Science
• Software as Engineering
• Software as Reliable Artifact
• Software as Labor Process
• Software as Economic Activity
As a member of the organizing committee, I was somewhat uncomfortable with some of these choices and the balance between them, although we reached a consensus. I'm quite sure my co-organizers had their misgivings too. For the fact that we did reach a consensus, we have to thank Arthur Norberg, Ulf Hashagen, and Reinhard Keil-Slawik who led the discussions. We ended up with a program of a set of excellent orthogonal discussions of the literature and possible research directions.
As I now look back on the topics, I am conscious that six years is a long time in software history. What, I wonder, could have induced us to give "Software as Dependable Artifact" weight equal to software engineering or the software industry? Perhaps we were still in thrall of the Y2K problem, and this had made us overly sensitive to issues of software quality. More likely we were influenced by the fact that we knew of an outstanding scholar and speaker, Donald MacKenzie of Edinburgh University, who we wanted to give a platform.
Why did we not include applications software as one of our topics? Why did we not include any cultural aspects of software such as videogaming, national styles in software, or the open-source community? The answer is worth stating. We were a group of experts seeking to reach a consensus and move the field forward a few paces, not to push it off the edge of a cliff. This is just the kind of well-judged leadership we have come to expect from the CBI and Arthur Norberg.
The year 2000 was an excellent time for software history. Six months after the Paderborn meeting, the CBI and the Software History Center jointly organized a conference on the early years of the packaged-software industry at the Xerox Palo Alto Research Center. With the inspired title Unbundling History, the conference focused on the enterprise software products industry that flourished after IBM's unbundling decision in 1970. At that time this "old" software industry was being eclipsed in the media by Microsoft and the other personal-computer software makers, and the meeting was a timely corrective. Speakers came from academia and industry in equal numbers, building a lasting bridge between the two communities.
For reasons that will become apparent in my closing section, I think it is too early to set a research agenda for software history. In any case some of the obvious holes are already being plugged. For example, application software is getting much-needed attention in Jim Cortada's path-braking trilogy The Digital Hand. There is a glimmering of interest in video games and software cultures. Open source has received attention from both economic and cultural historians.
Software history records
In the early 1980s, I was one of many people Arthur Norberg consulted about the kinds of records that historians would find useful and that the CBI might collect. One issue at the time was the "manuals problem." I think we all agreed that indiscriminately collecting computer manuals was not the answer to software history or any other kind of history. There was a story circulating at the time that since the launch of System/360, IBM had become, in terms of the number of titles issued, the largest publisher in the US apart from the federal government. This may have been an urban myth, but it certainly made the point. Collecting manuals was about what you could collect, not what you should collect. Lay people often misunderstand records collection: quality and selection is paramount; we can't keep every thing, and choices have to be made.
When I wrote my history of the software industry, I became aware that one of the most potentially useful sources was the tens of thousands of industry reports produced since about 1970 by firms such as IDC, ICP, INPUT, Forrester Research, Datapro, and several others. My research fellow and I wrote to all the firms that still existed, asking what reports they had kept and if we could see them. Most of those that troubled to reply gave essentially the same answer: they did not keep reports going back more than 10 years, and they could not give us access to those they had retained. For us, it was a case of disappointment quickly followed by relief: how could we process thousands of reports anyway? Moreover, those familiar with the genre I think will agree that industry reports are not the most compelling reading.
Almost as my manuscript was being shipped to MIT Press, two analysts in fact decided to make their archives available to historians. This was a direct outcome of the Unbundling History conference, and it stands as a testimony to the way in which the CBI and the Software History Center have built bridges to the practitioner community. First, Larry Welke donated the ICP archive to the CBI. I was able to draw on this before my book shipped to the publisher. Actually, I may be overstating the case—there were several bankers' boxes of material and I had just two days to explore them. Second, Peter Cunningham of INPUT stated he would make his firm's archive available to historians. This was a truly generous offer, but my book was in press before I could take advantage of it. But again, I'm not sure how a lone historian could have used such an enormous volume of material.
It may be that the time has come to re-evaluate what records computer archives keep. This is not an issue for Arthur Norberg, but rather his successor Tom Misa. Most of the archives I use are still in the pencil-and-yellow-pad era. Recently, however, I was engaged as a historical expert for a software lawsuit. In many ways lawyers do the same job as historians, though to a shorter timescale, and perhaps with less objectivity. Lawyers have lots of money, and this has enabled them on occasion to take a lead in the application of information technology. For example, Lexis and WestLaw were among the first pioneering suppliers of online information. 9
What best-practice lawyers are currently doing is ahead of where historians are today. In the case I was involved in, there were several million pages of testimony and subpoenaed documents. These had all been scanned, OCR processed, and resided in a database in Los Angeles. Over the Internet, from Warwick University, I was able to interrogate this mass of material with a powerful query language. For someone more used to a pencil, notebook, and a photocopy request form, it was quite an eye-opener. Particularly as born-digital records emerge, all modern-records archives will need to implement electronic searching and remote access. It would be very appropriate if the CBI were to be in the vanguard of this transition.
What software history may become
All computer historians would agree that the state of software history is not what we would wish. It is easy enough to articulate particular faults (as I have done here); it is much more difficult to suggest remedies. Below I quote from the social historian Harold Perkins who put his thumb on the problem rather well. I do this with some hesitation because the quotation comes from a review of my history of the software industry From Airline Reservations to Sonic the Hedgehog (2003), which appeared in the Times Literary Supplement:
Campbell-Kelly is a master of technical detail and the alphabet soup of acronyms but, like most specialists in an arcane activity, he has tunnel vision and provides little social context. He does "internalist" history, rather like old-fashioned art history or history of science, full of innovators and heroes driven by creative opportunism. The impact of the computer industry on society, on the way people live and communicate, is largely left to the reader's imagination. Even the state and military applications are touched on rather than explained. The computer and its software nervous system brought a revolution in human development as significant as the steam engine, the automobile or the aeroplane, and even more effective in shrinking the planet. This technically expert book is rather like old railway history written by railway buffs who know the number of wheels and the horsepower, the names of the engineers and companies, but take for granted how they changed the world. 10
They say there is no such thing as bad publicity. I hope so. Actually, I think Perkins' criticisms are overstated, and he was perhaps not the right reviewer for my particular book. The book I wrote was a fairly standard, competent if undistinguished, business history, and most business historians would recognize it as such. Nonetheless, Perkins is surely right in characterizing the kind of software history we would all like to see. Let's read again his key sentence: "The computer and its software nervous system brought a revolution in human development as significant as the steam engine, the automobile or the aeroplane, and even more effective in shrinking the planet."
The fact is, we are years away from writing a book like this. I would hazard 10 or 15 years. What is it that stops us? To answer this question, allow me to digress to consider the history of the office.
In Table 2
and the "Bibliography: Office-Based Information Processing"
sidebar, I have listed some of the major works on the history of offices and office-based information processing. What we see is an evolution in publications not unlike that which is currently unfolding in software history. First, in the 1920 and 1930s, there was a flurry of books about office machine technologies—the typewriter, adding machines, and punched card machines. After a rather lean period during the Depression years and World War II, in the late 1940s and 1950s there appeared books and articles about the office machine industry—IBM, NCR, and others—and this business—history tradition continues in the writings of Cortada and others. The 1950s and 1960s saw a major strand of literature about office work and office workers; in the 1970s and 1980s this aspect of office history engaged with gender studies. In the 1980s there was a growing interest in office applications, such as the census and banking. To write a holistic history of the office, it was necessary to draw on all these different genres—technology, industry, applications, labor history, and the social and institutional contexts.
Table 2. Selected works on offices and office-based information processing, with shortened titles. (For full titles, see the "Bibliography: Office-Based Information Processing" sidebar.)
In the 1990s, especially the past few years, this holistic literature has started to appear. I would single out JoAnne Yates's Structuring the Information Age (2005; a longitudinal history of information processing in the insurance industry) and Jon Agar's Government Machine (2003; a study of British bureaucracy). Both books cover an enormous time span—Yates starting in the 1890s and Agar starting even earlier, until recent times. Both books integrate the histories of office machinery from manual methods through punched cards to computers, the supply-side industries, labor and employment issues, as well as complex social, governmental, and regulatory issues. If we replace the words "computer" and "software" by the concepts of "office" and "clerks," the Perkins sentence might read: "The office and its clerical nervous system brought a revolution in human development as significant as the steam engine, the automobile or the aeroplane, and even more effective in shrinking the planet."
This sentence captures perfectly the scale and scope of Yates's and Agar's books. But it is only in the past decade or so, following 70 or 80 years of cumulative historical activity, that it has been possible to write such histories.
My hypothesis is that the history of software is following a similar trajectory. Software history began with narrow but essential technical studies in the 1960s and 1970s. We then looked at the supply-side industry in the 1980s and 1990s. We have only just begun to study applications. The study of institutional, social, and political issues has barely begun. Software history is heading in the right direction, but we need a solid base of secondary literature before it is possible to write the major holistic works that will do our profession justice.
Finally, when one comments upon the shortcomings of software histories, it is important to keep in mind that the history of software is dynamic and still unfolding. I do not mean, of course, that the history of the office has ended. But it has evolved through a distinct set of changes of location and process from which we can construct solid trajectories that give us a sense of where we have come from and perhaps where we are going. Thus, office locations have evolved from small to large, and from decentralized to centralized. Office processes have evolved from manual techniques, through business machinery, such as typewriters and punched-card machines, to highly automated computer systems. These trajectories are clearly reflected in the Yates's and Agar's narratives.
Software has not yet passed though comparably long trajectories. For example, in terms of process, we have evolved from a craft discipline to software engineering, but not much beyond. We have evolved from bespoke software programs to software products, but, again, not much beyond. These trajectories lack enough data points to clearly indicate where we are headed. Indeed, there is a sense that software is still turbulent and evolving. For example, as I write, many software companies are in a state of transition from products to services: such software-as-a-service companies deploy software on their own servers on behalf of users, instead of users licensing software products as such.
If a software-as-a-service company is an enterprise that writes, hosts, and deploys software over the Internet, then what is an online bank? It too is software driven—users store financial data and make transactions in a way not dissimilar, for example, to Web-based software. To take a less provocative example, consider eBay, which is very much a software creation. In a technical sense, eBay is a software platform 11
—it publishes its Application Program Interfaces, and hundreds of firms exist by offering software and Web-service complements. 12
It is possible that we are observing a convergence between ordinary information businesses and software enterprises. As software firms migrate to computer services, they will become increasingly indistinguishable from other information businesses that conduct their operations over the Internet. In 20 or 30 years' time, the outcome of these changes will be known, and software may have entered a settled state, rather like today's offices. It will then at last be possible to write satisfactorily about "a revolution in human development as significant as the steam engine, the automobile or the aeroplane."
is a professor in the Department of Computer Science at the University of Warwick, where he specializes in the history of computing.