The house on Bagley Avenue was a mystery to the residents of its quiet, suburban neighborhood. They knew little about the owner, a former executive from the local telephone company, and less about the two men who arrived each morning in separate cars and left in the early evening.

Some speculated that the residence might be an illegal methamphetamine laboratory or a safe house for the CIA. One individual called the city government to see if the building was using an inordinate amount of electricity. Another homeowner, a retired office worker, decided to conduct her own investigation.

This neighbor began her study of the Bagley Avenue address by walking her dog past the home twice a day. She would peer at the big front window and wave if she saw any shapes moving behind the glass. On one occasion, she intercepted the postman and unsuccessfully volunteered to deliver the mail to the front door.

Despite her efforts, this neighbor gained no understanding of the activities within the house. Her raps on the door would bring a polite reply from Owen, the owner of the home, but they brought no invitations to step inside and no further information.

Eventually, she abandoned this tactic and would linger at the front lawn long enough to allow her little poodle, who was clearly not built for extended exercise, to empty his bowels on the front lawn. She might have been hoping her dog's actions would draw the attention of Owen or one of the other occupants, and from the ensuing confrontation, she might then learn the secret of the house. If this was her plan, it was never fulfilled. No matter how often she visited the property, she was unable to discern the activities of the three men.

Owen had reason to be wary about talking with his neighbors. He was using the house as the office of a new technology firm and was aware that he might be violating local zoning covenants. He was the president of the company and was doing the financial work in the guest bedroom. The company's server was installed in a spare bathroom. Ryan, the vice president for marketing, was preparing the sales plan in the den. Les, the technician, was developing the first product in a laboratory that began at a 220 volt outlet in the kitchen, snaked past two workbenches in the hallway (with acid burns on the carpet), and ended at a large computer installation in the living room.

Their working environment belied the fact that the three partners had once been highly successful employees of a large telephone company. Owen had produced record profits from the division that he had overseen. Ryan had managed a dozen of the company's products and found large markets for each of them. Les was known as a designer without a peer and had a reputation for creative, efficient work.

Owen, Ryan, and Les had been operating their firm, "Bagley Avenue Systems," for more than a year. They had produced a business plan, found initial funding, recruited a distinguished board, and even gained the interest of a few potential customers. However, they had yet to produce a working prototype of their first product, a device that would transmit data at high speeds over ordinary power cables.

No matter how hard Les worked, he could not make the prototype function properly. "The oscilloscope looked like a scene from the first Star Wars movie," recalled Owen. "Flashing points of light would appear at the bottom of the screen and slowly float to the top. If you stared at it long enough, you might believe that you could see a giant spacecraft traveling through the universe."

Owen was worried about the prototype's status. Every day of delay made another demand on the firm's precious capital and increased the likelihood that an inquisitive neighbor would expose the company. After watching Les struggle for four months with the prototype, he decided to seek the advice of a hired debugger, a programmer named Kai, who had a reputation for fixing other peoples' computer systems.

Les was not pleased to learn that Owen had hired a consultant, especially one who clearly was not trained in engineering. He was cool when Kai arrived at the house. After the initial pleasantries, Les wasted no time in challenging the newcomer. "How much do you actually know about circuit design?" he asked.

"Nothing," Kai said after a brief pause, "or maybe a little bit less than that."

Giving a sigh of obvious contempt, Les took Kai over to the oscilloscope and showed him the picture of little green stars rolling through the lumineferous ether.

"What do you think you can do with it?" Les asked the consultant.

"Depends," Kai replied, "on what it's supposed to look like."

"A straight line," Les responded.

The two stared at the screen in silence. Les took comfort in the idea that Kai's tenure would probably be short.

"What do you think is wrong?" Kai finally asked

"Everything is perfect," Les stated confidently. "I've checked every element of the circuit and looked at every line of code." Using a term that normally describes a power source that occasionally has random flucuations, he said, "I believe the problem is dirty electricity."

"Hmmmp," replied Kai. "So you think the electricity passed through a porn video on its way into the house?"

This remark irritated Les, but before he could respond, Kai asked a second question. "Are you sure that the input to the device is correct?"

Feeling that he was being mocked, Les decided to put Kai in his place. He uttered a well-known expletive, disconnected a cable, flipped a switch, and turned back to the oscilloscope. A green sine wave appeared on the screen.

"Is that the right signal?" Kai asked.

"No," said Les.

"Good," said Kai. "Let's begin here. It's time for us to remove the bugs."

The concept of the "bug," a small error in design or implementation, can be traced back to the earliest days of the computing science field. In February 1944, a staff member in Howard Aiken's Harvard laboratory wrote that they were running test problems on the electromechanical Mark I to "find bugs." In a story that Grace Hopper recounted, Aiken's staff actually found a moth in their second machine, the Mark II, and taped it in their log book with the notation, "first actual case of a bug being found."

In fact, the term is much older than the computer era. Thomas Edison used the term as early as 1878 to describe problems with electrical systems. "Bugs," he wrote to a friend, "show themselves, and months of anxious watching, study, and labor are requisite before commercial success—or failure—is certainly reached."

Debugging, the process of removing conceptual errors form programs, has been one of the crucial skills of computer science. Only in those brief days of naiveté that followed the introduction of high-level programming languages did anyone profess that a computer system could be built without hours of debugging, yet debugging has held an anomalous role in the field.

"Program testing and debugging occupies more than 50 percent of a professional programmer's time," complained an early author in the field. He noted that the process of debugging a program was "time-consuming, difficult, poorly planned, and often slighted." At the same time, he observed that most computer scientists could offer little guidance on the subject. "Students and teachers are encouraged to minimize errors by having good work habits, being methodical and neat, and checking for clerical details. This is certainly good advice, but it doesn't really help when it comes to fixing a problem."

The topic of debugging has never commanded enough attention to create its own comprehensive theory, its own journal, or even a regular conference devoted to the subject. Indeed, the theoretical literature has largely ignored debugging. The journals devoted to computer theory have published only a couple dozen articles on the subject in the years since the ENIAC began operation in 1948.

Debugging receives better treatment in the applied computing literature, but even in these journals, the coverage is incomplete. "Debugging is said to be the least established area in software development," observed the authors of a 1991 article. "Industrial developers have no clear ideas about general debugging tools, yet they debug programs anyway."

One aspect of debugging that separates it from other aspects of computer science is the fact that it is an empirical task rather than a synthetic activity. Good debuggers must reason from effects back to causes, a process that is fraught with problems. They must build a mental model of how the machine should behave under the control of the program and hypothesize the machine's state at points during the execution of the code. From observing the machine's actual state and comparing it to their hypothesis, debuggers must assess their model and determine how they should adjust their ideas and then adjust the code itself.

Much of the debugging literature has presented ideas for software tools that would help with one part of the process, the part that connects the code to the machine's state. Articles on such tools began to appear in the 1960s when high-level computer languages began to separate the programmers from the machines. The authors of a 1963 article stated, "It appears that many programmers of large-scale computers today do not understand and often do not even wish to understand symbolic machine language." "This puts severe restrictions on the types of debugging procedures that may be used, and, in fact, restricts them to debugging at the compiler language level." Such judgment was harsh and shortsighted. In 1963, several systems had symbolic debuggers, software that would connect user programs to the operations of the machines. Within a decade, symbolic debuggers would be common on most computers.

During the 1980s, the debugging literature expanded for a second time with the introduction of more sophisticated programming ideas such as concurrency, parallelism, distributed programming, and optimizing compilers. For programs that used these ideas, the old symbolic debuggers no longer gave a clear picture of machine operation. Such ideas complicate "the mapping between the source code and the object code due to code duplication, elimination, and reordering," observed one research team. They also make "reporting values of source variables either inconsistent with what the user expects or simply impossible."

However, the problem of connecting a program to the machine's underlying state is only a small part of debugging. The bigger task, that of hypothesizing the machine's state and reasoning from the actual state back to the program's logic, defies a general engineering solution. "Debugging," wrote one author in 2001, remains "as labor-intensive and painful as it was five decades ago."

Some researchers have developed algorithms to handle some of the debugging work or to automate the task of checking all possible paths through the code, but the basic work remains unchanged. It is no easier to find the bugs in a program than it is to discern the activities inside a home by standing on the lawn and observing the shadows inside the windows.

At Bagley Avenue Systems, the debugging took four days. The hardware was the easiest part to fix. The software was considerably harder. It controlled two processors, one that handled the incoming data and another that prepared the output.

At each step of the code, Kai demanded to know how the machine was to transform the flows of information and asked Les to demonstrate that the processors were behaving correctly. Les responded to each request, but he was no more patient with this process than he had been during his initial encounter with Kai.

As they came to the end of the debugging, they were no closer to working as a team than they had been on the first day. Kai continued to make his requests, often punctuating his questions with comments that irritated Les. Les did all that was asked of him, but he still resented the idea that he was working with someone who couldn't understand the details of his design.

By the end of the last day, the rolling stars had vanished from the oscilloscope. In their place were a gentle wave and a few dots that would appear at random moments and quickly vanish. Les was not pleased with the picture and turned on his partner. "You've added a new error to the code," he claimed in frustration. "It's supposed to be straight."

"Prove it," Kai responded.

Les pushed a pile of cables to one corner of the table, found a piece of paper, and started to write the equations that described the system's operation. When he reached the end of the page, he started to search for a second sheet, but Kai stopped him. Pointing to one line, Kai said, "This statement is only an approximation."

"But it's good enough," blurted Les.

"Perhaps," said Kai, "but if you add a second term, you'll see that there's a wave in output."

Kai paused to let Les fully comprehend the issue. Then he said, "If you add a third term "

"What " Les interrupted.

"If you add a third term, you'll see that it is sensitive to small changes." Kai hesitated and then corrected himself, "Sensitive to dirty electricity."

David Alan Grier is the editor in chief, IEEE Annals of the History of Computing, and the author of When Computers Were Human (Princeton University Press, 2005). Grier is an associate professor in the Center for International Science and Technology Policy at the George Washington University. Contact him at grier@gwu.edu.