Issue No.04 - July/August (2000 vol.17)
Published by the IEEE Computer Society
During the first half of the 1990s, IEEE Software reported the benefits of successful process improvement programs from single sites or divisions in companies such as Hughes, 1 Raytheon, 2 and Motorola. 3 During the latter half of the 1990s, many organizations have learned how to implement improvement programs across multiple sites and to spread best practices and lessons learned.
In This Issue
This focus section offers reports from Asia and North America on corporate-wide implementations of software process improvement programs guided by process standards such as ISO 9000-3 and the Capability Maturity Model. 4-6 Unfortunately, a paper describing a successful corporate-wide program in a European company was withdrawn just as this issue was being sent to the printer.
The section begins with Gargi Keeni's "Evolution of Quality Processes at Tata Consulting Services." Keeni describes the decade-long effort to achieve high maturity at TCS in India. The article describes how Tata took a centralized approach to complying with ISO 9000-3, but a phased approach in deploying the CMM. She discusses some of the challenges and methods used in deploying practices at higher CMM levels. She also presents some of the results that high-maturity centers achieved within Tata and how some TCS centers brought important processes and results under statistical control. Some of the most impressive results indicate the continued growth in estimating accuracy and how control limits for this measure have grown steadily narrower. She also describes some of the methods used to increase suggestions and spread lessons learned across TCS centers.
In the second article, "Telcordia Technologies: The Journey to High Maturity," Bill Pitterman describes an improvement program at Telcordia Technologies (formerly Bellcore) to make consistent, coordinated improvements in all eight software-producing divisions. The effort started with the definition of a high-level process at the corporate level that Telcordia could expand within each division, product line, or project to respond to specific needs. Telcordia's bolstered its program with strong executive leadership that held divisions accountable for making improvements. Pitterman describes in detail some of the challenges and solutions Telcordia encountered while working to implement a quantitative management capability (CMM Level 4). Pitterman describes quantitative techniques that they implemented for controlling both schedule adherence and progress through test. The desire to steadily reduce the cost per test case drove the quantitative control of test results. The article concludes with the dramatic improvements Telcordia achieved in field fault density, on-time releases, customer satisfaction, and cost per test case.
The final article in this section, "Analyzing a Space Shuttle Onboard Software Process with Statistical Process Control," gives deeper insight into one of the most challenging aspects of process improvement: the implementation and use of quantitative process management. Bill Florac and Anita Carleton of the Software Engineering Institute and Julie Barnard of United Space Alliance report the analyses they performed in implementing statistical process control of the review preparation process for Onboard Space Shuttle software. Their analysis is one of the best examples of the effort needed to interpret and effectively use statistical process control techniques in software development. Many high-maturity organizations have commented on the difficulty they experienced in interpreting and using statistical data at the process event level. This article gives an excellent description of how they used data plotted in statistical process control charts to decompose a "process" into component processes that behaved differently. They discovered that a review preparation process really comprised a combination of several distinctly different review preparation processes for different module types and sizes. What makes this work even more impressive is that the Onboard Space Shuttle software development processes' integrity has survived three corporate mergers since the mid-1990s (IBM to Loral to Lockheed Martin to United Space Alliance).
Although the organizations included in this section differ in continents and cultures, several common themes emerge from these reports. All organizations had quantifiable business targets and managed their pursuit of these targets empirically. Most of them used a combination of process standards to guide their program. The organizations had central groups that coordinated improvement activities with local improvement staff in each division. They also demonstrated improvements on multiple criteria, one of which was often time to market. The common traits of mature software organizations are probably more numerous than their national and cultural differences.
Future Process Challenges
The software process improvement decade is over. Although many software development organizations learned how to implement successful process improvement programs during the 1990s, others struggled through without making lasting improvements. These latter organizations will spend the next decade trying to catch up while the leaders undertake new challenges. The process challenges of the coming decade include process integration, harmonization, and acceleration.
The challenge of process integration comes in several forms. As software organizations began improving their processes, systems engineering and other development organizations began making similar efforts to improve. Other engineering disciplines created their own improvement models, often guided by the CMM. However, organizations often struggled when attempting to integrate improvements directed by multiple models lacking common architectures. The CMM Integration Project at the Software Engineering Institute represents one effort to manage these integration difficulties. The challenge to systems organizations in the next decade will be to integrate improvement activities across engineering disciplines, so that they can ultimately integrate rather than just coordinate their system development processes.
A decade of process improvement affected not only developers, but also customers. The 1990s began with the US Department of Defense evaluating its bidders on the maturity of their processes, and ended with increasing numbers of customers evaluating the development processes of their suppliers and outsourcers. As developers matured, customers often realized their own immature acquisition processes hindered their supplier's efficiency. Accordingly, customers began improving their acquisition processes so they could work more effectively with a mature developer. As customers continue to mature, they will want to integrate their processes with those of their software suppliers. Thus, the next challenge for developers will be to create flexible processes that they can tailor for integration with the acquisition processes of different customers. An important technical challenge will involve the detection of inconsistencies among merged processes from different organizations.
Software organizations are being evaluated against myriad standards, including CMM, ISO 9000-3, SPICE, and Bootstrap. Additionally, they are trying to make improvements guided by many models such as the People CMM, the Project Management Institute's Body of Knowledge, and IEEE standards. Software organizations that find themselves tangled in a thicket of standards and improvement models, a problem that afflicts many segments of industry. Many of these models share large overlapping content. 5 Software organizations will want to show how their various development processes relate to similar processes contained in several standards. Thus, developers will need to harmonize their development processes with similar elements shared among several process standards in order to trace how their processes satisfy these standards.
Having tuned their processes for reliable budget, schedule, and quality performance, software organizations are now being challenged by competitors to make substantial reductions in time to delivery. Developers must prune their processes to essential elements and improve coordination while still achieving quality and budget targets. Organizations that excel under these conditions must master and continually optimize their processes to achieve reliable results on accelerated schedules. The technical challenges will involve reorganizing processes to work in parallel without causing excessive iteration, pruning processes to their efficient core, automating processes that require little knowledge, and shortening the lead time for developers to master accelerated processes.
If the 1990s was the decade of software process improvement, then the 2000s will be the decade of software process mastery. Those who can integrate, harmonize, and accelerate their development and maintenance processes will prevail. This section presents reports from three organizations that are poised for the challenges of a new millennium.
Bill Curtis is cofounder and chief scientist of TeraQuest Metrics in Austin, Texas, where he works with companies to improve their software development capability. He is an author of both the Software and People CMMs. Contact him at firstname.lastname@example.org.