Pages: pp. 5-8
This issue's theme is a bit novel; it's about climate change. Climates are sets of assumptions so deeply ingrained that even when they're no longer true we might fail to take notice. If you grew up in an arid climate, it could take years of gradually increasing rainfall before you finally wake up on some dreary, damp morning and say to yourself, "I don't think we're in New Mexico anymore, Toto."
Software changes are a lot like rain. While a few drops here and there usually aren't that much of a problem, a steady downpour can be damaging—and a deluge can wipe out everything you've carefully built. Such damage isn't limited just to software. More than one IT-dependent business has folded after years of changes eroded its critical software infrastructure to the point where it could no longer support the company's full weight.
Consider a few examples. Twenty years ago, most software engineers would have burst out laughing if someone had asked them to accept daily binary patches to critical pieces of their basic software infrastructure. Nowadays, those same engineers worry when they don't see such patches for a week or two—perhaps an ominous sign that application providers have thrown in the towel in dealing with the exponential growth of spyware, viruses, worms, and Trojans. Similarly, software and hardware installations have undergone an astonishing transformation. Installations and upgrades that once took hours, days, or even weeks have been automated by installation processes that let even casual users add, change, or remove software, and even hardware, in minutes.
At a finer level of detail, one that's often below users' perception, changes are occurring even more rapidly. Mobile code's rapid growth has blurred the once-solid boundary between data and software, letting distressingly powerful programs wink in and out of existence on PCs like fireflies on a summer night. If we're prudent and a bit lucky, these ephemeral fireflies of functional change will remain as transient and harmless as their real-life counterparts. But even the most harmless of these mobile code fireflies represents a level of automated change to system configurations that two decades ago (and still today) would've had configuration managers waking up in a cold sweat.
We'd like to suggest, however, that accelerating change's most dangerous impact isn't configuration control per se, but something more subtle and ultimately far more dangerous: The rapid erosion of hard-won software properties—the "-ilities" of traditional software engineering—just when they're needed most. To put it another way: While we're moving from building isolated homes to creating globe-spanning edifices, the rains of rapid change are revealing our building materials to be no more enduring than sun-dried mud.
Take maintainability. In industry, maintainability has never really been a permanent software feature as much as a limited, nonrenewable resource that application originators have provided in varying degrees. Changes gradually use up this resource by muddling the intent and violating subtle but vital guarantees. As this supply of maintainability dries up, the cost of changing a module rises until it becomes cheaper to replace than fix. Rapid changes accelerate this process, leading to swift extinction of maintainability—and, too often, a wide range of other software properties that similarly depend on the original software design's clarity and implied guarantees.
Take scalability. As a property that's well suited to tasks that change rapidly in scale and scope, scalability's importance has grown with the Internet. Unfortunately, effective scalability in a network of shared resources requires delicate balancing of control, functionality, protocols, and willing participation. This kind of precision and cooperation is easily damaged or derailed by random changes to a network's underlying configurations, modules, and user agreements.
Take security and privacy. Many examples show how minor changes to the wrong system settings can devastate properties, but the problem doesn't end there. The role of properties in keeping mischief out of a system also makes them prime targets for that mischief. So, minor change erosion that for scalability might only reduce the maximum peak capacity could for security and privacy lead to potentially devastating break-ins.
Take adaptability. Having "almost" the right software functionality does little good if the fit isn't close enough to work—and too often the only alternative is costly redevelopment. Making software truly adaptable is one of software design's trickiest challenges, requiring both an uncanny knack for estimating how software will change over time and an artistic ability to express change needs in terms that later users can easily understand. Good adaptability can capture changes before they undermine deeper infrastructure, but too many changes can overwhelm such features and lead to "quick fixes" that solve immediate issues at the expense of a system's long-term ability to adapt.
The real message, then, is this: We need new software engineering building materials, ones whose properties won't crumble like mud under the rains of constant change. We need updated methods for achieving maintainability, scalability, security, privacy, adaptability, and a long list of other -ilities—methods that embrace change instead of flinching from it. We'll end up with the same -ilities, but this time wrought in steel and stone instead of sand and clay. The authors of the articles in this focus section describe how they've embraced this difficult challenge of building such persistent software attributes. Their lessons show that constructing PSAs will necessarily be more automated, complex, dynamic, and geographically distributed than constructing simple -ilities.
The biggest challenge, however, lies ahead. In a world in which it seems that everyone is writing software, it's time for software engineering to take up the challenge of architectural leadership. We need to not just understand and demonstrate how to design software that stands the tests of time but also show how such ility-first designs will increase range, scale, and even beauty of future globe-spanning software creations.
We can't afford to live in a world of clay.
Once we learn how to build and sustain industrial-strength software properties that can weather even unrelenting downpours of change, how should we use them? Should we continue designing software much as we've done in the past, or will this new capability lead to deeper changes?
I believe the latter to be true. Although we can benefit from persistent software attributes in any form, taking full advantage of them will require a fundamental change in software engineering. I can sum up that change in two words: space awareness. Since its earliest days, software engineering has been unique among engineering disciplines in its ability to ignore real space. Many early programs and processes sprang up, lived, and died entirely within the confines of isolated computers, never having to face real-space location and geometry issues that are critical to success in disciplines such as civil, mechanical, and power engineering.
That world is gone. Nowadays, a task as simple as asking Google how to spell a word can result in electronic queries that travel across continents, and the success of many space programs depends on our ability to build and use interplanetary networks. In just a few decades, software has expanded from its cozy niches within isolated computers to vast distributed systems that encompass global and even interplanetary volumes of real space.
So, how is software's expansion into real space related to creating enduring software properties? Simple: When you build in real space, you use different properties for different system parts. Although a house made using only steel would certainly be strong, it would also be cold from a lack of insulation, dark from a lack of windows, dangerous to use with electricity, and far more costly than a house made using materials with varying properties. As we become better at building software that exhibits specific industrial-strength properties such as persistent security, availability, adaptability, and scalability, I believe that better options for composing such properties in real space will emerge. With that realization, software engineering will expand to take on a great new challenge: understanding how best to configure and compose software components in real space.
In this New Software Engineering, we'll select off-the-shelf components on the basis of the specific persistent properties they can contribute to larger systems, not just on functionality. Global skeletons with persistent availability will support local nodes that focus on persistent adaptability and scalability, while complex, location-aware security systems guard the entire system from malicious attacks. The New Software Engineering will be more complex, but it'll also lower costs and increase capabilities the way that good house construction does: by placing the right properties at the right real-space locations. Ironically, bringing awareness of real space issues into software design will also bring software engineering closer to its roots in older forms of engineering, ones that haven't had the luxury of neglecting a concept as basic as where components are located in space.
IEEE Software, I expect, will be sharing more of this vision of New Software Engineering in upcoming issues.