, University of Helsinki
, IT University of Copenhagen
, University of British Columbia
Pages: pp. 16-22
Agile development has significantly impacted industrial software development practices. However, despite its wide popularity, there's an increasing perplexity about software architecture's role and importance in agile approaches. Advocates of architecture's vital role in achieving quality goals for large software-intensive systems doubt the scalability of any development approach that doesn't pay sufficient attention to architecture. This especially applies to domains such as automobiles,
telecommunications, finance, and medical devices. Companies where architectural practices are well developed often tend to see agile practices as amateurish, unproven, and limited to very small Web-based sociotechnical systems. 1
Conversely, proponents of agile approaches usually see little value for a system's customers in the upfront design and evaluation of architecture. They perceive software architecture as something from the past, equating it with big design up-front (BDUF)—a bad thing—leading to massive documentation and implementation of YAGNI (you ain't gonna need it) features. They believe that architectural design has little value, that a metaphor should suffice in most cases, 2 and that the architecture should emerge gradually sprint after sprint, as a result of successive small refactoring.
Interest is growing in separating the facts from myths about the necessity, importance, advantages, and disadvantages of having agile and architectural approaches coexist, and that's the theme of this special issue.
Any debate, discussion, or effort to assess the necessity of combining agile and architecture should start with questions such as: Are these views contradictory, opposing, or complementary? Do the proclaimed dichotomies between agile and architecture have any truth? What steps will let project teams benefit from the best of both by ignoring unnecessary values or requirements?
Jim Highsmith defines agility as "the ability of an organization to both create and respond to change in order to profit in a turbulent business environment." 3 Sanjiv Augustine notes that agile development methods such as Extreme Programming (XP), scrum, feature-driven development, lean, Crystal, and so on have common characteristics, such as
They all also more or less adhere to the values of the Agile Manifesto ( www.agilemanifesto.org).
The Rational Unified Process (RUP) defines software architecture as the
set of significant decisions about the organization of a software system, the selection of the structural elements and their interfaces by which the system is composed together with their behavior as specified in the collaboration among those elements, the composition of these elements into progressively larger subsystems, the architectural style that guides this organization, these elements and their interfaces, their collaborations, and their composition. Software architecture is concerned with not only structure and behavior but also usage, functionality, performance, resilience, reuse, comprehensibility, economic and technological constraints and trade-offs, and aesthetics. 5
The tension seems to lie on the axis of adaptation versus anticipation. Agile methods want to be resolutely adaptive: deciding at the "last responsible moment" or when changes occur. Agile methods perceive software architecture as pushing too hard on the anticipation side: planning too much in advance. Perhaps we can find a balance between these two extreme approaches and mind-sets.
When discussing the direction of this special issue, Craig Larman asserted that this tension between agility and architecture might be a false dichotomy. Indeed, there are plenty of such artificial splits. Some are inadvertent; some are intentional, to prop up a certain message: agile versus waterfall or agile versus disciplined. Like many others in software development research and practice, we strongly believe that a healthy focus on architecture isn't antithetic to any agile process. The tenors of various agile methods also seem to agree. Along these lines, Satoshi Basaki noted, "It seems that many agile method users misunderstand what agile methods are, just ignore architecture, and jump onto refactoring" as the one and only panacea.
There's also the drive to "deliver value to the stakeholders" right from the first sprint or iteration. But what if the developers, not just the end users, are a key class of stakeholders? Alistair Cockburn developed strategies for starting with a walking skeleton, then evolving it iteratively. 6 Mary and Tom Poppendieck came up with the notion of "divisible system architecture." 7 And finally, Kent Beck's advises that "architecture is just as important in XP projects as it is in any software project. Part of the architecture is captured by the system metaphor [one of the XP practices]." 2
So what issues must we address to begin reconciliation?
There are multiple levels to understanding the apparent conflict between agile development and architecture: semantics, scope, life cycle, role, documentation, methods, value, and cost.
What does a particular project or organization mean by architecture? The concept often has fuzzy boundaries. In particular, not all design is architecture. Agreeing on a definition is a useful exercise and a good starting point.
If the yellow circle in Figure 1a represents all decisions made for a software system, design decisions (purple) will be a subset, leaving many decisions at the programming level. In turn, a small subset of these design decisions will be architecturally significant (red). Some decisions are made "upstream" in the form of requirements constraints (green).
Figure 1 Reality vs. perception. (a) While programmers make a great deal of design decisions when developing a software system, only few are architecturally significant. (b) Most design decisions, even minor ones, are perceived as having an impact on the software architecture.
Unfortunately, the decision landscape is beginning to look more like Figure 1b, where not much distinction is left between design and architecture (purple equals red). Mary Shaw warned us a long time ago: "Do not dilute the meaning of the term architecture by applying it to everything in sight." 8 Not all design decisions are architectural. Few are, actually, and in many projects, they're already made on day one.
How much architectural activity will a project need? It usually fluctuates widely depending on the project's context. By context we mean the project's environment—the organization, the domain, and so on—as well as specific factors such as project size, stable architecture, business model, team distribution, rate of change, the system's age, criticality, and governance (see Figure 2). Other influences can include the market situation, the company's power and politics, the product's expected life span, product type, organizational culture, and history.
Figure 2 Some factors making up a project's context. Like other software design and implementation activities, the project's context, including the customer, needs to drive the project's architectural activities.
In an agile project's "sweet spot," few architectural activities might be needed. However, many large, complicated projects require significant architectural effort. For these projects, agile methods must suit the specific circumstances inherent in the development's context.
When should we focus on architecture? Well, early enough, because "architecture encompasses the set of significant decisions about the structure and behavior of the system." 5 These decisions will prove the hardest to undo, change, and refactor, which means to not only focus on architecture but also interleave architectural "stories" and functional "stories" in early iterations.
Who are the architects? On a large, challenging, novel system, you might need a good mix of experience:
to follow Martin Fowler's metaphors. 8
How much of an explicit description of the architecture does the project need? In most cases, an architectural prototype will suffice, starting with a walking skeleton along with a small number of solid metaphors to convey the message. But some circumstances require more explicit software architecture documentation—for example, to communicate to a large audience or to comply with external regulations.
How do we identify and resolve architectural issues? Although we've shown that some agile methods aren't opposed to the concept of architecture, they're all rather silent on how to identify architecturally significant requirements, perform incremental architectural design, validate architectural features, and so on. Architectural methods exist (see the "Related Work" sidebar for a quick inventory) but aren't well known.
What's the cost of architecture anyhow? All agile approaches strive to deliver business value early and often. The problem often seems that whereas the architecture's cost is somewhat visible, its value is hard to grasp, because it remains invisible. An approach such as incremental funding 2 might let us find the right compromise between architecture and functionality, without falling into the trap of BDUF.
On the basis of our observations, experiences, ongoing field studies, and the articles in this special issue, we conclude that emphasis has increased on software architects' vital role and responsibilities in successfully combining agile and architectural methodologies. Software architects are expected to facilitate software development projects as well as represent the overall system's quality attributes. How should software architects' roles change? Or what new responsibilities should they take? We already know that architects can be quite satisfied with agile development. Kati Vilkki surveyed more than 2,400 developers, testers, architects, and managers at Nokia Siemens Networks. She concluded that more than 70 percent of the architects were either satisfied or very satisfied when considering agile development's impact on their work. 9 One reason might be the improved feedback cycle and their new role closer to the actual development.
It's also becoming clear that software developers are equally important in successfully combining agile and architecture approaches because it's up to the development team how to use various architectural artifacts and documents. So, we need to know agile software developers' perceptions of software architecture's relevance and usefulness in their daily activities. It's also important to understand what agile developers think about combining architectural principles and agile approaches in development projects. One way is to find out how agile teams use software architecture. If agile developers don't consider software architecture relevant to their day-to-day activities, it would be difficult, even impossible, to convince them to use architectural principles and integrate artifacts in agile development.
Contrary to the perception that architecture is less relevant to agile developers, Davide Falessi, Giovanni Cantone, and Salvatore Alessandro Sarcia' found that agile developers perceive software architecture as relevant on the basis of aspects such as communication among team members, inputs to subsequent design decisions, documenting design assumptions, and evaluating design alternatives. These findings are consistent with other reports that agile teams tend to have some sort of architectural documentation. 10
Falessi's results also suggest that agile developers usually focus on software architecture while working on complex software systems that can be characterized by geographical distributed development, many stakeholders, or many requirements or LOC. This is in line with the common observation that as software complexities increase, so too does the relevance and importance of software architecture and its related documents.
Each development approach is usually based on some fundamental requirements. From the outset, agile values appear to contradict good architectural practices. Any attempt to combine agile and architecture approaches must consider the potential value clash that development teams might perceive. Falessi and his colleagues concluded that the respondents in their study found that agile values and architectural principles support each other.
Timeliness is vital for architectural decisions because they can prove difficult or costly to change. So, architects working with agile or nonagile teams must know the most appropriate time for key architectural decisions. Making such decisions too early can constrain development teams in general and agile development teams in particular. Waiting too long to take care of architecturally significant decisions can put the whole project in chaos. To help stakeholders make timely decisions, Stuart Blair, Tim Cull, and Richard Watt present an approach called responsibility-driven architecture that exploits concepts of the real options theory. A simple spreadsheet-based tool and a responsible, accountable, consulted, and informed (RACI) matrix support their approach to track stakeholders' decision-making and responsibilities.
User stories in agile development relate primarily to functional requirements; this means that nonfunctional requirements can sometimes get completely ignored. Unfulfilled nonfunctional requirements can make an otherwise fully functioning system useless or risky. A main objective of integrating architectural approaches in agile processes is to enable software development teams to pay attention to both functional and nonfunctional requirements. Roland Faber, in "Architects as Service Providers," proposes that architecture should represent nonfunctional requirements. He presents an architectural process in the context of agile projects that exploits well-known architectural concepts and principles. He also provides advice on tailoring scrum to incorporate the concept of architecture as a service and facilitating communication between architecture and development teams.
This special issue closes with an article by James Madison, who advocates the coexistence of agile and architecture as complementary approaches and principles. He emphasizes the software architect's vital role as a linchpin for combining the two. Madison's approach, called agile architecture, advocates using agile to get to a good architecture by appropriately applying suitable combinations of architectural functions (such as communication, quality attributes, and design patterns) and architectural skills at four points (up-front planning, storyboarding, sprint, and working software) in the development life cycle. Madison provides several examples from successful applications of agile architecture to 14 projects in various domains.
So, what does this special issue offer for those interested in designing and deploying agile processes engrained with sound architectural principles and practices?
In a large software organization, implementing agile approaches isn't a straightforward adoption problem. Most likely, it will take several years to shorten the feedback cycles to benefit from the adaptability and earlier value-creation opportunities. Failure is a natural part of process improvement. We believe that stories and studies of failure often shed more light than those of success. We encourage practitioners and researchers to publish these more actively to stimulate discussion and increase learning in the community.