Issue No. 05 - September/October (2010 vol. 27)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/MS.2010.130
Hakan Erdogmus , Kalemun Research
Considered a gross misnomer by some, earned value for progress tracking in software development projects is still little known in the software industry. But the concept shouldn't be so unfamiliar: tracking progress by earned value is similar to tracking progress through burndown charts ( http://xprogramming.com/articles/bigvisiblecharts)—the ubiquitous, simple and powerful visuals that are so popular in agile software development. The two techniques, although developed independently in very different contexts, are similar in terms of their information content. This isn't a new discovery: Alistair Cockburn talked about the relationship between earned value and burndown charts as early as 2004 in Chapter 3 of his book Crystal Clear (Addison-Wesley, 2004). Section 7.3 of Grigori Melnik's and Gerard Meszaros' volume 1 of Acceptance Test Engineering Guide ( http://testingguidance.codeplex.com) also discusses the same relationship.
Burndown charts track progress using a "count-down to zero" approach. Remaining work scheduled to be in a future release or project is represented as the sum of features that are weighted using an effort estimate, such as story points or ideal programming days. Earned value achieves the same effect by adopting a "count up to 100 percent complete functionality" approach. Cockburn argues that while burndown charts are emotionally more powerful, they treat planned work as fixed rather than expandable. This limitation makes them less than ideal when the scope changes midstream within the tracking period.
Why Consider Earned Value?
Earned value differs from the burndown approach in two ways. First, earned value expresses the amount of work in monetary terms by weighting a feature's percent completeness by the feature's estimated cost. Detractors of earned value see one major problem with this representation: by referring to the effort as "value," the monetary equivalent of a feature is seemingly equated to the feature's business value. They are right. Earned value is essentially an expression of the feature's completeness in cost, or effort, terms.
The second distinction is the emphasis placed by earned value on tracking progress against a plan, an emphasis that is absent in the burndown approach. The presence of a point of reference against which to compare progress introduces a number of additional metrics that burndown charts do not incorporate. Tracking progress through earned value and related metrics give rise to earned value management. Let's elaborate on this larger concept a bit.
What's Earned Value Management?
Earned value management (EVM) is a budget and schedule tracking technique developed by the Project Management Institute ( Practice and Standard for Earned Value Management, 2006). EVM connects tightly with other PMI standards and concepts, such as work breakdown structures. The sidebar ("Earned Value Management and Agile Projects") I contributed to volume 1 of Acceptance Test Engineering Guide summarizes EVM in the following way. Earned value tracks the relative progress of a scheduled work unit as a percentage of the allocated budget that has been spent for that work unit. In software development, the work unit might correspond to a single feature represented by a textual requirement, a use case, or a story card. Or it might correspond to a bug fix, or a milestone such as "architecture defined." If a work unit is 80 percent complete and was allocated a $100 budget, then its current earned value equals $80. The total value earned by the project is the sum of the earned values of all the work units scheduled and budgeted for that project. When the project is complete, its total earned value equals the project's total budget. Therefore, unlike the name suggests, earned value is essentially a relative effort tracking metric.
Cost and Schedule Variance
In EVM, actual costs are also tracked. If at any point in the project, the actual accumulated cost exceeds the earned value of the project at that time, the project is considered over budget, with the shortfall representing a budget overrun. In EVM terms, this shortfall, or slack if actual costs are below earned value, is called cost variance.
If the accumulated earned value at a point in time is below the estimated total expenditures according to the planned budget, then the project is considered over schedule, with the difference representing the monetary expression of the project's schedule shortfall at that point. In EVM terms, the schedule shortfall or slack is called schedule variance. Schedule variance can also be expressed temporally, in calendar time.
Figure 1 illustrates schedule variance (in both monetary and temporal terms) and cost variance for a project with an estimated total budget of $5 million and an estimated schedule of one year. In July 2010, at an earned value of $2.5 million, the project has a total earned value that's only 50 percent of its planned total cost, but it should have accumulated 75 percent of that total cost according to the project plan. Therefore it's over schedule by $1.25 million, still 50 percent, in monetary terms. Since the project should have accumulated that much earned value back in April 2010, it's three months behind in calendar time terms. This is the schedule variance. The project is also 110 percent over budget on July 2010 because it has accumulated an actual cost of $5.25 million at that date compared to the $2.5 million it has earned.
EVM in a Time-Boxed Incremental Context
In "Earned value and Agile Reporting" ( Proceedings of AGILE '06, IEEE Computer Society Press, 2006, pp. 17–22), Mike Griffiths and Anthony Cabri advocated applying EVM incrementally, on an iteration-by-iteration or release-by-release basis. In some projects, a grand plan for the whole project may not exist. However, a local plan in terms of work scheduled for the current scope (features or stories) together with estimated costs (resources allocated to the current scope) could be available and reliable enough to serve as a reference point on a per-iteration or per-release basis. Budget and schedule overruns can then be calculated within each such increment, whether for a single iteration or release, using EVM. Effort or a proxy for effort such as story points can be substituted for cost, just as done in burndown charts.
Even if mini, local plans aren't available and scope changes over the course of the project, EVM can still be applied. Cockburn describes how to handle scope changes using an idea suggested by Phil Goodwin and Russ Rufer ( http://alistair.cockburn.us/Earned-value+and+burn+charts).
EVM in a Kanban-Style Incremental Context
If a Kanban-based, continuous-workflow approach is adopted instead of timeboxes, applying EVM becomes problematic. Because typically budget plans or estimates are absent altogether in Kanban systems, variance-tracking metrics cease to exist, but tracking actual throughput is still feasible. Tracking project progress reduces to tracking flow. In Chapter 12 of his book Kanban: Successful Evolutionary Change for Your Technology Business (Blue Hole Press, 2010), David J. Anderson explains the metrics and reporting mechanism applicable in a Kanban context. Anderson recommends the use of cumulative flow diagrams that plot total number of work items as a function of time. These diagrams can be converted to actual cost curves of the kind depicted in Figure 1 by weighting each work item with the item's actual cost. Earned value can then be defined to coincide with cumulative actual cost rather than cumulative relative cost. Effectively, the plan equals reality: there's no need to reason about schedule or cost variance.
Substituting Business Value for Earned Value
When customer-assessed value, or business value, is more critical to project funders than textbook earned value as defined in EVM, a substitution of concepts may be warranted. An EVM-like model can track the business value provided that each work unit has a tangible financial value or an intangible utility assigned by the product owner or project funder.
Business value can be expressed in absolute, monetary terms (in currency) or in abstract, utility terms (for example, in utils). Before the fact, the business value is an estimate based on the product owner's or project funder's assessment. After the fact, it represents a reassessment in an accepted and delivered product, thus proxying actual value. Unlike actual cost in classical EVM, actual business value is thus often subjective rather than observed or measured. Although actual business value is hard to measure, especially at the feature level, it can still be informed and backed up by meaningful economic analysis.
When business value replaces earned value, adjustments might be required. For example, only fully deployed features may earn their respective business value. Similarly, if a feature is useful in the field only when delivered together with another feature, it shouldn't earn its respective business value until all the features on which it depends are also deployed. Percentage-based adjustments proportional to a feature's status of completeness might not make much sense either for tracking business value. Partial or percentage business value earned is only sensible at the whole project level.
What's the verdict on EVM? EVM might not be adaptable in an out-of-the-box form for our industry, but it does offer a few extras that better-known alternatives don't mention. The EVM approach deserves more attention from the software development community in environments where planning and estimation is part of the culture.
Selected CS articles and columns are also available for free at http://ComputingNow.computer.org.