This Article 
 Bibliographic References 
 Add to: 
Software Testing and Verification in Climate Model Development
Nov.-Dec. 2011 (vol. 28 no. 6)
pp. 49-55
Thomas Clune, NASA/Goddard Space Flight Center
Richard Rood, University of Michigan
Over the past 30 years, most climate models have grown from relatively simple representations of a few atmospheric processes to complex multidisciplinary systems. Computer infrastructure over that period has gone from punchcard mainframes to modern parallel clusters. Model implementations have become complex, brittle, and increasingly difficult to extend and maintain. Verification processes for model implementations rely almost exclusively on some combination of detailed analyses of output from full climate simulations and system-level regression tests. Besides being costly in terms of developer time and computing resources, these testing methodologies are limited in the types of defects they can detect, isolate, and diagnose. Mitigating these weaknesses of coarse-grained testing with finer-grained unit tests has been perceived as cumbersome and counterproductive. Recent advances in commercial software tools and methodologies have led to a renaissance of systematic fine-grained testing. This opens new possibilities for testing climate-modeling-software methodologies.

1. S. Shackley, "Epistemic Lifestyles in Climate Change Modeling," Changing the Atmosphere, C.C. Miller, and P.N. Edwards eds., MIT Press, 2001, pp. 107–133.
2. D.E. Post, and R.P. Kendall, "Software Project Management and Quality Engineering Practices for Complex, Coupled Multiphysics, Massively Parallel Computational Simulations: Lessons Learned from ASCI," Int'l J. High Performance Computing Applications, vol. 18, no. 4, 2004, pp. 399–416.
3. S.M. Easterbrook and T.C. Johns, "Engineering the Software for Understanding Climate Change," Computing in Science and Eng., vol. 11, no. 6, 2009, pp. 64–74.
4. C. Hill et al., "Architecture of the Earth System Modeling Framework," Computing in Science and Eng., vol. 6, no. 1, 2004, pp. 18–28.
5. N. Oreskes, K. Shrader-Frechette, and K. Belitz, "Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences," Science, vol. 263, no. 5147, 1994, pp. 641–646.
6. H. Guillemot, "Connections between Simulations and Observation in Climate Computer Modeling: Scientist's Practices and 'Bottom-Up Epistemology' Lessons," Studies in History and Philosophy of Modern Physics, vol. 41, no. 3, 2010, pp. 242–252.
7. R.B. Rood et al., Algorithm Theoretical Basis Document Version 1.01, tech. report, Data Assimilation Office, Goddard Space Flight Center, 1996; for_scientists/atbd/docs/DAOatbd-dao.pdf .
8. RTI, The Economic Impacts of Inadequate Infrastructure for Software Testing, tech. report, US Nat'l Inst. Standards and Technology, 2002; .
9. S. Vasquez, S. Murphy, and C. DeLuca, "Earth System Modeling Framework Software Developer's Guide,"29 Aug. 2011; .
10. G.A. Schmidt et al., "Present-Day Atmospheric Simulations Using GISS ModelE: Comparison to In Situ, Satellite, and Reanalysis Data," J. Climate, vol. 19, no. 2, 2006, pp. 153–192.
11. K. Beck, Test-Driven Development: By Example, Addison-Wesley, 2003, p. 240.
12. M. Feathers, Working Effectively with Legacy Code, Prentice-Hall, 2004, p. 456.
13. J. Gravner and D. Griffeath, "Modeling Snow Crystal Growth: A Three-Dimensional Mesoscopic Approach," Physical Rev. E., vol. 79, no. 1, 2009, doi:10.1103/PhysRevE.79.011601.
14. M.R. Schoeberl and L. Sparling, "Trajectory Modeling," Diagnostic Tools in Atmospheric Physics, J.C. Gille, and G. Visconti eds., IOS Press, 1995, pp. 419–430.
15. J. Gravner, and D. Griffeath, "Modeling Snow Crystal Growth: A Three-Dimensional Mesoscopic Approach," Physical Rev. E, vol. 79, no. 1, 2009.
16. "Photran—an Integrated Development Environment and Refactoring Tool for Fortran," Eclipse Foundation, 2011; www.eclipse.orgphotran.

Index Terms:
software engineering, program verification, testing strategies, testing and debugging
Thomas Clune, Richard Rood, "Software Testing and Verification in Climate Model Development," IEEE Software, vol. 28, no. 6, pp. 49-55, Nov.-Dec. 2011, doi:10.1109/MS.2011.117
Usage of this product signifies your acceptance of the Terms of Use.