This Article 
 Bibliographic References 
 Add to: 
Systematic Testing of Model-Based Code Generators
September 2007 (vol. 33 no. 9)
pp. 622-634
Unlike for conventional compilers for imperative programming languages such as C or ADA, no establishedmethods for safeguarding artifacts generated by model-based code generators exist despite progress in the field of formalverification. Several test approaches dominate the engineering practice. This paper describes a general and toolindependenttest architecture for code generators used in model-based development. We evaluate the effectiveness of ourtest approach by means of testing optimizations performed by the TargetLink code generator, a widely accepted andcomplex development tool used in automotive model-based development.

[1] Simulink and Stateflow User Guide, The Mathworks, http:/, 2004.
[2] TargetLink Production Code Generation Guide, version 2, dSPACE, 2004.
[3] Real-Time Workshop Embedded Coder (User's Guide), version 4, TheMathWorks, 2004.
[4] G. Hamon and J. Rushby, “An Operational Semantics for Stateflow,” Proc. Fundamental Approaches to Software Eng. Conf. (FASE), pp. 229-243, 2004.
[5] P. Lapsley et al., DSP Processor Fundamentals. IEEE Press, 1997.
[6] H. Hanselmann et al., “Automatic Generation of Production Quality Code,” Proc. Embedded Intelligence Conf., Mar. 1999.
[7] T. Ueda and A. Ohata, “Trends of Future Powertrain Development and the Evolution of Powertrain Control Systems,” Proc. 30th Int'l Congress Transportation Electronics (Convergence '04), pp. 439-449, Oct. 2004.
[8] S. Muchnick, Advanced Compiler Design and Implementation. Morgan Kaufmann, 1997.
[9] T. Thomsen, “Integration of International Standards for Production Code Generation,” Proc. Soc. Automotive Eng. World Congress, Mar. 2003.
[10] W.J. Aldrich, “Using Model Coverage Analysis to Improve the Controls Development Process,” Proc. AIAA Modeling and Simulation Technologies Conf. and Exhibit, Aug. 2002.
[11] G.J. Myers, The Art of Software Testing. John Wiley & Sons, 1979.
[12] B. Beizer, Software-Testing Techniques. Van Nostrand Reinhold, 1990.
[13] I. Stürmer, D. Weinberg, and M. Conrad, “Overview of Existing Safeguarding Techniques for Automatically Generated Code,” Proc. Second Int'l ICSE Workshop Software Eng. for Automotive Systems (SEAS '05), May 2005.
[14] S. Glesner, “Using Program Checking to Ensure the Correctness of Compiler Implementations,” J. Universal Computer Science, vol. 9, no. 3, pp. 191-222, 2003.
[15] D. Scott, “Outline of a Mathematical Theory of Computation,” Proc. Fourth Ann. Princeton Conf. Information Systems and Sciences, pp. 169-176, 1970.
[16] M. Conrad and E. Sax, “Mixed Signals,” Testing Embedded Software, B.Broekman and E. Notenboom, eds., Addison-Wesley, 2003.
[17] M. Conrad, S. Sadeghipour, and H.W. Wiesbrock, “Automatic Evaluation of ECU Software Tests,” Proc. Soc. Automotive Eng. Int'l Congress, Mar. 2006.
[18] “The Ada Conformity Assessment Test Suite (ACATS),” version2.5, Ada Conformity Assessment Authority, http://www. , Apr. 2002.
[19] A. Burnard, “Verifying and Validating Automatically Generated Code,” Proc. Int'l Automotive Conf. (IAC), pp. 71-78, 2004.
[20] V. Santhanam, “The Anatomy of an FAA-Qualifiable Ada Subset Compiler,” Proc. Ann. ACM SIGAda Int'l Conf. Ada, pp. 40-43, 2002.
[21] R.D. Craig and S.P. Jaskiel, Systematic Software Testing. Artech House, 2002.
[22] S. Toeppe et al., “Practical Validation of Model Based Code Generation for Automotive Applications,” Proc. 18th AIAA/IEEE/SAE Digital Avionics System Conf., Oct. 1999.
[23] M. Beine, R. Otterbach, and M. Jungmann, “Development of Safety-Critical Software Using Automatic Code Generation,” Proc. Soc. Automotive Eng. World Congress, Mar. 2004.
[24] M.A. Dave, “Compiler Verification: A Bibliography,” ACM SIGSOFT Software Eng. Notes, vol. 28, no. 6, 2003.
[25] G. Goos and W. Zimmermann, “Verifying Compilers and ASMs, Abstract State Machines,” Lecture Notes in Computer Science, vol.1912, pp. 177-202, 2000.
[26] G. Necula, ”Translation Validation for an Optimizing Compiler,” Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, 2000.
[27] L. Zuck, A. Pnueli, Y. Fang, and B. Goldberg, “VOC: A Methodology for the Translation Validation of Optimizing Compilers” J.Universal Computer Science, vol. 9, no. 3, pp. 223-247, 2003.
[28] K.V. Hanford, “Automatic Generation of Test Cases,” IBM SystemJ., vol. 9, no. 4, pp. 242-257, 1970.
[29] J. Bauer and A. Finger, “Test Plan Generation Using Formal Grammars,” Proc. Fourth Int'l Conf. Software Eng., pp. 425-432, 1979.
[30] A. Celentano et al., “Compiler Testing Using a Sentence Generator,” Software—Practice and Experience, vol. 10, pp.887-918, 1980.
[31] F. Bazzichi and L. Spadafora, “An Automatic Generator for Compiler Testing,” IEEE Trans. Software Eng., vol. 8, no. 4, pp.343-353, 1982.
[32] W. Homer and R. Schooler, “Independent Testing of Compiler Phases Using a Test Case Generator,” Software—Practice and Experience, vol. 19, no. 1, pp. 53-62, 1989.
[33] A.S. Boujarwah, K. Saleh, and J. Al-Dallal, “Testing Syntax and Semantic Coverage of Java Language Compilers,” Information and Software Technology, vol. 41, pp. 15-28, 1999.
[34] E.G. Sirer and B.N. Bershad, “Using Production Grammars in Software Testing,” Proc. Second Conf. Domain Specific Languages, Oct. 1999.
[35] J. Harm and R. Lämmel, “Testing Attribute Grammars,” Proc. Third Workshop Attribute Grammars and Their Application (WAGA), 2000.
[36] S.V. Zelenov et al., ”Test Generation for Compilers and Other Formal Text Processors,” Programming and Computer Software, vol. 29, no. 2, pp. 104-111, 2003.
[37] P. Purdom, “A Sentence Generator for Testing Parsers,” BIT, vol. 12, no. 3, pp. 366-375, 1972.
[38] A.S. Boujarwah and K. Saleh, “Compiler Test Case Generation Methods: A Survey and Assessment,” Information and Software Technology, vol. 39, pp. 617-625, 1997.
[39] ANSI/ISO FIPS-160 C Validation Suite (ACVS), http:/www., Mar. 2005.
[40] Plum Hall Validation Suite for ANSI C, http:/www.plumhall. com, Mar. 2005.
[41] Associated Compiler Experts (ACE) SuperTest C&C++ Test and Validation Suite, http:/, Mar. 2005.
[42] Nullstone for C, http:/, Dec. 2005.
[43] M. Tonndorf, “Ada Conformity Assessments: A Model for Other Programming Languages?“ ACM SIGAda Ada Letters, vol. 19, no. 3, pp. 89-99, 1999.
[44] K. Marriott and B. Meyer, “On the Classification of Visual Languages by Grammar Hierarchies,” J. Visual Languages and Computing, vol. 8, no. 4, pp. 375-402, 1997.
[45] “Functional Safety of Electrical/Electronic/Programmable Electronic Safety-Related Systems,” IEC 61508, Int'l Electrotechnical Commission, 1999.
[46] “ISO/WD 26262: Road Vehicles—Functional Safety,” working draft, Sept. 2005.
[47] “Software Considerations in Airbone Systems and Equipment Certification,” RTCA/DO-178B, Requirements and Technical Concepts for Aviation Inc., Dec. 1992.
[48] I. Stürmer and M. Conrad, “Test Suite Design for Code Generation Tools,” Proc. 18th Int'l IEEE Conf. Automated Software Eng., pp. 286-290, 2003.
[49] G. Rozenberg, Handbook of Graph Grammars and Computing by Graph Transformations, vol. 1. World Scientific, 1997.
[50] P. Baldan, B. König, and I. Stürmer, “Generating Test Cases for Code Generators by Unfolding Graph Transformation Systems,” Proc. Int'l Conf. Graph Transformation (ICGT '04), pp. 194-210, 2004.
[51] M. Grochtmann and K. Grimm, ”Classification Trees for Partition Testing,” Software Testing, Verification, and Reliability, vol. 3, pp. 63-82, 1993.
[52] I. Stürmer, “Systematic Testing of Code Generation Tools— ATestsuite-Oriented Approach for Safeguarding Model-Based Code Generation,” dissertation, Pro Business, 2006.
[53] E. Lehmann and J. Wegener, ”Test Case Design by Means of the CTE/XL,” Proc. Eighth European Int'l Conf. Software Testing, Analysis, and Rev. (EuroSTAR '00), 2000.
[54] I. Stürmer, P. Pepper, and S. Heck, ”The Test Model Generator ModeSSa,” Software & System Modeling, submitted for publication.
[55] G. Karsai et al., ”On the Use of Graph Transformation in the Formal Specification of Model Interpreters,” J. Universal Computer Science, vol. 9, no. 11, pp. 1296-1321, 2003.
[56] Reactis User's Guide, Reactive Systems, Aug. 2002.
[57] A. Baresel et al., “The Interplay between Model Coverage and Code Coverage,” Proc. 11th European Int'l Conf. Software Testing, Analysis, and Rev., Dec. 2003.
[58] J. Wegener, H. Stahmer, and A. Baresel, ”Evolutionary Test Environment for Automatic Structural Testing,” Information and Software Technology, vol. 43, pp. 851-854, 2001.
[59] K. Lamberg et al., “Model-Based Testing of Embedded Automotive Software Using MTest,” J. Passenger Cars—Electronic and Electrical Systems, vol. 7, pp. 132-140, July, 2005.
[60] P.D. Edwards, “The Use of Automatic Code Generation Tools in the Development of Safety-Related Embedded Systems,” Proc. Vehicle Electronic Systems European Conf. and Exhibition, 1999.
[61] C. Kaner, Lessons Learned in Software Testing. Wiley & Sons, 2001.

Index Terms:
Testing and Debugging
Ingo Stuermer, Mirko Conrad, Heiko Doerr, Peter Pepper, "Systematic Testing of Model-Based Code Generators," IEEE Transactions on Software Engineering, vol. 33, no. 9, pp. 622-634, Sept. 2007, doi:10.1109/TSE.2007.70708
Usage of this product signifies your acceptance of the Terms of Use.