This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Using Diagram Generation Software to Improve Diagram Recognition: A Case Study of Music Notation
November 1999 (vol. 21 no. 11)
pp. 1121-1136

Abstract—Diagrams are widely used in society, to transmit information such as circuit designs, music, mathematical formulae, architectural plans, and molecular structure. Computers must process diagrams both as images (marks on paper) and as information. A diagram recognizer translates from image to information and a diagram generator translates from information to image. Current technology for diagram generation is ahead of the technology for diagram recognition. Diagram generators have extensive knowledge of notational conventions which relate to readability and aesthetics, whereas current diagram recognizers focus on the hard constraints of the notation. To create a recognizer capable of exploiting layout information, it is expedient to reuse the expertise in existing diagram generators. In particular, we discuss the use of Lime (our editor and generator for music notation) to proofread and correct the raw output of MIDIScan (a third-party commercial recognizer for music notation). Over the past several years, this combination of software has been distributed to thousands of users.

[1] A. Abak, U. Barns, and B. Sankur, “The Performance Evaluation of Thresholding Algorithms for Optical Character Recognition,” Proc. Fourth Int'l Conf. Document Analysis and Recognition, pp. 697-700, Ulm, Germany, Aug. 1997.
[2] D. Bainbridge and T. Bell, “An Extensible Optical Music Recognition System,” Proc. 19th Australasian Computer Science Conf., pp. 308-317, Melbourne, Australia, 1996.
[3] D. Bainbridge and N. Carter, “Automatic Reading of Music Notation,” Handbook of Character Recognition and Document Image Analysis, H. Bunke and P. Wang, eds., pp. 583-603, World Scientific, 1997.
[4] A. Belkin, “Macintosh Notation Software: Present and Future,” Computer Music J., vol. 18, no. 1, pp. 53-69, Spring 1994.
[5] J. Bertin, Semiology of Graphics: Diagrams, Networks, and Maps. Univ. of Wisconsin Press, 1983.
[6] D. Blostein, “General Diagram-Recognition Methodologies,” Graphics Recognition—Methods and Applications, R. Kasturi and K. Tombre, eds., pp. 106-122, Springer Verlag, 1996.
[7] D. Blostein and H. Baird, “A Critical Survey of Music Image Analysis,” Structured Document Image Analysis, H. Baird, H. Bunke, and K. Yamamoto, eds., pp. 405-434, Springer Verlag, 1992.
[8] D. Blostein and A. Grbavec, “Recognition of Mathematical Notation,” Handbook of Character Recognition and Document Image Analysis, H. Bunke and P. Wang, eds., pp. 557-582, World Scientific, 1997.
[9] D. Blostein and L. Haken, “Justification of Printed Music,” Comm. ACM, vol. 34, no. 3, pp. 88-99, Mar. 1991.
[10] D. Blostein and L. Haken, “The Lime Music Editor: A Diagram Editor Involving Complex Translations,” Software—Practice and Experience, vol. 24, no. 3, pp. 289-306, Mar. 1994.
[11] T. Chaundy, P. Barrett, and C. Batey, The Printing of Mathematics. Oxford Univ. Press, 1957.
[12] A. Chhabra, “Graphic Symbol Recognition: An Overview,” Graphics Recognition—Algorithms and Systems, K. Tombre and A. Chhabra, eds., pp. 68-79, Springer Verlag, 1998.
[13] L. Cordella and A. Marcelli, “An Alternative Approach to the Performance Evaluation of Thinning Algorithms for Document Processing Applications,” Graphics Recognition—Methods and Applications, R. Kasturi and K. Tombre, eds., pp. 13-22, Springer Verlag, 1996.
[14] P. Chou and G. Kopec, “A Stochastic Attribute Grammar Model of Document Production and Its Use in Document Image Decoding,” Proc. Document Recognition II, pp. 66-73, 1995.
[15] D. Dori, D. Doermann, C. Shin, R. Haralick, I. Phillips, M. Buchman, and D. Ross, “The Representation of Document Structure: A Generic Object-Process Analysis,” Handbook of Character Recognition and Document Image Analysis, H. Bunke and P. Wang, eds., pp. 421-456, World Scientific, 1997.
[16] J. Gourlay, “Spacing a Line of Music,” Technical Report OSU-CISRC-10/87-TR35, Dept. of Computer and Information Science, Ohio State Univ., 1987.
[17] L. Haken and D. Blostein, “The Tilia Music Representation: Extensibility, Abstraction, and Notation Contexts for the Lime Music Editor,” Computer Music J., vol. 17, no. 3, pp. 43-58, 1993.
[18] L. Haken and D. Blostein, “A New Algorithm for Horizontal Spacing of Printed Music,” Proc. Int'l Computer Music Conf., pp. 118-119, Banff, Canada, Sept. 1995.
[19] R.M. Haralick, “Document Image Understanding: Geometric and Logical Layout,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 385-390, 1994.
[20] http:/datura.cerl.uiuc.edu.
[21] http://mistral.ere.umontreal.ca~belkina(A. Belkin currently maintains the NIFF specification).
[22] Int'l J. Document Analysis and Recognition, Feb. 1998-present.
[23] S. Joseph, S. Ablameyko, and T. Pridmore, “Knowledge Based Interpretation of Engineering Drawings and Maps,” Proc. Int'l Workshop Graphics Recognition, pp. 189-199, University Park, Pa., Aug. 1995.
[24] T. Kanungo, R. Haralick, and D. Dori, “Understanding Engineering Drawings: A Survey,” Proc. Int'l Workshop Graphics Recognition, pp. 119-130, University Park, Pa., Aug. 1995.
[25] D. Knuth, “Mathematical Typography,” Bulletin Am. Math. Soc., vol. 1, no. 2, Mar. 1979.
[26] G.E. Kopec and P.A. Chou, “Document Image Decoding Using Markov Source Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 6, pp. 602-617, June 1994.
[27] G. Kopec, P. Chou, and D. Maltz, “Markov Source Model for Printed Music Decoding,” J. Electronic Imaging, vol. 5, no. 1, pp. 7-14, Jan. 1996.
[28] G.L. Lohse et al., "A Classification of Visual Representations," CACM, Vol. 37, No. 12, Dec. 1994, pp. 36-49.
[29] S. Nieminen, J. Sauvola, T. Seppänen, and M. Pietikäinen, “Benchmarking System for Document Analysis Algorithms,” Proc. Document Recognition V, pp. 100-111, 1998.
[30] L. O'Gorman and R. Kasturi, Document Image Analysis. IEEE CS Press, 1995.
[31] F. Paulisch and W. Tichy, “EDGE: An Extendible Graph Editor,” Software—Practice and Experience, vol. 20, pp. 63-88, 1990.
[32] M. Petre, Why Looking Isn't Always Seeing: Readership Skills and Graphical Programming Comm. ACM, vol. 38, no. 6, pp. 33-44, June 1995.
[33] I. Phillips, J. Liang, A. Chhabra, and R. Haralick, “A Performance Evaluation Protocol for Graphics Recognition Systems,” Graphics Recognition: Algorithms and Systems, Second Int'l Workshop, GREC '97, Selected Papers, pp. 372-389, Lecture Notes in Computer Science, vol. 1389.Berlin: Springer-Verlag, 1998.
[34] L. Protsko, P. Sorenson, J. Tremblay, and D. Schaefer, “Towards the Automatic Generation of Software Diagrams,” IEEE Trans. Software Eng., vol. 17, no. 1, pp. 10-21, Jan. 1991.
[35] Proc. Ann. Symposia Document Analysis and Information Retrieval, Las Vegas, 1992, 1993, 1994, 1995, 1996, 1997.
[36] Proc. IAPR Workshop Document Analysis Systems, Kaiserslautern, Germany, Oct. 1994; Malvern, Pa., Oct. 1996; Nagano, Japan, Nov. 1998.
[37] Proc. IAPR Workshop Graphics Recognition, Pennsylvania, 1995; Nancy, France, 1997; Jaipur, India, 1999, http://graphics.basit.com/iapr-tc10GREC99 /.
[38] Proc. Int'l Conf. Document Analysis and Recognition, France, 1991; Japan 1993; Canada, 1995; Germany, 1997; Japan, 1999.
[39] G. Read, Music Notation: A Manual of Modern Practice, second ed. New York: Taplinger Publishing, 1979.
[40] T. Ross, The Art of Music Engraving and Processing, second ed. Miami: Hansen Books, 1970.
[41] D. Roush, “Music Formatting Guidelines,” Technical Report OSU-CISRC-3/88-TR10, Dept. of Computer and Information Science, The Ohio State Univ., 1988.
[42] J. Serrano, “The Use of Semantic Constraints on Diagram Editors,” Proc. 11th IEEE Symp. Visual Languages, Darmstadt, Germany, pp. 211-216, Sept. 1995.
[43] J. Sloboda, “The Uses of Space in Music Notation,” Visual Language, vol. XV, no 1, pp. 86-112, 1981.
[44] A. Smeulders and C. de Boer, “Design and Performance in Object Recognition,” Graphics Recognition—Algorithms and Systems, K. Tombre and A. Chhabra, eds., pp. 335-346, Springer Verlag, 1998.
[45] T. Strzalkowski, “Reversible Logic Grammars for Natural Language Parsing and Generation,” Canadian Computational Intelligence J., vol. 6, no 3, pp. 145-171, 1990.
[46] R. Tamassia, G. Di Battista, and C. Batini, “Automatic Graph Drawing and Readability of Diagrams,” IEEE Trans. Systems, Man, and Cybernetics, vol. 18, no. 1, pp. 61-79, 1988.
[47] E.R. Tufte, The Visual Display of Quantitative Information, Graphics Press, Cheshire, Conn., 1983, p. 111.
[48] W. Liu and D. Dori, “A Proposed Scheme for Performance Evaluation of Graphics/Text Separation Algorithms,” Graphics Recognition—Algorithms and Systems, K. Tombre and A.K. Chhabra, eds., pp. 359-371, 1998.

Index Terms:
Document image analysis, graphics recognition, diagram recognition, music notation.
Citation:
Dorothea Blostein, Lippold Haken, "Using Diagram Generation Software to Improve Diagram Recognition: A Case Study of Music Notation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 11, pp. 1121-1136, Nov. 1999, doi:10.1109/34.809106
Usage of this product signifies your acceptance of the Terms of Use.