This Article 
 Bibliographic References 
 Add to: 
Visualizing the Semantic Structure in Classical Music Works
January/February 2010 (vol. 16 no. 1)
pp. 161-173
Wing-Yi Chan, Hong Kong University of Science and Technology, Hong Kong
Huamin Qu, Hong Kong University of Science and Technology, Hong Kong
Wai-Ho Mak, Hong Kong University of Science and Technology, Hong Kong
A major obstacle in the appreciation of classical music is that extensive training is required to understand musical structure and compositional techniques toward comprehending the thoughts behind the musical work. In this paper, we propose an innovative visualization solution to reveal the semantic structure in classical orchestral works such that users can gain insights into musical structure and appreciate the beauty of music. We formulate the semantic structure into macrolevel layer interactions, microlevel theme variations, and macro-micro relationships between themes and layers to abstract the complicated construction of a musical composition. The visualization has been applied with success in understanding some classical music works as supported by highly promising user study results with the general audience and very positive feedback from music students and experts, demonstrating its effectiveness in conveying the sophistication and beauty of classical music to novice users with informative and intuitive displays.

[1] R. Bennett, Cambridge Assignments in Music Series. Cambridge Univ. Press, 1999-2001.
[2] C. Hoffer, Music Listening Today, third ed. Schirmer Books, 2006.
[3] M. Wattenberg, “Arc Diagrams: Visualizing Structure in Strings,” Proc. IEEE Symp. Information Visualization '02, pp. 110-116, 2002.
[4] T. Bergstrom, K. Karahalios, and J.C. Hart, “Isochords: Visualizing Structure in Music,” Proc. Int'l Conf. Graphics Interface, pp. 297-304, 2007.
[5] J. Snydal and M. Hearst, “ImproViz: Visual Explorations of Jazz Improvisations,” Proc. Conf. Extended Abstracts on Human Factors in Computing Systems, pp. 1805-1808, 2005.
[6] B. Evans, “The Graphic Design of Musical Structure: Scores for Listeners,” Proc. Electroacoustic Music Studies Conf., 2005.
[7] R. Hiraga, “Case Study: A Look of Performance Expression,” Proc. IEEE Conf. Visualization, pp. 501-504, 2002.
[8] R. Hiraga and N. Matsuda, “Graphical Expression of the Mood of Music,” Proc. Seventh IEEE Int'l Conf. Multimedia and Expo, pp.2035-2038, 2004.
[9] R. Hiraga, R. Mizaki, and I. Fujishiro, “Performance Visualization: A New Challenge to Music through Visualization,” Proc. ACM Int'l Conf. Multimedia, pp. 239-242, 2002.
[10] R. Miyazaki, I. Fujishiro, and R. Hiraga, “Exploring MIDI Data Sets,” Proc. ACM SIGGRAPH '03 Sketches & Applications, 2003.
[11] R. Miyazaki and I. Fujishiro, “comp-i: 3D Visualization of MIDI Data Set,” Proc. IEEE Conf. Visualization, 2002.
[12] J. Ox, “Two Performances in the 21st. Century Virtual Color Organ: GridJam and Im Januar am Nil,” Proc. Seventh Int'l Conf. Virtual Systems and Multimedia, p. 580, 2001.
[13] S.M. Smith and G.N. Williams, “A Visualization of Music,” Proc. IEEE Conf. Visualization '97, pp. 499-503, 1997.
[14] L. Valbom and A. Marcos, “An Immersive Musical Instrument Prototype,” IEEE Computer Graphics and Applications, vol. 27, no. 4, pp. 14-19, July/Aug. 2007.
[15] “Timesketch,” Electronic Courseware Systems, Inc., 2000.
[16] S. Malinowki, Music Animation Machine, 2001.
[17] M.M. Farbood, E. Pasztor, and K. Jennings, “Hyperscore: A Graphical Sketchpad for Novice Composers,” IEEE Computer Graphics and Applications, vol. 24, no. 1, pp. 50-54, Jan./Feb. 2004.
[18] Hyperscore, Harmony Line, 2006.
[19] C. Chen, Information Visualization: Beyond the Horizon, second ed. Springer, 2004.
[20] C. Ware, Information Visualization: Perception for Design, second ed. Morgan Kaufmann, 2004.
[21] W.S. Cleveland, The Elements of Graphing Data, second ed. Hobart Press, 1994.
[22] D. Borland and R.M. Taylor, “Rainbow Color Map (Still) Considered Harmful,” IEEE Computer Graphics and Applications, vol. 27, no. 2, pp. 14-17, Mar./Apr. 2007.
[23] “A Brief Introduction to Typography,” Dept. of Art, Western Illinois Univ., 2002.
[24] S. Havre, E. Hetzler, P. Whitney, and L. Nowell, “ThemeRiver: Visualizing Thematic Changes in Large Document Collections,” IEEE Trans. Visualization and Computer Graphics, vol. 8, no. 1, pp. 9-20, Jan.-Mar. 2002.
[25] H. Chernoff, “The Use of Faces to Represent Points in k-Dimensional Space Graphically,” J. Am. Statistical Assoc., vol. 68, no. 342, pp. 361-368, 1973.
[26] G. Kindlmann, “Superquadric Tensor Glyphs,” Proc. Joint Eurographics-IEEE TCVG Symp. Visualization, pp. 147-154, 2004.
[27] J.P. Lewis, R. Rosenholtz, N. Fong, and U. Neumann, “VisualIDs: Automatic Distinctive Icons for Desktop Interfaces,” Proc. ACM SIGGRAPH '04, pp. 416-423, 2004.
[28] P. Kolhoff, J. Preub, and J. Loviscach, “Music Icons: Procedural Glyphs for Audio Files,” Proc. 19th Brazilian Symp. Computer Graphics and Image Processing, pp. 289-296, 2006.
[29] M.D. Apperley, I. Tzavaras, and R. Spence, “A Bifocal Display Technique for Data Presentation,” Proc. Conf. Eurographics, pp. 27-43, 1982.
[30] S.K. Card, J.D. Mackinlay, and B. Shneiderman, Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann, 1999.
[31] J.D. Mackinlay, G.G. Robertson, and S.K. Card, “The Perspective Wall: Detail and Context Smoothly Integrated,” Proc. ACM Conf. Human Factors in Computing Systems, pp. 173-179, 1991.
[32] H. Abert and S. Spencer, W.A. Mozart, C. Eisen, ed. Yale Univ. Press, 2007.

Index Terms:
Classical music, macro-micro relationship, glyph, typography, threads, weaving, information visualization.
Wing-Yi Chan, Huamin Qu, Wai-Ho Mak, "Visualizing the Semantic Structure in Classical Music Works," IEEE Transactions on Visualization and Computer Graphics, vol. 16, no. 1, pp. 161-173, Jan.-Feb. 2010, doi:10.1109/TVCG.2009.63
Usage of this product signifies your acceptance of the Terms of Use.