The Community for Technology Leaders
RSS Icon
Issue No.02 - April-June (2013 vol.20)
pp: 52-60
Matthew Prockup , Drexel University
David Grunberg , Drexel University
Alex Hrybyk , Drexel University
Youngmoo E. Kim , Drexel University
Many people enjoy the symphony, but those without prior training often find it difficult to relate to the music. The authors have developed a system that guides listeners through orchestral performances in real time by presenting time-relevant annotations in a manner similar to that of a personal museum guide. These annotations are authored in partnership with musical experts prior to a performance to provide appropriate contextual information for a given concert program. Using acoustic features of the music, they align the live performance with that of a previously time-stamped recording. The aligned position is transmitted to an application on the users' handheld devices, which present the annotations using an intuitive and unobtrusive interface. To assess its utility, the system underwent a user beta testing stage accompanying orchestra concert broadcasts. It has since been adopted by the Philadelphia Orchestra for use during live concerts in its 2012–2013 subscription season and beyond.
Music, Information analysis, Performance evaluation, User interfaces, Acoustical engineering, dynamic time warping, multimedia, multimedia applications, score tracking, app development, music-information retrieval, music education
Matthew Prockup, David Grunberg, Alex Hrybyk, Youngmoo E. Kim, "Orchestral Performance Companion: Using Real-Time Audio to Score Alignment", IEEE MultiMedia, vol.20, no. 2, pp. 52-60, April-June 2013, doi:10.1109/MMUL.2013.26
1. M.A. Bartsch and G.H. Wakefield, "To Catch a Chorus: Using Chroma-Based Representations for Audio Thumbnailing," Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, IEEE CS, 2001, pp. 15–18.
2. D.P.W. Ellis and G.E. Poliner, "Identifying 'Cover Songs' with Chroma Features and Dynamic Programming Beat Tracking," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 7, IEEE CS, 2007, pp. IV-1429–1432.
3. J.H. Jensen et al., "A Tempo-Intensive Distance Measure for Cover Song Identification Based on Chroma Features," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, IEEE CS, 2009, pp. 2209–2212.
4. D.J. Berndt and J. Clifford, Using Dynamic Time Warping to Find Patterns in Time Series, tech. report WS-94-03, Assoc. Advancement of Artificial Intelligence, 1994.
5. R.B. Dannenberg, "An On-line Algorithm for Real-Time Accompaniment," Proc. Int'l Computer Music Conf., Computer Music Assoc., 1984, pp. 193–198.
6. R.B. Dannenberg and N. Hu, "Polyphonic Audio Matching for Score Following and Intelligent Audio Editors," Proc. Int'l Computer Music Conf., Computer Music Assoc., 2003, pp, 27–34.
7. J. Devaney and D.P.W. Ellis, "Handling Asynchrony in Audio-Score Alignment," Proc. Int'l Computer Music Conf., Computer Music Association, 2009, pp, 29–32.
8. S. Dixon, "Live Tracking of Musical Performances Using On-line Time Warping," Proc. 8th Int'l Conf. Digital Audio Effects, 2005, pp, 92–97.
9. N. Orio and D. Schwarz, "Alignment of Monophonic and Polyphonic Music to a Score," Proc. Int'l Computer Music Conf., Computer Music Assoc., 2001, pp. 129–132.
10. R. Macrae and S. Dixon, "Accurate Real-Time Windowed Time Warping," Proc. Int'l Soc. for Music Information Retrieval Conf., 2010, pp, 423–428.
11. E.J. Keogh and M.J. Pazzani, "Derivative Dynamic Time Warping," Proc. 1st SIAM Int'l Conf. Data Mining, SIAM, 2001, pp, 150–159.
31 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool