This Article 
 Bibliographic References 
 Add to: 
July-September 2008 (vol. 1 no. 3)
pp. 145-156
Stephan Repp, Hasso-Plattner-Institut für Softwaresystemtechnik GmbH, Potsdam
Andreas Groβ, Hasso-Plattner-Institut für Softwaresystemtechnik GmbH, Potsdam
Christoph Meinel, Hasso-Plattner-Institut für Softwaresystemtechnik GmbH, Potsdam
The number of digital lecture video recordings has increased dramatically since recording technology became easier to use. The accessibility and ability to search within this large archive are limited and difficult. Additionally, detailed browsing in videos is not supported due to the lack of an explicit annotation. Manual annotation and segmentation is time-consuming and therefore useless. A promising approach is based on using the audio layer of a lecture recording to obtain information about the lecture's contents. In this paper we're going to present an indexing method for computer science courses based on their existing recorded videos. The transcriptions from a speech-recognition engine (SRE) are sufficient to create a chain index for detailed browsing inside a lecture video. The index structure and the evaluation of the supplied keywords are presented. The user interface for dynamic browsing of the e-learning contents concludes this paper.

[1] S. Linckels, S. Repp, N. Karam, and C. Meinel, “The Virtual Tele-Task Professor: Semantic Search in Recorded Lectures,” Proc. 38th SIGCSE Technical Symp. Computer Science Education (SIGCSE '07), pp. 50-54, 2007.
[2] C.-W. Ngo, F. Wang, and T.-C. Pong, “Structuring Lecture Videos for Distance Learning Applications,” Proc. Multimedia Software Eng., pp. 215-222, 2003.
[3] N. Yamamoto, J. Ogata, and Y. Ariki, “Topic Segmentation and Retrieval System for Lecture Videos Based on Spontaneous Speech Recognition,” Proc. Eighth European Conf. Speech Comm. and Technology, pp. 961-964, 2003.
[4] A. Haubold and J.R. Kender, “Augmented Segmentation and Visualization for Presentation Videos,” Proc. 13th ACM Int'l Conf. Multimedia, pp. 51-60, 2005.
[5] L. Tang and J.R. Kender, “Designing an Intelligent User Interface for Instructional Video Indexing and Browsing,” Proc. Int'l Conf. Intelligent User Interfaces (IUI '06), pp. 318-320, 2006.
[6] S. Repp and C. Meinel, “Segmenting of Recorded Lecture Videos - The Algorithm VoiceSeg,” Proc. Conf. Signal Processing and Multimedia Applications (SIGMAP '06), pp. 317-322, 2006.
[7] J. Glass, T.J. Hazen, L. Hetherington, and C. Wang, “Analysis and Processing of Lecture Audio Data: Preliminary Investigations,” Proc. Workshop Interdisciplinary Approaches to Speech Indexing and Retrieval (HLT-NAACL '04), pp. 9-12, 2004.
[8] W. Hürst, T. Kreuzer, and M. Wiesenhütter, “A Qualitative Study towards Using Large Vocabulary Automatic Speech Recognition to Index Recorded Presentations for Search and Access over the Web,” Proc. IADIS Int'l Conf. WWW/Internet (ICWI '02), pp. 135-143, 2002.
[9] G. Tür, A. Stolcke, D. Hakkani-Tür, and E. Shriberg, “Integrating Prosodic and Lexical Cues for Automatic Topic Segmentation,” Computational Linguistics, vol. 27, no. 1, pp. 31-57, 2001.
[10] G. Miller, “An On-Line Lexical Database,” Int'l J. Lexicography, vol. 3, no. 4, pp. 235-312, 1990.
[11] H. Nohr, Grundlagen der automatischen Indexierung. Logos Verlag, 2005.
[12] R.A. Baeza-Yates and B.A. Ribeiro-Neto, Modern Information Retrieval. ACM/Addison-Wesley, 1999.
[13] W.-T. Chu and H.-Y. Chen, “Cross-Media Correlation: A Case Study of Navigated Hypermedia Documents,” Proc. 10th ACM Int'l Cong. Multimedia (MULTIMEDIA '02), pp. 57-66, 2002.
[14] G.D. Abowd, C.G. Atkeson, J.A. Brotherton, T. Enqvist, P. Gulley, and J. LeMon, “Investigating the Capture, Integration and Access Problem of Ubiquitous Computing in an Educational Setting,” Proc. Conf. Human Factors in Computing Systems (SIGCHI '98), pp.440-447, 1998.
[15] S. Mukhopadhyay and B. Smith, “Passive Capture and Structuring of Lectures,” Proc. ACM Int'l Conf. Multimedia (Part 1), pp. 477-487, 1999.
[16] R. Müller and T. Ottmann, “The Authoring on the Fly System for Automated Recording and Replay of (Tele)presentations,” Multimedia System, vol. 8, no. 3, pp. 158-176, 2000.
[17] H. Ney and S. Ortmanns, “Progress in Dynamic Programming Search for lvcsr,” Proc. IEEE, vol. 88, no. 8, pp. 1224-1240, 1997.
[18] S. Linckels and C. Meinel, “Applications of Description Logics to Improve Multimedia Information Retrieval for Efficient Educational Tools,” Proc. ACM SIGMM Int'l Conf. Multimedia Information Retrieval (MIR '08), 2008.
[19] The Description Logic Handbook: Theory, Implementation, and Applications, F. Baader, D. Calvanese, D.L. McGuinness, D. Nardi, and P.F. Patel-Schneider, eds. Cambridge Univ. Press, 2003.
[20] W.W.W.C. W3C, OWL Web Ontology Language, http://www., 2004.
[21] R.A. Schmidt, “Terminological Representation, Natural Language & Relation Algebra,” Proc. 16th German Conf. Artificial Intelligence (GWAI '93), pp. 357-371, 1993.
[22] S. Linckels and C. Meinel, “Resolving Ambiguities in the Semantic Interpretation of Natural Language Questions,” Proc. Seventh Int'l Conf. Intelligent Data Eng. and Automated Learning (IDEAL '06), pp.612-619, Sept. 2006.
[23] E.M. Voorhees, “The TREC-8 Question Answering Track Report,” Proc. Text REtrieval Conf. (TREC-8), 1999.
[24] S. Repp, S. Linckels, and C. Meinel, “Question Answering from Lecture Videos Based on Automatically-Generated Learning Objects,” Proc. Seventh Int'l Conf. Web-Based Learning (ICWL '08), pp. 509-520, 2008.
[25] S. Linckels, C. Dording, and C. Meinel, “Better Results in Mathematics Lessons with a Virtual Personal Teacher,” Proc. 34th Ann. ACM SIGUCCS Conf. User Services, pp. 201-209, 2006.
[26] M.G. Noll and C. Meinel, “Web Search Personalization via Social Bookmarking and Tagging,” The Semantic Web, Proc. Sixth Int'l Semantic Web Conf., Second Asian Semantic Web Conf. (ISWC/ASWC '07), pp. 367-380, 2007.
[27] S. Golder and B.A. Huberman, “Usage Patterns of Collaborative Tagging Systems,” J. Information Science, vol. 32, no. 2, pp. 198-208, 2006.
[28] C. Marlow, M. Naaman, D. Boyd, and M. Davis, “Ht06, Tagging Paper, Taxonomy, Flickr, Academic Article, to Read,” Proc. 17th ACM Conf. Hypertext and Hypermedia (HHYPERTEXT '06), pp. 31-40, 2006.
[29] H. Sack and J. Waitelonis, “Integrating Social Tagging and Document Annotation for Content-Based Search in Multimedia Data,” Proc. First Semantic Authoring and Annotation Workshop (SAAW '06), 2006.
[30] W. Hürst, “Multimediale Informationssuche in Vortrags- und Vorlesungsaufzeichnungen,” PhD dissertation, Fakultät füur Angewandte Wissenschaften, Universität Freiburg, 2005.
[31] H. Sack and J. Waitelonis, “Automated Annotations of Synchronized Multimedia Presentations,” Proc. Workshop Mastering the Gap: From Information Extraction to Semantic Representation (ESWC'06), 2006.
[32] S. Repp, J. Waitelonis, H. Sack, and C. Meinel, “Segmentation and Annotation of Audiovisual Recordings Based on Automated Speech Recognition,” Proc. Eighth Int'l Conf. Intelligent Data Eng. and Automated Learning (IDEAL '07), pp. 620-629, 2007.
[33] M.A. Hearst, “Multi-Paragraph Segmentation of Expository Text,” Proc. 32nd Ann. Meeting Assoc. for Computational Linguistics, pp. 9-16, 1994.
[34] D. Beeferman, A. Berger, and J.D. Lafferty, “Statistical Models for Text Segmentation,” Machine Learning, vol. 34, nos. 1-3, pp. 177-210, 1999.
[35] M. Utiyama and H. Isahara, “A Statistical Model for Domain-Independent Text Segmentation,” Proc. Conf. 39th Ann. Meeting of the Assoc. for Computational Linguistic and 10th Conf. European Chapter, pp. 491-498, 2001.
[36] F.Y.Y. Choi, P. Wiemer-Hastings, and J. Moore, “Latent Semantic Analysis for Text Segmentation,” Proc. Conf. Empirical Methods on Natural Language Processing (EMNLP '01), pp. 109-117, 2001.
[37] S. Repp and C. Meinel, “Segmentation of Lecture Videos Based on Spontaneous Speech Recognition,” Proc. IEEE Int'l Symp. Multimedia Workshops (ISMW '08), pp. 692-697, 2008.
[38] M. Porter, “An Algorithm for Suffix Stripping,” Program, vol. 14, no. 3, pp. 130-137, 1980.
[39] F.Y.Y. Choi, “Advances in Domain Independent Linear Text Segmentation,” Proc. First Conf. North American Chapter of the Assoc. for Computational Linguistics, pp. 26-33, 2000.
[40] M. Galley, K. McKeown, E. Fosler-Lussier, and H. Jing, “Discourse Segmentation of Multi-Party Conversation,” Proc. 41st Ann. Meeting of the Assoc. for Computational Linguistics (ACL '03), pp.562-569, 2003.
[41] M.A. Hearst, “TextTiling: Segmenting Text into Multi-Paragraph Subtopic Passages,” Computational Linguistics, vol. 23, no. 1, pp. 33-64, 1997.
[42] M. Lin, J. Nunamaker, M. Chau, and H. Chen, “Segmentation of Lecture Videos Based on Text: A Method Combining Multiple Linguistic Features,” Proc. 37th Ann. Hawaii Int'l Conf. System Sciences (HICSS '04)—Track 1, p. 10003c, 2004.
[43] L. Pevzner and M.A. Hearst, “A Critique and Improvement of an Evaluation Metric for Text Segmentation,” Computational Linguistics, vol. 28, no. 1, pp. 19-36, 2002.
[44] I. Malioutov and R. Barzilay, “Minimum Cut Model for Spoken Lecture Segmentation,” Proc. 21st Int'l Conf. Computational Linguistics and 44th Ann. Meeting of the Assoc. for Computational Linguistics (ACL '06), pp. 25-32, 2006.
[45] A. Hauptmann and H. Wactlar, “Indexing and Search of Multimodal Information,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing (ICASSP '97), p. 195, 1997.
[46] M. Halliday and R. Hasan, Cohesion in English. Longman, 1976.
[47] J. Reynar, “Topic Segmentation: Algorithm and Applications,” PhD dissertation, Univ. of Pennsylvania, 1998.
[48] S. Repp and C. Meinel, “Semantic Indexing for Recorded Educational Lecture Videos,” Proc. Fourth IEEE Conf. Pervasive Computing and Comm. Workshops (PerCom '06), pp. 240-245, 2006.
[49] R.V. Cox, B.G. Haskell, Y. LeCun, B. Shahraray, and L. Rabiner, “On the Applications of Multimedia Processing to Telecommunications,” Proc. Int'l Conf. Image Processing (ICIP '97), vol. 1, pp. 5-8, 1997.
[50] M.J. Bates, “What Is Browsing–Really? A Model Drawing from Behavioural Science Research,” Information Research, vol. 12, no. 4, , Oct. 2007.
[51] S. Repp, A. Groß, and C. Meinel, “Dynamic Browsing of Audiovisual Lecture Recordings Based on Automated Speech Recognition,” Proc. Ninth Int'l Conf. Intelligent Tutoring Systems (ITS '08), pp. 662-664, 2008.
[52] S. Repp, A. Groß, and C. Meinel, “Webbasierte Suche in Vorlesungsvideos auf Basis der Transkripte eines Spracherkenners,” e-Learning Fachtagung Informatik (DeLFI), pp. 427-429, 2008.
[53] S.F. Chang, T. Sikora, and A. Puri, “Overview of the MPEG-7 Standard,” IEEE Trans. Circuits and Systems for Video Technology, vol. 11, no. 6, pp. 688-695, 2001.

Index Terms:
Distance learning, Web Search, Content Analysis and Indexing, Information Search and Retrieval, Digital Libraries, Image/video retrieval, Multimedia Information Systems, User Interfaces
Stephan Repp, Andreas Groβ, Christoph Meinel, "Browsing within Lecture Videos Based on the Chain Index of Speech Transcription," IEEE Transactions on Learning Technologies, vol. 1, no. 3, pp. 145-156, July-Sept. 2008, doi:10.1109/TLT.2008.22
Usage of this product signifies your acceptance of the Terms of Use.